Well, an LM324 is a quad op amp, but it can be used as a comparator. Something like the LM311 is really more like it.
A comparator should make a very abrupt switch from high to low, or the opposite. The idea is to instantly signal when the voltage level has got to the setpoint.
Look at the slew rate of the opamp, and then look at the large signal performance.
Ideally a comparator output should be digital. It's is really hard to make the output of an opamp look like a logic signal.
Some of the newer comparators from OnSemi and others give you a choice of open collector or complementary outputs.
The LM311 is a sensitive beastie. I would suggest the LM393(dual) and the LM339(quad) or their single piece equivalents to get your feet wet. On the scope you will notice the rapid tranition of the open collector output when it goes low. The size of the pullup will determine the rise time in the other direction.
Some comparators are open collector where you need a pull up resistor to some voltage. This allows the comparator to operate at higher voltages than operating Vcc, but not higher than the devices ABS MAX rating on Vcc.
The LM311 is also has an open emitter to allow you to have a pull down resistor as well. If not, you can simply ground the emitter, and use a pull up on the collector output to Vcc.