diagnosing op-amp circuit malfunction

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
I'm looking for help understanding a bizarre circuit failure. The function of the circuit is simply to activate an output signal and light an LED if the voltage from a Hall Effect sensor goes low enough (indicating that a magnet got close enough to the sensor.) We've built hundreds (maybe thousands?) of this circuit with no problems at all except for this one board. I've attached two schematics - one with the complete circuit (two sensors, using both inputs on a two channel comparator) and one simplified schematic for just one side of the circuit (this board is failing only on channel 2):

hall-schematic_2.png hall-schematic_1.png

Normally the chip (Microchip Technology MCP6542-E/SN) operates as a comparator with crisp, clean switching and a small amount of deliberate hysteresis, but this one failing board outputs a variable voltage, inversely-proportional to the Hall Effect sensor voltage. It's as if the chip were running as a voltage-follower or amp of some sort instead of a comparator. The LED fades in and out as well, tracking with the output voltage. The sensitivity of the circuit also drifts slowly and gradually, tending towards making the circuit more and more sensitive until eventually the output is always on.

When the bad board was returned to us, I did a quick examination and couldn't find any obvious bad components, short circuits, or open circuits. Lacking any other explanation, I guessed that the chip itself was malfunctioning since it shouldn't output analog voltage when configured as a comparator. However, our board house replaced the chip with a new one, and the repaired board still behaves the same way, implying that something is wrong with the PCB itself or with one of the other components.

Is there any simple, plausible problem that could change this circuit and make it behave as a voltage follower instead of a comparator? Any short, open-circuit, or incorrect component value that would change this circuit so dramatically? I'm at a loss. I assume it can't be the Hall Effect sensor, simply because any analog voltage from the Hall Effect sensor should always result in a crisp on/off output from the comparator. My understanding of the myriad applications of op-amp circuits is still quite limited. I feel like there should be an obvious answer, but I'm at a loss.

Thanks in advance for any insights!

P.S. For this individual board, it would be cheaper to throw it away than to spend time diagnosing it, but I'd really like to figure this out in order to better understand these circuits and also so that we're prepared if similar issues come up again.
 

RichardO

Joined May 4, 2013
2,270
Some guesses:
1. The 51K feedback resistor is open, too high a value was installed or it has a bad solder joint. I don't think this is likely

2. The op-amp is not the correct part. This could be because your assembly house has mis-marked the reel/tube.

3. The resistor in series with the LED is too low a value. This is my preferred answer. I think that the value for this resistor is too low even under the best of conditions considering the limited output current rating of the op-amp. The particular batch of op-amps your assembly house happens to have could at the low end of the output current range allowed for the part.


I noticed something else. Because the two sections share the resistor divider for VREF their thresholds will interact. I doubt this is related to your problem but you should be aware of it.
 

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
Some guesses:
1. The 51K feedback resistor is open, too high a value was installed or it has a bad solder joint. I don't think this is likely

2. The op-amp is not the correct part. This could be because your assembly house has mis-marked the reel/tube.

3. The resistor in series with the LED is too low a value. This is my preferred answer. I think that the value for this resistor is too low even under the best of conditions considering the limited output current rating of the op-amp. The particular batch of op-amps your assembly house happens to have could at the low end of the output current range allowed for the part.


I noticed something else. Because the two sections share the resistor divider for VREF their thresholds will interact. I doubt this is related to your problem but you should be aware of it.
Thanks for your suggestions.
1) I tested resistance from pin 5 to pin 7 on the comparator, and I get 51k, so that connection appears to be good.
2) I verified part numbers on the original chip on this board and the replacement chip when it was sent back from repair. The writing on both chips is definitely right.
3) I'm curious about this one - the chip is rated for a max of 30mA, and the LED is only drawing 12mA. The way this board is used, one magnet is used to trigger either one sensor or the other, never both, so there's no way for both outputs to ever be active at once (barring malfunctions like this one.) Do you think 12mA is too much? (As a side note, I did change the BOM for our most recent production run, increasing from 249 to 340 ohm, reducing current to around 9mA, just so there would be more headroom even if both outputs did somehow go active at once.)

Regarding the last note about VREF interactions, you're totally right. I was embarrassed that I didn't realized that until many months after we went into production! The good news is that, since only one output is ever active at once, it's a non-issue in practice. Still, if I had realized from day-one, I probably would've provided separate VREFs just for good measure.
 

RichardO

Joined May 4, 2013
2,270
3) I'm curious about this one - the chip is rated for a max of 30mA, and the LED is only drawing 12mA. The way this board is used, one magnet is used to trigger either one sensor or the other, never both, so there's no way for both outputs to ever be active at once (barring malfunctions like this one.) Do you think 12mA is too much? (As a side note, I did change the BOM for our most recent production run, increasing from 249 to 340 ohm, reducing current to around 9mA, just so there would be more headroom even if both outputs did somehow go active at once.)

Regarding the last note about VREF interactions, you're totally right. I was embarrassed that I didn't realized that until many months after we went into production! The good news is that, since only one output is ever active at once, it's a non-issue in practice. Still, if I had realized from day-one, I probably would've provided separate VREFs just for good measure.
The 30ma spec is an "Absolute Maximum" number. Operation at this current will reduce the life of the chip.

This is what the data sheet says under "DC CHARACTERISTICS":
High-Level Output Voltage VOH VDD-0.2 — — V IOUT = -2 mA, VDD = 5V

So, they are saying that the output is not guaranteed to be more than 4.8 volts with only a 2ma load!
 

AlbertHall

Joined Jun 4, 2014
12,345
A possibility is that it is oscillating and your meter reads the average voltage. Can you check with an oscilloscope?
That might be because the 100n is duff or disconnected, or perhaps the reference isn't connected to the chip.
 

RichardO

Joined May 4, 2013
2,270
I had one other thought. Maybe there is some unexpected connection between the two halves of the circuit. Do you know if the circuit ever worked?
 

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
A possibility is that it is oscillating and your meter reads the average voltage. Can you check with an oscilloscope?
That might be because the 100n is duff or disconnected, or perhaps the reference isn't connected to the chip.
That's a good thought - haven't had a chance to try an oscilloscope on it, but I've tried simming every failure i can imagine, and the only ones that look like a proportional output are all really oscillating.
 

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
I had one other thought. Maybe there is some unexpected connection between the two halves of the circuit. Do you know if the circuit ever worked?
I'm not sure it ever worked perfectly, but it worked well enough to go unnoticed through three 4-hour power cycles.

It's an interesting failure in that, initially, full-off and full-on are both fine, and it's only the transition that's weird. The board receiving output signals from this has de-bouncing code, so ramping and/or oscillating output could easily go unnoticed.

What's really noticeable now is that the trip point drifts predictably over time until the output is stuck "on" all the time. This can take anywhere from 30 seconds to ten minutes or more, but it seems like it always eventually does it. We definitely would've noticed this part of the failure in our three test cycles, so this aspect is either new or worsening, but the ramping/oscillating might have been there all along.
 

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
I think I may have found the problem, although even after finding it, I don't really understand how or why it would cause the particular failure mode we're seeing...

When I check continuity from the Hall Effect sensor output to the comparator input, it's basically open circuit. I read some crazy-high mega-ohm resistance value, which I imagine could easily represent leakage through other parts of the circuit, not necessarily a high-resistance direct path through the PCB trace. When I get some more time, I'll try direct wiring those two pins together and see if that fixes it.
 

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
Well, I've done a fantastically ugly job of soldering a short wire between the two chips' pins (I'm not very good at hitting the tightly spaced pins on SMT chips) and the board appears to be working normally now. It's been going for several hours without issue. I'll keep an eye on it for the next few days and see if it keeps behaving normally, but I think that was probably it.

I suppose maybe some portion of the signal was getting through due to capacitive leakage, or simply conducting poorly through a very-high resistance connection. Not really sure.
 

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
Follow up:
After my tacked-on wire fixed things, I sent the board back to our assembly house to have them check it out, hoping to determine if there was a fault in the PCB or some other logical explanation. They removed my jumper wire... and found nothing wrong at all.

So now it seems like perhaps it was just a bad solder joint at the hall effect board, which was perhaps repaired as a side effect of the soldering I did to attach the jumper wire. It's a little frustrating if that's the case, as they supposedly checked all the joints when it went back for repair the first time, but I suppose I should just be glad that this is a very simple issue, and not some hard to understand problem that we'd have to watch for in the future.
 

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
I'm reviving this topic again because we now have a second board malfunctioning in a very similar way. The customer in the field reported one output being stuck on, which fits with the trip-point drift I experienced on the first board, but when the second one arrived here for testing, I couldn't get it to behave that badly. However, it does look like the indicator LED fades on and off gradually if you move the magnet slowly enough, which also matches the malfunction in the first board. I took the second board home and hooked up my oscilloscope, and it is definitely oscillating around the trip point quite a bit. I'm not super great with an oscilloscope, and the one I have is in terrible shape (bought it for $20 used, with giant cracks in it - amazed it works at all,) but I've got a little video and some screenshots. It looks like it oscillates at about 200-300kHz if I'm reading things right.

I've checked resistance/continuity readings everywhere I can think to check, verified part numbers, etc. and can't find anything wrong with the board, although I don't know how to test capacitors in circuit, so I could easily be missing any capacitor problems. Once I exhaust my abilities to troubleshoot this thing, I'll send it back to the assembly house for repair, but I'd love to understand the problem better than I currently do if at all possible. If you have any suggestions for what else to check, please let me know.

I've attached two oscilloscope pics, one of the proper circuit behavior, and the other showing the oscillation. Here's a link to video of the oscillation:

Thanks!
2017_0110_Ozo-Hall-Effect_SW1_a.png 2017_0110_Ozo-Hall-Effect_SW2.png
 

dannyf

Joined Sep 13, 2015
2,197
Here's a link to video of the oscillation:
tough to see what's going on. I would take the schematic you have, apply a variety of different voltage on the hall sensor input (by moving the magnet vs. the hall sensor), measure the voltage on the opamp input / output pins and mark them on the schematic so others can see. The interesting point would be around the values that flip the opamp's output, both on the way up and on the way down.
 

ronv

Joined Nov 12, 2008
3,770
I'm reviving this topic again because we now have a second board malfunctioning in a very similar way. The customer in the field reported one output being stuck on, which fits with the trip-point drift I experienced on the first board, but when the second one arrived here for testing, I couldn't get it to behave that badly. However, it does look like the indicator LED fades on and off gradually if you move the magnet slowly enough, which also matches the malfunction in the first board. I took the second board home and hooked up my oscilloscope, and it is definitely oscillating around the trip point quite a bit. I'm not super great with an oscilloscope, and the one I have is in terrible shape (bought it for $20 used, with giant cracks in it - amazed it works at all,) but I've got a little video and some screenshots. It looks like it oscillates at about 200-300kHz if I'm reading things right.

I've checked resistance/continuity readings everywhere I can think to check, verified part numbers, etc. and can't find anything wrong with the board, although I don't know how to test capacitors in circuit, so I could easily be missing any capacitor problems. Once I exhaust my abilities to troubleshoot this thing, I'll send it back to the assembly house for repair, but I'd love to understand the problem better than I currently do if at all possible. If you have any suggestions for what else to check, please let me know.

I've attached two oscilloscope pics, one of the proper circuit behavior, and the other showing the oscillation. Here's a link to video of the oscillation:

Thanks!
View attachment 118612 View attachment 118613
Keep scoping.....
Have a look at the input while it's chattering. Without knowing about the rest of the circuits it could be the input from the halls move due to a change in their voltage. Or maybe the interaction of the threshold. A very small change in the inputs will make it switch.
Edit:
You could make the 51K smaller to see if that fixes it.
 

RichardO

Joined May 4, 2013
2,270
Wow, is that a thing that happens?! That never would've occurred to me.
I have never seen an op-amp with input pins swapped but as a test technician I saw a lot of 1N4148 diodes that were marked backwards.

I once had the bad experience with a batch of op-amps that were not properly passivated. They would last just long enough to be shipped and fail for the customer.
 

Thread Starter

ebeowulf17

Joined Aug 12, 2014
3,307
Thanks for all the great advice. Long story, but couldn't get the oscilloscope on this circuit again. However, I think I have some interesting news just from some DMM work. I measured voltages at several key points while moving a magnet around each sensor.

Hall-Schematic_Ozo-Failure.png

The circuit is designed with a VREF threshold voltage of around 0.81VDC (0.76 to 0.86 depending on voltage of comparator outputs, with this swing setting hysteresis.) The Hall Effect Sensors swing from roughly 0-5VDC, and are ~2.5VDC if no magnetic field is present. So, as a magnet approaches, assuming the poles are facing the right direction, the Hall Sensor output gradually drops, and as it passes through 0.76V, that output of the comparator goes high, pulling the VREF up to 0.86V. The output then stays high until the magnet moves far enough away for the Hall Sensor output to climb past 0.86V.

All of this intended behavior is working perfectly on the U1/U3A side of things, but the U2/U3B side is crazy. Over there, my VREF still reads correctly, even at the comparator itself, and the Hall Sensor (U2) output readings perfectly match the comparator input (U3 VINB- at pin 6) readings, so I don't think I have any continuity problems, nor any wrong resistors for hysteresis etc. The weird thing is that the trip point on this side of the circuit is ~1.7V, around 0.9V higher than it should be. Every voltage I can check is normal, except that the comparator output B (pin 7) is going high when the inverting input is at 1.7V and the non-inverting input is at 0.76V.

Is this simply a bad comparator? Is that something that is at all likely? I can't find anything else wrong with the circuit. Let me know if there are other things I can check with a DMM that might illuminate the situation. This is really bizarre to me!

Thanks for all your help!
 

AlbertHall

Joined Jun 4, 2014
12,345
Long story, but couldn't get the oscilloscope on this circuit again. However, I think I have some interesting news just from some DMM work. I measured voltages at several key points while moving a magnet around each sensor.
If the circuit is oscillating your DMM readings are suspect. Try your DMM on AC volts. Any reading means the oscillation is still there (but no reading might just mean that the frequency is above what the meter can cope with).
 

#12

Joined Nov 30, 2010
18,224
If the circuit is oscillating your DMM readings are suspect. Try your DMM on AC volts. Any reading means the oscillation is still there (but no reading might just mean that the frequency is above what the meter can cope with).
I would like to volunteer that the act of measuring can cause or quench the oscillation. I have seen both of these in real life, and it was always related to the high gain of the op-amp. A few picofarads from the scope probe can quench oscillation by decreasing the bandwidth of the frequency response at the non-inverting input. I can't remember the schematic, but I had one that would burst into oscillation when measured with a scope probe because it was mis-wired. So, caution be to the trouble shooter. You can't know everything about the circuit until you understand it, and then prove the physical reality matches the schematic.
 
Top