Well, carry on. Let us know how you come out.

 

Got an ohmmeter? Next time, every time a unit blows make these measurements:

With the ohmmeter in normal (low voltage) mode, read each pin to Vdd, then repeat each pin to Vss. All should read open.

With the ohmmeter is diode mode (higher voltage) read each pin + to Vdd -, then again pin - to Vss +. In every case you should read a diode.

If a pin fails the first group of tests you have a shorted pin.

If a pin fails the second group of tests you have an open pin.

And fer chissake, just add the friggin bypass caps already.

 

Got an ohmmeter? Next time, every time a unit blows make these measurements:

With the ohmmeter in normal (low voltage) mode, read each pin to Vdd, then repeat each pin to Vss. All should read open.

With the ohmmeter is diode mode (higher voltage) read each pin + to Vdd -, then again pin - to Vss +. In every case you should read a diode.

If a pin fails the first group of tests you have a shorted pin.

If a pin fails the second group of tests you have an open pin.

I've measure some of the bad chips. I haven't found an open or short yet. I must reiterate that when I get the board back to the bench it works fine. I think this is being overlooked. What type of failure is it when is fails 50% of the time in the machine then passes 100% of the times when its removed and tested on the bench?

And fer chissake, just add the friggin bypass caps already.

It was one of the first things that crossed my mind and everyone elses mind here. I soldered a cap to the back of the board, a ceramic 1uF. The board still failed.

 

how about showing what the external circuit looks like?

 

Where does J3 go? It seems to be the only place that is not isolated by an opto.

 

Where does J3 go? It seems to be the only place that is not isolated by an opto.

J3 goes straight through to J2 on the next board/battery with a cable. There it is isolated by optocouplers. The top pin on J2 goes to the top pin on J3. The second to the top pin on j2 goes to the second to the top pin on J3 ... etc. I've put a hipot tester to J2 and connector J1 and J3 on the board. It never breaks down.

 

how about showing what the external circuit looks like?

Its a huge board. Its not necessarily the point of failure, either. The last problem I had was on a board almost right in the middle of the string. It was board number 12 of 20. There was one of the boards I've shown on the right and left of it. Each board was connected to another battery. I don't think the external board can cause a problem, because it wasn't connected. The board I've shown fails when connect to the other boards that follow the same layout, schematic, BOM.

 

What type of failure is it when is fails 50% of the time in the machine then passes 100% of the times when its removed and tested on the bench?

Obviously the bench is not a proper simulation of the system.

Do units that fail in the machine then pass on the bench?

 

Obviously the bench is not a proper simulation of the system.

I guess not. Some differences are the input into J2 shares a ground with the board/battery, which would not be true in the machine, but those are fed into optocouplers, so it shouldn't matter. The 12V supply has more impedance behind it, since it isn't a car battery. The big difference that comes to mind is that in the system the board can be hundreds of volts below earth ground. This is the most obvious difference, but I cannot think of why it would cause this failure.

Do units that fail in the machine then pass on the bench?

Yes.

 

Sure sounds like latch up. The only other thing I can think of is maybe the inverter doesn't have ground or 15 volts to the ground or Vcc pin. Sometimes it's possible to run thru a substrate diode form an input.
Since everything is isolated by an opto the battery shouldn't make a difference. How does the signal finally get of the boards to where you can see a failure?

 

Sure sounds like latch up.

I'm interested in this phenomena. The two articles I have read have given conflicting causes of latchup. What could cause latch-up in your opinion?

The only other thing I can think of is maybe the inverter doesn't have ground or 15 volts to the ground or Vcc pin. Sometimes it's possible to run thru a substrate diode form an input.

The grounds and vcc have been confirmed on the layout and I got a board and toned it out. The ICs are connected to the battery terminals.

Since everything is isolated by an opto the battery shouldn't make a difference. How does the signal finally get of the boards to where you can see a failure?

The signals go through maybe a dozen other boards then to the monitoring board.

 

Does this happen right when you hook up the system or does it need to run for a while. Sometimes "hot plugging" can cause it.

Your outputs may be susceptible due to the long runs.
Here is a common sense app note:

http://ww1.microchip.com/downloads/en/AppNotes/00763b.pdf

 

I've found the problem, at least "a" problem. Its confounding me can someone explain it?

I've looked at the supply like a billion times even with caps on the chips. I'm still claiming omitting bypass caps is not the problem. The supply is constant.

What I have found is that there is jitter on the clock. This is causing extra spurious clock pulses near the rising and falling.

The perplexing part is that the cause of the clock problems is noise from the optocopler, but its not always noisy. The noise only exists when the battery stack is connected to the inverter.

The supply is NOT noisy when the inverter is attached. The supply and rails are indistinguishable, when the inverter is connected or not, but noise is coming out of the optocopler. There is no galvanic connection from the optocopler to the inverter.

Our product is a ups.

Everything else is exactly the same when the battery breaker is open or closed. I cannot fathom how the inverter noise is induced into the optocopler. The inverter is always on. But when the battery stack is connecter to the inverter the noise appears.

 

I didn't read through the entire thread, but you are confusing batteries and capacitors. Local caps, located right next to the chips, suppress noise before they enter the chips or exit the chips. A battery will not do that.

Logic and comparators (actually the same thing) are very fast. They will pick up on pulses you may not see on a scope.

I have found bypass caps to be beneficial on small circuits with low draw, where you really think it doesn't matter. It can and does.

It is also possible you have RF noise entering the circuit. I was calibrating a power supply and noticed a pesky oscillation. To make a long story short it was the local AM station, they can be sneaky.

 

Per Bill, add decoupling caps directly between every IC power pin and ground. Then tell us if you still have a problem. Just saying that a lack of decoupling capacitors is not a problem doesn't mean that it's not a problem.

 

I put a bypass cap on the ICS it did nothing.

I am not confused by capacitors and batteries.

I have made a board that monitors a battery in a battery stack that is connected to an inverter through a circuit breaker.

I'll reiterate I put in a bypass capacitor. It did nothing. Somehow when the circuit breaker between the batteries and the inverter is closed the noise comes out of the optocopler.
The board members s completely isolated through optocouplers and transformers from the batteries and the inverter. No radiated noise is present to n the supply or anywhere else.

I put in bypass capacitors

It did nothing.

 

It is good house keeping to include bypass capacitors, but you need understand why they are unnecessary here.

1. The ripple on the supply is 10mV from the charger
2. The battery is a 40F capacitor with only .005ohms source impedance and is the source
3. the logic transition @5Hz

there is little reason to put capacitors where there is no AC and the source is already so stiff(a car battery).

If I put a cap on the board I'd have 40F(battery) + .00001F(electrolytic) = why even bother?

It's false economy to omit bypass caps in a circuit like this.
As others have said, it's the impedance at high frequencies that counts, even at slow clock frequencies, the pulse edges contain very high frequency spectra.
A board full of flip flops is aching to misbehave without bypass caps.

 

It is good house keeping to include bypass capacitors, but you need understand why they are unnecessary here.

1. The ripple on the supply is 10mV from the charger
2. The battery is a 40F capacitor with only .005ohms source impedance and is the source
3. the logic transition @5Hz

there is little reason to put capacitors where there is no AC and the source is already so stiff(a car battery).

If I put a cap on the board I'd have 40F(battery) + .00001F(electrolytic) = why even bother?

You need a bypass capacitor. You see your ICs are failing right?
You need a Z-Diode too with 18 volts.

 

I put a bypass cap on the ICS it did nothing.
......................................

So you put 0.1uF ceramic bypass capacitors from every IC power pin directly to ground with as short a lead as possible?

 

I put a bypass cap on the ICS it did nothing.

I am not confused by capacitors and batteries.

I have made a board that monitors a battery in a battery stack that is connected to an inverter through a circuit breaker.

I'll reiterate I put in a bypass capacitor. It did nothing. Somehow when the circuit breaker between the batteries and the inverter is closed the noise comes out of the optocopler.
The board members s completely isolated through optocouplers and transformers from the batteries and the inverter. No radiated noise is present to n the supply or anywhere else.

I put in bypass capacitors

It did nothing.

I have experienced problems with opto-couplers in the past when I wrongly assumed that they truly isolated things perfectly.
There is always some capacitance between input and output in any isolator, this combined with a 10K pullup and a slow 4N35 coupler means that the signal spends a long time in a vulnerable meta state where this minute capacitance can couple enough signal in to cause multiple transitions.

A lower value pull up would help.
A faster coupler would help.
A lower capacitance coupler would help.
A schmitt trigger input would help.