What do you do when faced with an intermittent problem? (PIC SPI)

sparky 1

Joined Nov 3, 2018
757
I tryed to find HV events because of the interest in power.
found a video on automotive secondary ignition with intermittent issues , what it looks like on a pico scope as we push run stop and zoom in.
It is extreme for I2C overshoot ringing. However, sometimes extreme is useful, at HV transient effects are wild. Can very small carbon tracks be present on a compromised pcb board ? and why cleaning only helps temporarily like misfires inrush and impulse are causing instability and dielectric break down. When we know what the timing should look like then the intermittent usually that which is abnormal.
After removing small intestine the impreza sounds and handles much better.
 
Last edited:

Sensacell

Joined Jun 19, 2012
3,447
Some thoughts on intermittent problems:

1) Relax and become hyper-observant, look at the "big picture."
2) Confirm the obvious - discard all assumptions like "it cannot be the power supply" re-check everything.
3) Divide and Conquer - separate the system into blocks - test and observe them independently, simplify the system until the failure goes away.
4) Create software "traps" that immediately halt the system and set a spare IO pin high- use this to trigger your scope so you can capture exactly what happened just before the failure.
 

Thread Starter

camerart

Joined Feb 25, 2013
3,730
i like your .03! It is an eloquent explanation of my earlier point. The important takeaway here is that coding a system doesn’t stop at the system. A good test suite is also necessary, particularly in an application as complex as this one.

@camerart If this is beyond your current capabilities, I propose that your system may also be beyond your capabilities. You are learning that coding is more than cut and paste of someone else’s code. Hopefully, you can continue to learn and have success in the next step of creating a complex system by being able to code a test suite for all of its discrete functions.

I’ve watched several of your posts. And I don’t believe success is beyond your reach. With help from @John P , you have made incredible progress. You just have a little more to learn and a little more to do.
Hi D,
If you've read some of my past posts, you will know, that I will never be able to learn how to CODE, and have had a lot of help over years, from JT and many others.

I do you copy and paste, which is my best method of of not making dyslexic mistakes, and understand that it can be dangerous, especially as I have sometimes copied an error or two. I try not do do this on finer point like this one.

I kepp, what I call 'modules' as examples of different sections of CODE for any device.

C.
 

Thread Starter

camerart

Joined Feb 25, 2013
3,730
I tryed to find HV events because of the interest in power.
found a video on automotive secondary ignition with intermittent issues , what it looks like on a pico scope as we push run stop and zoom in.
It is extreme for I2C overshoot ringing. However, sometimes extreme is useful, at HV transient effects are wild. Can very small carbon tracks be present on a compromised pcb board ? and why cleaning only helps temporarily like misfires inrush and impulse are causing instability and dielectric break down. When we know what the timing should look like then the intermittent usually that which is abnormal.
After removing small intestine the impreza sounds and handles much better.
Hi S,
I can see from my scope readings, that this problem is not like the example, as the OSC line sometimes stays HIGH or LOW as the error occures. I have two of the 4x peripherals where the MISO output is a low voltage and both error.
C
 

Thread Starter

camerart

Joined Feb 25, 2013
3,730
Some thoughts on intermittent problems:

1) Relax and become hyper-observant, look at the "big picture."
2) Confirm the obvious - discard all assumptions like "it cannot be the power supply" re-check everything.
3) Divide and Conquer - separate the system into blocks - test and observe them independently, simplify the system until the failure goes away.
4) Create software "traps" that immediately halt the system and set a spare IO pin high- use this to trigger your scope so you can capture exactly what happened just before the failure.
Hi S2,
I'll bear in mind "traps", thanks.
C.
 

Thread Starter

camerart

Joined Feb 25, 2013
3,730
i like your .03! It is an eloquent explanation of my earlier point. The important takeaway here is that coding a system doesn’t stop at the system. A good test suite is also necessary, particularly in an application as complex as this one.

@camerart If this is beyond your current capabilities, I propose that your system may also be beyond your capabilities. You are learning that coding is more than cut and paste of someone else’s code. Hopefully, you can continue to learn and have success in the next step of creating a complex system by being able to code a test suite for all of its discrete functions.

I’ve watched several of your posts. And I don’t believe success is beyond your reach. With help from @John P , you have made incredible progress. You just have a little more to learn and a little more to do.
Hi D,
This project has been running for a few years now, and almost from the start it 'is beyoned my capabilities' and is sometimes quite painful, but I started so I'll finish o_O
C.
 

Thread Starter

camerart

Joined Feb 25, 2013
3,730
Good suggestions above.

As you might guess from our earlier work I like to split up things into functional modules and in this case an SPI transceiver would be one of those. The issue could be hardware or more likely software so if it were me I'd test along these lines:
Split off whatever you have going on and write a test suite that exercises the SPI and flags any errors. From the master write 0-255 and have the slave echo +1 whatever it gets back to the master.
The master writes 00h to get things started and ignores the return value.
The slave receives 00h, increments it and puts that value in SSPBUF to send as the return value for the next byte Tx/Rx.
The master writes 01h to SPI and receives the 01h from the slave.
Master compares the values and if OK, continues with 02h etc. Wrap around at FFh-00H and keep going forever.
If there ever is a mismatch flag the error. I like to have the scope set up on the data lines and post-triggered in the one-shot mode. That way, I can scroll back through the scope's memory to see what the signals were that caused the problem.

When both PICs are paying full-time attention to the SPI, any issues will likely be in hardware. If you can run the sequence for a few hours without fail, you probably can assume your hardware is OK but the software isn't keeping up.

When you have this little test code working, archive it. Then whenever you encounter future problems you can reload the test code, run it and re-validate the hardware link.

Once the hardware is solid and testable (and repeatable) then you know whatever your current problem is is likely the code.... Be sure to check the SPI mode error flags.

Do remember that ANY communication between devices means that the firmware of the receiver has to be responsive enough to catch any character sent as fast as it is sent If the receiver is off doing painfully slow OSH string manipulations when multiple characters come in, you're in trouble. In master-slave SPI you should be able to send a character and wait for the character back as a single operation but that will take time away from other things you are doing..

Just my .03
Hi JT,
The present CODE from 'E' seems reliable, in that he tested it in a simulation mode, and I test it on 2x PCBS (for comparison). If it starts running properly, it stays like that and if it strats with errors it also stays like that. I haven't tried it for hours though.

I presume OSH is Oshonsoft? and the RECEIVE is only running this short CODE in HW.

The CODE seems very similar to your suggestion. Here is an example of the result:

I have seen reports that the SLAVE should run fater thatn the MASTER. The master XTL is 8MHz the SLAVE XTL is 8MHz x PLL 4 = 32MHZ. The SLAVE SPI is set to switch from the SS PIN.

'[00]Master Ready=1234' is show at switch ON. Where the errors occured (Note they are all the same) is when I added another UNSELECTED by CS peripheral, then removed it. In this case it errored, but sometime this will happen without doing anything and sometimes it doesn't error even when another peripheral is added.
C.
 

Attachments

JohnInTX

Joined Jun 26, 2012
4,787
Sorry to be redundant but if you are still having problems,
The present CODE from 'E' seems reliable, in that he tested it in a simulation mode, and I test it on 2x PCBS (for comparison). If it starts running properly, it stays like that and if it strats with errors it also stays like that. I haven't tried it for hours though.
The traces in Error.jpg above looks like more than one slave is selected and arguing on MISO. I noted that at least one CS/ in the test code in the other thread used PORTA instead of LATA:
Code:
Symbol ss = PORTA.5
Using PORT instead of LAT for CS/ can cause r-m-w problems on the hardware that won't show up on the simulator. The scope shot and intermittent nature of the problem is consistent with that thesis.
Of course, you could also be inadvertently selecting multiple slaves via bugs in your code, too.
I know you're working with other members and I don't want to cause confusion so take it for what it's worth and disregard if you've already visited all of that.
Good luck!
 
Last edited:

Thread Starter

camerart

Joined Feb 25, 2013
3,730
Hi,
Here's what the OSC looks like when erro and working ok.
C
Sorry to be redundant but if you are still having problems,
The traces in Error.jpg above looks like more than one slave is selected and arguing on MISO. I noted that at least one CS/ in the test code in the other thread used PORTA instead of LATA:
Code:
Symbol ss = PORTA.5
Using PORT instead of LAT for CS/ can cause r-m-w problems on the hardware that won't show up on the simulator. The scope shot and intermittent nature of the problem is consistent with that thesis.
Of course, you could also be inadvertently selecting multiple slaves via bugs in your code, too.
I know you're working with other members and I don't want to cause confusion so take it for what it's worth and disregard if you've already visited all of that.
Good luck!
Hi J,
I'm going over the whole thing, while checking and changing.

I had previously received messages regarding PORT LAT, and changed them, then forgot.

The 'error' image sometimes errors when no other peripherals are connected.

I hope other members won't be affended, if you correct their points, and take them as group learning, but for me your comments are welcome, either as a verification or reminder, as todays.

Regarding the 4431 SLAVE! As this routine is now HW, should I use PIN 44 as SS along with the other designated SPI PINs or does it matter?

Cheers, C.
 

BobaMosfet

Joined Jul 1, 2009
2,113
@camerart There is something else to consider as well- See if there is a ham-radio or electronics group in your area- usually you can find very knowledgable folks there that would be more than happy to help you using tools and skills you may not possess. For example, they can put an oscilloscope on your signals and look at things in a way you may not be able to, to help you.

Generally speaking, those of us in electronics, LOVE using our toys to help other people with circuits because we can show you so many cool things :)
 

Thread Starter

camerart

Joined Feb 25, 2013
3,730
@camerart There is something else to consider as well- See if there is a ham-radio or electronics group in your area- usually you can find very knowledgable folks there that would be more than happy to help you using tools and skills you may not possess. For example, they can put an oscilloscope on your signals and look at things in a way you may not be able to, to help you.

Generally speaking, those of us in electronics, LOVE using our toys to help other people with circuits because we can show you so many cool things :)
Hi B,
I'm a radio HAM! All of my group have heard my sagas, and a couple of them are able to help, a little. I get much more detailed help here.

I have an oscilloscope, as previous posts.
C.
 

BobaMosfet

Joined Jul 1, 2009
2,113
Hi,
This is a general question, not a specific one.

EDIT: Later answers have made it more specific to CODE and SPI, so I've changed the title.

I'm using SPI on PICs with peripherals, and have asked questions on this and other forums, and received lots of help, but my problem persists.

Sometimes my PCBs work fine, but for 'no' reason they error, so what do you do in these cases? I'm sure it all depends on what area the error(s) are in, so maybe this is an impossible question, but there it is.

Any ideas, please?
Camerart.
Why don't you post your schematic and/or PCB photo so we can see what the SPI physical looks like, compared to the schematic? SPI is useful to 2MHz (as I last remember), but has no lower-speed limit. If your traces are too close, or done in such away as they have a capactive or inductive crosstalk, you can get intermittent issues. The _easiest_ way to test this, is slow your SPI communications down and see if the problem goes away. If it does, you know you need to redesign your SPI layout on your PCB to better manage this problem.
 

Thread Starter

camerart

Joined Feb 25, 2013
3,730
Why don't you post your schematic and/or PCB photo so we can see what the SPI physical looks like, compared to the schematic? SPI is useful to 2MHz (as I last remember), but has no lower-speed limit. If your traces are too close, or done in such away as they have a capactive or inductive crosstalk, you can get intermittent issues. The _easiest_ way to test this, is slow your SPI communications down and see if the problem goes away. If it does, you know you need to redesign your SPI layout on your PCB to better manage this problem.
Hi B,
For some reason, I contantly get mixed up with Hardware and Software. I was talking to someone today, that pointed out that the CODE here is actually Firmware. WHAT!!

Following the above and other suggestions, I need to go through it all again, and check the CODE and carefully check e,g PORT/LAT etc. Once finished I'll post it all.

C.
 

JohnInTX

Joined Jun 26, 2012
4,787
For your purposes you can use the terms Firmware and Software interchangeably.

Regarding the 4431 SLAVE! As this routine is now HW, should I use PIN 44 as SS along with the other designated SPI PINs or does it matter?
On the schematic I currently have, you don't have the SPI hooked up to the 4431. Pin 44 is used as the UART TX line. Post the latest schematic?

But yes, if you have the SPI hardware hooked up now, and the 4431 will be an SPI slave then yes, it matters. You need SS/ to keep the 4431 SPI from driving MISO (SDO, pin 44 on the chip) on the SPI bus when the SPI master is talking to another SPI device. You also have to set SSPCON SSPM<3:0> to 0100 and set RC6 as an input to enable SS/. Not doing so can cause the bus contention shown in ERROR.jpg above.

Note that you can't use the hardware SPI and the hardware UART at the same time on this chip since the TX/RX pins for the UART are shared with the SPI bus.

Also, it looks like the master should NOT use SPICSon/SPICSoff if you are using multiple SPI devices on the same bus. Those statements affect the pin you define using SPI_CS_REG and SPI_CS_BIT. If you have the REG/BIT defined, use SPICSon AND manually select another CS/ pin (as in the last old code I have) you will get more than one SPI device selected as active and the two selected devices will argue on the bus like in the ERROR.jpg pic. As it happens, in the code I have, you do both BUT there is no definition for the CS REG/BIT so the compiler ignores SPICSon/off. Still something to keep in mind. Whatever you are doing, it still looks like more than one SPI slave is on the bus at times.

Hope I am remembering all of this correctly..
 
Last edited:

Thread Starter

camerart

Joined Feb 25, 2013
3,730
For your purposes you can use the terms Firmware and Software interchangeably.

On the schematic I currently have, you don't have the SPI hooked up to the 4431. Pin 44 is used as the UART TX line. Post the latest schematic?

But yes, if you have the SPI hardware hooked up now, and the 4431 will be an SPI slave then yes, it matters. You need SS/ to keep the 4431 SPI from driving MISO (SDO, pin 44 on the chip) on the SPI bus when the SPI master is talking to another SPI device. You also have to set SSPCON SSPM<3:0> to 0100 and set RC6 as an input to enable SS/. Not doing so can cause the bus contention shown in ERROR.jpg above.

Note that you can't use the hardware SPI and the hardware UART at the same time on this chip since the TX/RX pins for the UART are shared with the SPI bus.

Also, it looks like the master should NOT use SPICSon/SPICSoff if you are using multiple SPI devices on the same bus. Those statements affect the pin you define using SPI_CS_REG and SPI_CS_BIT. If you have the REG/BIT defined, use SPICSon AND manually select another CS/ pin (as in the last old code I have) you will get more than one SPI device selected as active and the two selected devices will argue on the bus like in the ERROR.jpg pic. As it happens, in the code I have, you do both BUT there is no definition for the CS REG/BIT so the compiler ignores SPICSon/off. Still something to keep in mind. Whatever you are doing, it still looks like more than one SPI slave is on the bus at times.

Hope I am remembering all of this correctly..
Hi JT,
Ok, from now on I'll call this type of program, FW (Firmware) which for me will punctuate this date.

I have hooked up the (As D/S) SPI PINs see PCB5. I have also added an 8MHz XTL xPLL = 32MHz, to the SLAVE, as I read, it needs to be faster than it's MASTER.

Re- HW SPI and UART, as seen, I changed to another PIN.

Re- HW PINs vs FW PINs. If all written in FW, then which PINs must be used, or can any be used?

Re- SSPCON SSPM, can you clarify, MASTER or SLAVE?

Re- SPICSon/SPICSoff, this is now FW so Oshonsoft (Edit: Oshonsoft SPI) is not used here.

If we can start again from here, it would be helpful, as there are a lot of comments rattling round, which is confusing.

Here are 2x CODEs MASTER and SLAVE, hopefully correct, but check, also todays result and PCB5.

EDIT: PCB SCH and programs removed as they are being updated.

C.
 
Last edited:

JohnInTX

Joined Jun 26, 2012
4,787
Ok, from now on I'll call this type of program, FW (Firmware) which for me will punctuate this date.

I have hooked up the (As D/S) SPI PINs see PCB5. I have also added an 8MHz XTL xPLL = 32MHz, to the SLAVE, as I read, it needs to be faster than it's MASTER.

You picked up the SPI bus OK and hooked SCK, MOSI, MISO to the correct pins for the hardware MSSP SPI peripheral on the 4431 slave BUT, you can't use the same SS/ (CS/). Each SPI slave must have its own CS/ line or you will get bus contention due to multiple slaves driving MISO.

Re- HW SPI and UART, as seen, I changed to another PIN.
Yeah, it's a shame that SS/ on the 4431 is not relocatable to another pin so that you could use both the hardware UART and MSSP SPI peripherals at the same time. If you must use the SPI for a bi-directional data link you might consider using SPI to replace the UART data, just poll it when you want it. The alternative is the firmware UART as you indicate. As long as you are TX only and not using an interrupt driven transmit buffer, either way consumes lots of CPU time when it transmits.

Re- HW PINs vs FW PINs. If all written in FW, then which PINs must be used, or can any be used?
Some of the hardware pins can be relocated on the 4431 i.e. SCK and MOSI but that doesn't help since the SS/ and TX pins are co-located and not changable. If I understand the OSH manual, you can use the firmware bit-banged UART/SPI functions on almost any pin that you specify.

Re- SSPCON SSPM, can you clarify, MASTER or SLAVE?
I am working on the notion that the 4431 is the SPI slave i.e. the 4620 initiates all SPI comms and drives SCK. That makes the 4431 an SPI slave and since there are multiple SPI slaves on the bus, requires that you use a unique SS/ to select the 4431 slave for comms.

Re- SPICSon/SPICSoff, this is now FW so Oshonsoft (Edit: Oshonsoft SPI) is not used here.
Yep. If I understand correctly, OSH doesn't support the hardware SPI peripheral in their library functions so you're on your own.

If we can start again from here, it would be helpful, as there are a lot of comments rattling round, which is confusing.
We can try that.

Here are 2x CODEs MASTER and SLAVE, hopefully correct, but check, also todays result and PCB5.
Haven't looked in detail but some of the firmware UART messages are commented out...

How's that?
 
Last edited:

Thread Starter

camerart

Joined Feb 25, 2013
3,730
Ok, from now on I'll call this type of program, FW (Firmware) which for me will punctuate this date.

I have hooked up the (As D/S) SPI PINs see PCB5. I have also added an 8MHz XTL xPLL = 32MHz, to the SLAVE, as I read, it needs to be faster than it's MASTER.

You picked up the SPI bus OK and hooked SCK, MOSI, MISO to the correct pins for the hardware MSSP SPI peripheral on the 4431 slave BUT, you can't use the same SS/ (CS/). Each SPI slave must have its own CS/ line or you will get bus contention due to multiple slaves driving MISO.

Re- HW SPI and UART, as seen, I changed to another PIN.
Yeah, it's a shame that SS/ on the 4431 is not relocatable to another pin so that you could use both the hardware UART and MSSP SPI peripherals at the same time. If you must use the SPI for a bi-directional data link you might consider using SPI to replace the UART data, just poll it when you want it. The alternative is the firmware UART as you indicate. As long as you are TX only and not using an interrupt driven transmit buffer, either way consumes lots of CPU time when it transmits.

Re- HW PINs vs FW PINs. If all written in FW, then which PINs must be used, or can any be used?
Some of the hardware pins can be relocated on the 4431 i.e. SCK and MOSI but that doesn't help since the SS/ and TX pins are co-located and not changable. If I understand the OSH manual, you can use the firmware bit-banged UART/SPI functions on almost any pin that you specify.

Re- SSPCON SSPM, can you clarify, MASTER or SLAVE?
I am working on the notion that the 4431 is the SPI slave i.e. the 4620 initiates all SPI comms and drives SCK. That makes the 4431 an SPI slave and since there are multiple SPI slaves on the bus, requires that you use a unique SS/ to select the 4431 slave for comms.

Re- SPICSon/SPICSoff, this is now FW so Oshonsoft (Edit: Oshonsoft SPI) is not used here.
Yep. If I understand correctly, OSH doesn't support the hardware SPI peripheral in their library functions so you're on your own.

If we can start again from here, it would be helpful, as there are a lot of comments rattling round, which is confusing.
We can try that.

Here are 2x CODEs MASTER and SLAVE, hopefully correct, but check, also todays result and PCB5.
Haven't looked in detail but some of the firmware UART messages are commented out...

How's that?
ME- I have hooked up the (As D/S) SPI PINs see PCB5. I have also added an 8MHz XTL xPLL = 32MHz, to the SLAVE, as I read, it needs to be faster than it's MASTER.
JY- You picked up the SPI bus OK and hooked SCK, MOSI, MISO to the correct pins for the hardware MSSP SPI peripheral on the 4431 slave BUT, you can't use the same SS/ (CS/). Each SPI slave must have its own CS/ line or you will get bus contention due to multiple slaves driving MISO.


Me- Are you referring to this line in 4431? Symbol slave_cs = PORTC.6 The SLAVE doesn't have any of it's own SLAVES, so I think this line can be removed?

ME- HW SPI and UART, as seen, I changed to another PIN.
JT- Yeah, it's a shame that SS/ on the 4431 is not relocatable to another pin so that you could use both the hardware UART and MSSP SPI peripherals at the same time. If you must use the SPI for a bi-directional data link you might consider using SPI to replace the UART data, just poll it when you want it. The alternative is the firmware UART as you indicate. As long as you are TX only and not using an interrupt driven transmit buffer, either way consumes lots of CPU time when it transmits.


ME- The 4431 SLAVE doesn't need a UART, if all of the information can be tranferred via SPI.

ME- HW PINs vs FW PINs. If all written in FW, then which PINs must be used, or can any be used?
JT-Some of the hardware pins can be relocated on the 4431 i.e. SCK and MOSI but that doesn't help since the SS/ and TX pins are co-located and not changable. If I understand the OSH manual, you can use the firmware bit-banged UART/SPI functions on almost any pin that you specify.


ME- Ok, best use the designated SPI PINS only, as PCB5.

ME- SSPCON SSPM, can you clarify, MASTER or SLAVE?
JT-I am working on the notion that the 4431 is the SPI slave i.e. the 4620 initiates all SPI comms and drives SCK. That makes the 4431 an SPI slave and since there are multiple SPI slaves on the bus, requires that you use a unique SS/ to select the 4431 slave for comms.


ME- The initial idea was to switch the 4431 between MASTER-SLAVE, so some CODE may still be there from that idea. If you spot any I'll remove it and re-post it in #38. So only MASTER slave_cs = PORTD.2 to SLAVE PORTC.6 IN (SS)

Me- SPICSon/SPICSoff, this is now FW so Oshonsoft (Edit: Oshonsoft SPI) is not used here.
JT- Yep. If I understand correctly, OSH doesn't support the hardware SPI peripheral in their library functions so you're on your own.


Me- It does have it in the library, but as there were errors, we have been testing this different method. (Perhaps Oshonsoft SPI is ok??) Probably best to carry on this way?

ME-If we can start again from here (#38), it would be helpful, as there are a lot of comments rattling round, which is confusing.
JT- We can try that.

Here are 2x CODEs MASTER and SLAVE, hopefully correct, but check, also todays result and PCB5.
Haven't looked in detail but some of the firmware UART messages are commented out...


Me- the intial CODE writer for this (thanks) was able to simulate it, but I am unable to due to there being 2x PICS, so I commented out the not used lines.

How's that?

Me- Just about perfect:)
English translation 'How's that?' HOWZAT is used in cricket as an appeal to the Umpire as to whether a batsman is out. ('Sometimes the Umpire strikes back', example of poor English joke.

C.
[/QUOTE]
 
Top