best error detection of serial data

WBahn

Joined Mar 31, 2012
33,169
I don't want to send continuously as I have other applications using 443MHz, But I could send three times and do a majority vote.
How is your data currently being framed?

Is each byte sent as a separate unit, with start and stop bits? Or is it sent as a stream of 56 data bits with additional framing information?
 

WBahn

Joined Mar 31, 2012
33,169
The seven bytes sent as block with start and end marker bytes added.
Assuming that your current system is able to receive data with this framing, then doing the majority-vote approach should work.

Do keep in mind that there is a tradeoff between error correction and error detection.

With majority-of-three, you can correct any single bit error, but some two-bit errors will be accepted as valid. So you need to decide which is more important, correcting single-bit errors at the risk of accepting some two-bit errors, or being able to detect all one- and two-bit errors but not being able to correct the one-bit errors. You've already indicated that you can live with discarding data and waiting for the next one if it is in error, so you might be better off accepting the data only if all three data sets agree. Part of this decision needs to reflect the seriousness of accepting wrong information (which, if they are date/time stamps, can also be validated based on are they valid data/time stamps within reasonable limits), but another part is how often do you get bit errors of various types, including burst errors. It should be pretty easy to collect data to give you an idea of your bit error rate profile.
 

drjohsmith

Joined Dec 13, 2021
1,647
Assuming that your current system is able to receive data with this framing, then doing the majority-vote approach should work.

Do keep in mind that there is a tradeoff between error correction and error detection.

With majority-of-three, you can correct any single bit error, but some two-bit errors will be accepted as valid. So you need to decide which is more important, correcting single-bit errors at the risk of accepting some two-bit errors, or being able to detect all one- and two-bit errors but not being able to correct the one-bit errors. You've already indicated that you can live with discarding data and waiting for the next one if it is in error, so you might be better off accepting the data only if all three data sets agree. Part of this decision needs to reflect the seriousness of accepting wrong information (which, if they are date/time stamps, can also be validated based on are they valid data/time stamps within reasonable limits), but another part is how often do you get bit errors of various types, including burst errors. It should be pretty easy to collect data to give you an idea of your bit error rate profile.
the two bits in error , would have to be in the same bit in two out of the three bytes ,
thats incredable unlikely, normaly one would expect interferance to be in adjecent bits,
as above , if you put parity on the 7 bits your sending, then the probability of errors when also majority voting is now getting small.
only accepting if all three transmisions are the same has probkem in that if you have average of one bit in 56 being in error, your never going to receive data. the problem is you have increased the amount of data , so for a given error rate , your more likely to receive an error.
 

WBahn

Joined Mar 31, 2012
33,169
the two bits in error , would have to be in the same bit in two out of the three bytes ,
thats incredable unlikely, normaly one would expect interferance to be in adjecent bits,
as above , if you put parity on the 7 bits your sending, then the probability of errors when also majority voting is now getting small.
only accepting if all three transmisions are the same has probkem in that if you have average of one bit in 56 being in error, your never going to receive data. the problem is you have increased the amount of data , so for a given error rate , your more likely to receive an error.
Notice that I didn't say that any two-bit errors, would be detected, only that SOME two-bit errors would be detected. The strength of error-detecting and -correcting codes are primarily stated in terms of their ability to detect/correct ALL bit errors up to a certain count.

The problem of not getting any data through is a real one and requires looking at those bit error rates I was talking about. Using a simple scheme like this one, which has a poor code rate (a.k.a., a high coding overhead), is not a good choice in high bit-error rate environments for just this reason.

If the protocol allows for sending 9-bit data chunks so that parity can be added to each byte, then sending that plus an eighth word that is the bitwise parity would allow the detection of all errors up to three bits (and almost all four-bit errors) and is pretty resilient to burst errors. It also has a pretty low coding overhead (it's a 7/9 rate code as opposed to a 1/3 rate code for the majority approach).
 

drjohsmith

Joined Dec 13, 2021
1,647
Notice that I didn't say that any two-bit errors, would be detected, only that SOME two-bit errors would be detected. The strength of error-detecting and -correcting codes are primarily stated in terms of their ability to detect/correct ALL bit errors up to a certain count.

The problem of not getting any data through is a real one and requires looking at those bit error rates I was talking about. Using a simple scheme like this one, which has a poor code rate (a.k.a., a high coding overhead), is not a good choice in high bit-error rate environments for just this reason.

If the protocol allows for sending 9-bit data chunks so that parity can be added to each byte, then sending that plus an eighth word that is the bitwise parity would allow the detection of all errors up to three bits (and almost all four-bit errors) and is pretty resilient to burst errors. It also has a pretty low coding overhead (it's a 7/9 rate code as opposed to a 1/3 rate code for the majority approach).
i think the op said they had 7 bit data.
 

WBahn

Joined Mar 31, 2012
33,169
i think the op said they had 7 bit data.
From the OP: " I want to send 7 bytes of data using HC12 "

Unless they clarified it in a later post.

But they did say in a later post that their data was date/time information. The example they gave was:

Data is time and date.
e.g. 0, 34, 12, 0, 6, 9, 26
I'm guessing here, but if that is
HR (0-23)
MIN: (0-59)
SEC: (0-59)
1/100 sec (0-99)
DAY: (1-31)
MONTH: (1-12)
YEAR: (0-99) (note the Y2.1K problem here)

Then they can get away with seven data bits and use the eighth bit as a parity bit, then adding an eight byte as the bitwise-parity.

They could compress that into fewer bits by converting it into a serial number. If it was the number of 1/100 seconds since the start of 2000, then 39 bits would be sufficient to span a century, so use five bytes instead of eight. That reduces the chance of transmission bit errors by removing the redundancy, but it adds complexity in the translations.
 

drjohsmith

Joined Dec 13, 2021
1,647
From the OP: " I want to send 7 bytes of data using HC12 "

Unless they clarified it in a later post.

But they did say in a later post that their data was date/time information. The example they gave was:



I'm guessing here, but if that is
HR (0-23)
MIN: (0-59)
SEC: (0-59)
1/100 sec (0-99)
DAY: (1-31)
MONTH: (1-12)
YEAR: (0-99) (note the Y2.1K problem here)

Then they can get away with seven data bits and use the eighth bit as a parity bit, then adding an eight byte as the bitwise-parity.

They could compress that into fewer bits by converting it into a serial number. If it was the number of 1/100 seconds since the start of 2000, then 39 bits would be sufficient to span a century, so use five bytes instead of eight. That reduces the chance of transmission bit errors by removing the redundancy, but it adds complexity in the translations.
my apologies , read bits not bytes !
 

MrChips

Joined Oct 2, 2009
35,157
There are other ways to improve the efficacy.
Why transmit day, month year, every time?
You could transmit this as four bytes: D, M. YHI, YLO.
Time in four bytes, H, M, S, S00.
Do you really need 1/100 seconds?
Each of H, M, S require only 6 bits. You can encode everything into 26 data bits and append 5-bit parity.
 

joeyd999

Joined Jun 6, 2011
6,456
There are other ways to improve the efficacy.
Why transmit day, month year, every time?
You could transmit this as four bytes: D, M. YHI, YLO.
Time in four bytes, H, M, S, S00.
Do you really need 1/100 seconds?
Each of H, M, S require only 6 bits. You can encode everything into 26 data bits and append 5-bit parity.
It can be encoded further:

just send the delta since the last time stamp.
 

joeyd999

Joined Jun 6, 2011
6,456
would that not lower the error protection , as an undetected error is now propergated !
all these great ideas, wonder what the op thinks , or are they just after something simple, ?
Once encoded, there are plenty of ways to ensure error free transmission in as few bits as possible.

I've done similar back in the 90s to essentially allow a nearly 1 Mbit transmission rate on a 115kbaud channel.

At the time, I called it XYZ compression, because there were three different dimensions over which the data could be compressed adaptively.
 

WBahn

Joined Mar 31, 2012
33,169
Received as BCD and will be used as BCD
Okay. I'll assume that this is a hard requirement, for whatever reason.

Was my guess at your format correct (and why are we having to guess)? Are transmitting timestamps with 1/100 second resolution?

How frequent are the transmissions?

What are the minimum/maximum times between timestamp transmissions?

What is the baud rate?

How many bits per symbol (I'm guessing one, but there I am guessing again).

How critical is it that an incorrectly received transmission not be accepted as valid?

How many successive transmissions can be missed before it is unacceptable?

What is the bit error rate of your channel?

You mentioned other users of the spectrum. How are you deconflicting with them?

Do you have two-way communication such that sending an acknowledgement or repeat request is an option?
 
Top