best error detection of serial data

Thread Starter

AlbertHall

Joined Jun 4, 2014
12,653
I want to send 7 bytes of data using HC12. If incorrect data is received then it is OK to wait for the next data. But it is important to be able to detect that the received data is correct as transmitted.

Microchip PIC and XC8.

What is the best method of doing this, please?
 

Jerry-Hat-Trick

Joined Aug 31, 2022
845
How about sending 8 bytes where the 8th byte is the most significant bits of the sum of the 7 bytes? I suspect this would be a pretty robust check sum.
 

MrChips

Joined Oct 2, 2009
35,145
There are simple well tested techniques for error detection and error correction.

The first method is to send 8 bits, (7 bits plus parity). This will detect a single bit error, but not errors in two bits. It will not allow you to correct the error. This simple parity check relies on the probability of n-bit errors being much higher for 1-bit error versus multi-bit errors.

If you are sending just one byte of data, then a secure method is to send the data three times. You can devise different encoding schemes but the end result is that you can correct the error.

You can send 8-bit data and append 4 bits of error correction (Hamming code). In effect, you are sending 12 bits of data. This will allow you to detect and correct a single bit error. It will detect two bits in error but not correct them.

A common scheme for transmitting a sequence of bytes, is to send 7 bits plus parity, followed by one extra 8-bit checksum. (Or it can be encoded as longitudinal parity). This is a very efficient method of ensuring packet integrity. You then have to device a procedure as to what to do when an error is detected.

Edit: I misread the OP. I thought I read 7 bits and not 7 bytes.
In that case, a simple solution is to send 7-bit plus parity and terminate the message with checksum byte.
 

WBahn

Joined Mar 31, 2012
33,152
I want to send 7 bytes of data using HC12. If incorrect data is received then it is OK to wait for the next data. But it is important to be able to detect that the received data is correct as transmitted.

Microchip PIC and XC8.

What is the best method of doing this, please?
There's no way to absolutely ensure that it is correct, so it really comes down to how low does the risk have to be of accepting bad data has to get before it is acceptable.

If you have the time and the processing ability on both ends, there are lots of ways to drive it down quite low.

Perhaps the simplest is to simply to transmit the data twice and throw out any data packet that doesn't match. Or, send it three times and if two of the three match, accept that one. If they are all different, reject the packet. To get something in between, accept the majority vote at each bit position (which is how one variant of Bluetooth does it, in essence).

If your channel is not congested, this would provide a very robust means.

Adding an eighth byte that is the parity of the other seven bytes will detect all one-bit errors, but can fail to identify some two-bit errors. Doing parity by rows and columns would allow you to correct all one-bit errors and detect all two- and three-bit errors, but fail to detect some four-bit errors.

Then there are several checksum and CRC approaches, many of which are specifically designed to be simple to implement in hardware or firmware.
 

WBahn

Joined Mar 31, 2012
33,152
How about sending 8 bytes where the 8th byte is the most significant bits of the sum of the 7 bytes? I suspect this would be a pretty robust check sum.
This will fail to detect many 1-bit errors. It also makes it so that errors in some bit positions are more detectable than others, which is generally not preferred.

Just sending an 8th byte that is the bitwise XOR of the data bytes means that any 1-bit error is detected and for a 2-bit error to fail to detect, it would have to occur in the same bit position in two different data bytes.
 

Rf300

Joined Apr 18, 2025
126
A good solution would be a CRC8 or even better CRC16 which leads to one or two additional bytes. Many background information, theory and C programs can be found at the "Painless guide to CRC". And a CRC is much better than a simple checksum and with the algorithm presented in the paper it doesn't need much computation power.
 

Rf300

Joined Apr 18, 2025
126
Oops, I just saw that my link doesn't work. Hopefully I can find that paper somewhere else.

Edit: Try this link.
 
Last edited:

ericgibbs

Joined Jan 29, 2010
21,567
Hi Rf,
Your link worked OK for me.
E

https://web.archive.org/web/20190519134454/http://www.ross.net/crc/

Welcome to The CRC Pitstop. The purpose of this web is to act as a repository for information on CRC and other checking algorithms. This web contains all kinds of information on CRC algorithms ranging from a paper explaining in detail how they work, to actual code you can drive off with. So whether you are studying the theory of CRCs, or just want to grab some C code and go, I hope you enjoy your pitstop here.

Ross N. Williams
ross@ross.net
2 May 1996.
 

MrChips

Joined Oct 2, 2009
35,145
As usual, a lot depends on the application, the probability of error, and the consequences of receiving false data.

In one application, I needed to transmit wirelessly, four status conditions, GREEN, YELLOW, RED, and OFF.
Since I only needed to send two bits of information, there was ample room for redundancy in an 8-bit byte.
 

Rf300

Joined Apr 18, 2025
126
Hi Rf,
Your link worked OK for me.
E

https://web.archive.org/web/20190519134454/http://www.ross.net/crc/

Welcome to The CRC Pitstop. The purpose of this web is to act as a repository for information on CRC and other checking algorithms. This web contains all kinds of information on CRC algorithms ranging from a paper explaining in detail how they work, to actual code you can drive off with. So whether you are studying the theory of CRCs, or just want to grab some C code and go, I hope you enjoy your pitstop here.
That's true, but from there I couldn't proceed to the full paper. Therefore the next link, which leads directly to an HTML version of the original article.
 

drjohsmith

Joined Dec 13, 2021
1,647
I want to send 7 bytes of data using HC12. If incorrect data is received then it is OK to wait for the next data. But it is important to be able to detect that the received data is correct as transmitted.

Microchip PIC and XC8.

What is the best method of doing this, please?
as a general rule
the more extra data you can send, the more reliable you can make the connection.
it all depends on how reliable you want it.
for instance , if it was to turn a traffic light on, you might want better than if it was your door bell.

parity is simple
crc is more
three packet , send responce confirm is more
identifying a problem is different to recovering the real data

what do you actualy want ?
 

nsaspook

Joined Aug 27, 2009
16,474
For a HC12 with a 9600bps serial link IMO I wouldn't worry about it unless there are some sort of special source of data contamination for a short cable run.

An example check (validate formatting, length and other factors ) of CSV streams (4MHz) that are encoded in a crc checked data packet.

C:
/*
* check the BMC CSV data string for proper format, size and check-mark
*/
char * validate_bmc_text(const char * text, bool * valid)
{
    char tmp_test_ptr[SYSLOG_SIZ], *tmp_p = (char *) text;
    uint32_t len = 0, starts = 0, checkmark = 0;
    bool end_data = false;
    char *jtoken;

    validate_failure = 0;
    strncpy(tmp_test_ptr, text, SYSLOG_SIZ - 1);
    valid[0] = true;
    if (tmp_test_ptr[0] == '^') {
        starts++;
        tmp_p = (char *) &text[0]; // return pointer to start of possible data
        if ((len = strlen(tmp_test_ptr)) >= VALIDATE_LEN) {
            for (int i = 1; i <= len; i++) {
                if (tmp_test_ptr[i] == '^') {
                    starts++;
                    end_data = false;
                    tmp_p = (char *) &text[i]; // return pointer to start of possible data
                }
                if (tmp_test_ptr[i] == '~') {
                    if (starts) {
                        end_data = true;
                    }
                }
            }
            if (end_data == false) {
                valid[0] = false;
                validate_failure = 3;
            }
            strncpy(tmp_test_ptr, tmp_p, SYSLOG_SIZ - 1);
            jtoken = strtok(tmp_test_ptr, ",");
            if (jtoken != NULL) {
                for (int i = 0; i < CSV_COUNT; i++) {
                    jtoken = strtok(NULL, ",");
                    if (jtoken == NULL) {
                        valid[0] = false;
                        validate_failure = 4;
                    }
                }
                jtoken = strtok(NULL, ",");
                if (jtoken != NULL) {
                    checkmark = atoi(jtoken);
                    if (checkmark != CHECKMARK) {
                        valid[0] = false;
                        validate_failure = 6;
                    }
                } else {
                    valid[0] = false;
                    validate_failure = 5;
                }
            }
        } else {
            valid[0] = false;
            validate_failure = 2; // data frame length error
        }
    } else {
        valid[0] = false;
        validate_failure = 1; // no/incorrect ASCII data-frame character
    }

    return tmp_p;
}

String format:
Code:
    const char log_format1[] = "^,%d,%3.2f,%3.2f,%3.2f,%3.2f,%3.2f,%3.2f,%3.2f,%3.2f,%d.%01d,%d.%01d,%d.%01d,%d,%d,%d,%d,%d,%llu,%7.4f,%7.4f,%7.4f,%7.4f,1957,~EOT                                                  \r\n";
#define LOG_VARS1    BMC4.d_id,((float) em.vl3l1) / 10.0f,em_tmp.al1, ((float) em.wl1) / 10.0f, ((float) em.wl2) / 10.0f, \
    ((float) em.val1) / 10.0f, ((float) em.varl1) / 10.0f,  ((float) em.pfsys) / 10.0f, em_tmp.hz,vw, vf, pvw, pvf, bat_amp_whole - 128, \
    bat_amp_frac - 128, bat_amp_panel - 128, panel_watts, BM.FM80_online, cc_mode, C.data_ok,BM.node_id, \
    ha_daq_calib.scaler4, ha_daq_calib.scaler5, ha_daq_calib.A200_Z, ha_daq_calib.A200_S

    const char log_format2[] = "^,%d,%3.2f,%3.2f,%3.2f,%3.2f,%3.2f,%3.2f,%3.2f,%3.2f,%d.%01d,%d.%01d,%d.%01d,%d,%d,%d,%d,%d,%llu,%6.4f,%6.4f,%6.4f,%6.4f,1957,~EOT                                                  \r\n";
#define LOG_VARS2    BMC4.d_id,((float) em.vl1n) / 10.0f,((float) em.vl2n) / 10.0f, ((float) em.vl3n) / 10.0f, ((float) em.al2) / 1000.0f, \
    ((float) em.wl3) / 10.0f, ((float) em.wsys) / 10.0f,  ((float) em.pfl1) / 10.0f, ((float) em.pfl2) / 10.0f,vw, vf, pvw, pvf, bat_amp_whole - 128, \
    bat_amp_frac - 128, bat_amp_panel - 128, BM.benergy, BM.FM80_online, cc_mode, C.data_ok,BM.node_id, \
    imd_tmp.wl1, imd_tmp.wl2, imd_tmp.wl3, imd_tmp.varsys
 
Last edited:

panic mode

Joined Oct 10, 2011
5,230
I want to send 7 bytes of data using HC12. If incorrect data is received then it is OK to wait for the next data. But it is important to be able to detect that the received data is correct as transmitted.
why not send it continuously as is. no CRC needed. on the receiving side treat data as valid if two consecutive telegrams match. it does not get simpler than that.
 

MrChips

Joined Oct 2, 2009
35,145
why not send it continuously as is. no CRC needed. on the receiving side treat data as valid if two consecutive telegrams match. it does not get simpler than that.
CRC or no CRC, packet should contain unique bytes to indicate start-of-frame and end-of-frame.
 

drjohsmith

Joined Dec 13, 2021
1,647
I don't want to send continuously as I have other applications using 443MHz, But I could send three times and do a majority vote.
Sending three times is a great simple way in theory.
What happens if you miss the first one ? How do you know this packet is the start of three ?
A time out might be needed , that then limits how often you can send.
As you have 7 bits data
How about sending 8 , the 8th being a parity ? If your first has it's own parity , you now have two parity bits , which should agree , set one as positive , one negative parity and they should be opposite. .
 
Top