Digital Filtering Advice

Discussion in 'Embedded Systems and Microcontrollers' started by jpanhalt, Apr 13, 2016.

  1. jpanhalt

    Thread Starter Expert

    Jan 18, 2008
    Spring has not totally interrupted my sessions with WWVB (See: ). My codes work fine for reading the sequence when there is no interference. The first attachment (in-sync scan) shows what the signal and marks can look like. Magenta is the signal from my receiver. Yellow marks the 1's and 0's being recorded. It is set with a "1" and cleared after a "0." Cyan marks the "zero minute" transition.
    in-sync scan.png
    The next attachment shows an out-of-sync screen - note the small blip (^).
    Out of sync_readable copy.png
    The pattern is still readable by the eye. However, unless the noise is really bad, it does not show on the oscilloscope (Rigol DS1054Z), but still screws up my reading of the signal. My current software will detect the noise, signal that the reception is not reliable, and reset, but it doesn't fix the noise. I would like to address the noise, much as the Rigol scope does. I have tried ignoring runt transitions (i.e., set a minimum peak width and simply continue timing without reset when a peak is very small). But, that did not work as well as I wanted it to.

    Rigol averages multiple readings of each transition to get the signal it plots. Superficially, that may resemble debouncing a switch, but it uses an average, instead of a state. I am not sure of Rigol's algorithm, but following upon this reference and others, I am considering using a Finite Impulse Response (FIR) algorithm and am considering sampling about every 32 us and averaging 31 samples ( or 15, a nibble) to decide when there is a true transition. That will give me about 1 "averaged" sample every millisecond.

    1) Does that sampling rate sound about right? (remember, the baud for WWVB is just 1).
    2) Does anyone know with certainty what algorithm Rigol and other scopes use?

    Regards, John
    Last edited: Apr 14, 2016
    JohnInTX likes this.
  2. JohnInTX


    Jun 26, 2012
    2) You might want to try the PEAK DETECT acquisition mode. IIRC, in HighRes mode you're using, the scope averages neighboring points in a single sweep. In AVERAGE mode, it averages multiple repetitive sweeps of a periodic waveform to reduce the noise. Either mode will slow the waveforms / second rate i.e. after one sweep, it takes time to process and display the data before the next sweep can be set up (dead time). The more processing the scope has to do (more channels, long trace memory, more displayed measurements etc.) the fewer wfms/sec, sometimes enough to cause you to miss things. You can observe the wfm/sec rate by measuring the pulse freq. on TRIGGER OUT.

    Don't know about the FIR filter.
    Good luck.
    jpanhalt likes this.
  3. jpanhalt

    Thread Starter Expert

    Jan 18, 2008
    I will try that setting. Thanks for the suggestion. My previous digital scope was and is the Tek TDS210. Getting used to the delay in the Rigol has not been entirely easy. When I want to see what is really happening, I go to the TEK. But that is a pain, since I do not have the capture/interface module.

  4. JohnInTX


    Jun 26, 2012
    I miss my Tek TDS420. It expired last year (bad power supply due to the great electrolytic cap fiasco, I think), did the research and bought a new Keysight unit (formerly HP/Agilent). They think differently. I'm told that Keysight's basic stuff is Rigol so we are in the same boat.

    Here's one of the ApNotes that got me thinking about wfms/sec. Never considered it before but it makes sense.

    Thanx for the link on filtering - nice.
    jpanhalt likes this.
  5. jpanhalt

    Thread Starter Expert

    Jan 18, 2008

    That is a well written and useful link. Had to wait until morning to test your advice.

    This tracing at about 0700 EDT is consistent with what I was seeing with my capture program:
    Peak det w.noise.png

    Thank you for the pointers.

    BTW, the Rigol display is 12 time-units wide.

    JohnInTX likes this.
  6. jpanhalt

    Thread Starter Expert

    Jan 18, 2008
    By the time I finished yard work yesterday, the WWVB signal was too clear to test my filter. Here are the results from this morning:
    1) First attachment is a no-noise WWVB signal (yellow = receiver output, cyan = filtered signal)
    DigFilter s. noise 041516.png
    2) Second is WWVB with noise
    DigFilter w. noise 041516.png
    The code is in Assembly, so I felt that a flowchart might be more readable to most (See: pdf attachment).

    In brief, the code samples the signal at fixed intervals and outputs a filtered state based on a majority. In this instance, I used a long delay to accentuate the effect on the signal. Peak widths are changed slightly, but that effect can be factored into the decoding scheme. If actually implemented, I would probably use an interrupt clocked by TMR2 for the sampling. I did the population count manually, and the table just represents the outcome from that.


    Code (Microchip Assembler):
    2. DSP
    3.      bsf       STATUS,0  
    4.      btfss     PORTA,2        ;read input
    5.      bcf       STATUS,0
    6.      rlf       DSP_Reg,f      ;put RA2 into DSP_Reg
    7.      movlw     b'00111111'    ;
    8.      andwf     DSP_Reg,f      ;mask bts <6,7> in DSP_Reg
    9.      movf      DSP_Reg,w
    10.      call      Table
    11.      movlb     2              ;LATC bank
    12.      btfss     WREG,0
    13.      bcf       LATC,7         ;output pin
    14.      btfsc     WREG,0
    15.      bsf       LATC,7         ;output pin
    16.      movlb     0
    17.      DelayCy   (20*msecs)     ;delay macro (temporary)
    18.      goto      DSP
    19. Table
    20.      brw
    21.      dt 0,0,0,0,0,0,0,0,0,0
    22.      dt 0,0,0,0,0,1,0,0,0,0
    23.      dt 0,0,0,1,0,0,0,1,0,1
    24.      dt 1,1,0,0,0,0,0,0,0,1
    25.      dt 0,0,0,1,0,1,1,1,0,0
    26.      dt 0,1,0,1,1,1,0,1,1,1
    27.      dt 1,1,1,1
    Edit:04/21/16 corrected error in table line2, positions 5&6 were reversed. jpa
    • DSP.pdf
      File size:
      12.8 KB
    Last edited: Apr 21, 2016
  7. JohnInTX


    Jun 26, 2012
    @jpanhalt Nice work.
    I see you are using enhanced midrange. Be sure to check the errata for your particular chip - some of them don't execute BRW or BRA correctly when interrupts are used. I know, right?
  8. jpanhalt

    Thread Starter Expert

    Jan 18, 2008
    A problem with the above method (post #6) is that the time lag introduced is not symmetrical when going high to low versus low to high. I knew that when first testing it, but then decided it was not a good idea. The obvious fix was to take 7 measurements, which gives a larger table (n=128). There was a transposition in the original table, which has been fixed and noted.

    Cold and rainy today, so maybe I will see some atmospheric disturbances to see how well the filter works. In the meantime, I had nothing to do, so I re-wrote code for the Kernigham algorithm for population count into Assembly. It is not faster than a table, but takes up less space. In the rare event someone may be interested:

    Code (Microchip Assembler):
    2. ;Kernigham algorithm for population count
    3. ;Source:
    5.      list      p=12F1840
    6.      #include  <>
    7.      radix     dec
    9.      cblock 0x20
    10.      integer
    11.      popcount
    12.      endc
    14.      clrf      popcount
    15.      movlw     13             ;enter integer to test
    16.      movwf     integer
    18. Start
    19.      movf      integer,w      ;check for n=0
    20.      btfsc     STATUS,2
    21.      goto      Done
    22. Loop
    23.      incf      popcount
    24.      decf      integer,w
    25.      andwf     integer,f
    26.      btfss     STATUS,2
    27.      goto      Loop
    28. Done
    29.      NOP                      ;result in popcount
    30.      end
    Last edited: Apr 21, 2016
  9. jpanhalt

    Thread Starter Expert

    Jan 18, 2008
    UPDATE #3

    Finally, thunderstorms. There is a storm approaching Fort Wayne, IN (about 200 miles West on a line to Fort Collins, CO from Cleveland, OH), and I am picking up some interference.

    Using the 7-bit filter, delays of 2, 4, and 10 msecs helped a little, but the signal from WWVB was still too corrupted to be read by my algorithm. With sampling at 20 msecs, better results are obtained.

    Here is some noise that was cleaned up nicely:


    But a really big strike doesn't do so well:


    My conclusion is that this simple filter works well enough if all you need to do is to capture time to the nearest second and it is not a big storm. On the other hand, if you want to measure a few milliseconds drift of the RTC, you had better wait for clear weather.


    PS: I used the TEK 210 scope because it doesn't have a frustrating pause between sweeps.
  10. NorthGuy

    Active Member

    Jun 28, 2014
    I think FIR is not the best idea for WWVB.

    The general goal is to synchronize the down transition at the beginning of every second. Once you get it synchronized, it's easy to predict and only needs gentle adjustments if your PIC's clock strats wandering away. To make adjustments, you can do heavy sampling at the beginning of the second, and if you detect that the transition wandered in some direction and is consistently off, you adjust your clock to WWVB. This synchronization puts you in sync with real time on sub-second basis and is the most important.

    Once you have it synchronized, you only need to sample in the interval between 0.2 and 0.5s from the beginning of the second (and possibly 0.5 to 0.8 if you don't know yet where the minute starts). If you make 100 samples from 0.25 to 0.45 and then decide the bit by the majority of them (1 if you get 50 ones or more, 0 if you get less), it'll be much more reliable than detecting edges.

    Since for most bits you already know what you're about to receive, it's easy to spot errors when your detection goes wrong.
  11. jpanhalt

    Thread Starter Expert

    Jan 18, 2008
    Thank you for the comment. This project has a history, and it is completely understandable that you may not have seen or remembered those details. In March, I posted how I was determining the RTC module's drift:

    That was very similar to the process you describe, except at that time, I counted edges in the WWVB signal instead of a less noise prone signal. Of course it worked when I had a clean signal. When there was noise, it did not. So, I just changed to counting the edges on the 1 Hz square wave that the RTC can be configured to provide. So long as the actual zero-second signal from WWVB is not too noisy, it works as expected. Here is an example with the auto cursor enabled to show the current out of sync relationship between the RTC and WWVB:
    RTC SQW sync.png
    Yellow = raw WWVB; cyan = nearest rising edge of 1 Hz RTC synchronized to zero-second; and magenta = RTC 1Hz square wave. Removing that error is easily done. I just haven't done that.

    The vast majority of consumer "atomic" clocks and watches only synchronize to WWVB about once a day -- usually late at night -- when interference is low. However, I was intrigued about reading the signal when there was interference, and the digital filter was just a flight/tangent in that direction. I was happy with the result.

    Regards, John