Digital Filtering Advice

Thread Starter

jpanhalt

Joined Jan 18, 2008
11,087
Spring has not totally interrupted my sessions with WWVB (See: http://forum.allaboutcircuits.com/threads/selecting-best-antenna-type-for-wwvb.121386/#post-986466 ). My codes work fine for reading the sequence when there is no interference. The first attachment (in-sync scan) shows what the signal and marks can look like. Magenta is the signal from my receiver. Yellow marks the 1's and 0's being recorded. It is set with a "1" and cleared after a "0." Cyan marks the "zero minute" transition.
in-sync scan.png
The next attachment shows an out-of-sync screen - note the small blip (^).
Out of sync_readable copy.png
The pattern is still readable by the eye. However, unless the noise is really bad, it does not show on the oscilloscope (Rigol DS1054Z), but still screws up my reading of the signal. My current software will detect the noise, signal that the reception is not reliable, and reset, but it doesn't fix the noise. I would like to address the noise, much as the Rigol scope does. I have tried ignoring runt transitions (i.e., set a minimum peak width and simply continue timing without reset when a peak is very small). But, that did not work as well as I wanted it to.

Rigol averages multiple readings of each transition to get the signal it plots. Superficially, that may resemble debouncing a switch, but it uses an average, instead of a state. I am not sure of Rigol's algorithm, but following upon this reference and others, I am considering using a Finite Impulse Response (FIR) algorithm and am considering sampling about every 32 us and averaging 31 samples ( or 15, a nibble) to decide when there is a true transition. That will give me about 1 "averaged" sample every millisecond.

1) Does that sampling rate sound about right? (remember, the baud for WWVB is just 1).
2) Does anyone know with certainty what algorithm Rigol and other scopes use?

Regards, John
 
Last edited:

JohnInTX

Joined Jun 26, 2012
4,787
2) You might want to try the PEAK DETECT acquisition mode. IIRC, in HighRes mode you're using, the scope averages neighboring points in a single sweep. In AVERAGE mode, it averages multiple repetitive sweeps of a periodic waveform to reduce the noise. Either mode will slow the waveforms / second rate i.e. after one sweep, it takes time to process and display the data before the next sweep can be set up (dead time). The more processing the scope has to do (more channels, long trace memory, more displayed measurements etc.) the fewer wfms/sec, sometimes enough to cause you to miss things. You can observe the wfm/sec rate by measuring the pulse freq. on TRIGGER OUT.

Don't know about the FIR filter.
Good luck.
 

Thread Starter

jpanhalt

Joined Jan 18, 2008
11,087
I will try that setting. Thanks for the suggestion. My previous digital scope was and is the Tek TDS210. Getting used to the delay in the Rigol has not been entirely easy. When I want to see what is really happening, I go to the TEK. But that is a pain, since I do not have the capture/interface module.

John
 

JohnInTX

Joined Jun 26, 2012
4,787
I will try that setting. Thanks for the suggestion. My previous digital scope was and is the Tek TDS210. Getting used to the delay in the Rigol has not been entirely easy. When I want to see what is really happening, I go to the TEK. But that is a pain, since I do not have the capture/interface module.
I miss my Tek TDS420. It expired last year (bad power supply due to the great electrolytic cap fiasco, I think), did the research and bought a new Keysight unit (formerly HP/Agilent). They think differently. I'm told that Keysight's basic stuff is Rigol so we are in the same boat.

Here's one of the ApNotes that got me thinking about wfms/sec. Never considered it before but it makes sense.
http://www.keysight.com/main/redire...IAL&ckey=1374518&lc=eng&cc=US&nfr=-11143.0.00

Thanx for the link on filtering - nice.
Best,
John
 

Thread Starter

jpanhalt

Joined Jan 18, 2008
11,087
@JohnInTX

That is a well written and useful link. Had to wait until morning to test your advice.

This tracing at about 0700 EDT is consistent with what I was seeing with my capture program:
Peak det w.noise.png

Thank you for the pointers.

BTW, the Rigol display is 12 time-units wide.

John
 

Thread Starter

jpanhalt

Joined Jan 18, 2008
11,087
UPDATE
By the time I finished yard work yesterday, the WWVB signal was too clear to test my filter. Here are the results from this morning:
1) First attachment is a no-noise WWVB signal (yellow = receiver output, cyan = filtered signal)
DigFilter s. noise 041516.png
2) Second is WWVB with noise
DigFilter w. noise 041516.png
The code is in Assembly, so I felt that a flowchart might be more readable to most (See: pdf attachment).

In brief, the code samples the signal at fixed intervals and outputs a filtered state based on a majority. In this instance, I used a long delay to accentuate the effect on the signal. Peak widths are changed slightly, but that effect can be factored into the decoding scheme. If actually implemented, I would probably use an interrupt clocked by TMR2 for the sampling. I did the population count manually, and the table just represents the outcome from that.

John

Code:
DSP
     bsf       STATUS,0  
     btfss     PORTA,2        ;read input
     bcf       STATUS,0
     rlf       DSP_Reg,f      ;put RA2 into DSP_Reg
     movlw     b'00111111'    ;
     andwf     DSP_Reg,f      ;mask bts <6,7> in DSP_Reg
     movf      DSP_Reg,w
     call      Table
     movlb     2              ;LATC bank
     btfss     WREG,0
     bcf       LATC,7         ;output pin
     btfsc     WREG,0
     bsf       LATC,7         ;output pin
     movlb     0
     DelayCy   (20*msecs)     ;delay macro (temporary)
     goto      DSP
Table
     brw
     dt 0,0,0,0,0,0,0,0,0,0
     dt 0,0,0,0,0,1,0,0,0,0
     dt 0,0,0,1,0,0,0,1,0,1
     dt 1,1,0,0,0,0,0,0,0,1
     dt 0,0,0,1,0,1,1,1,0,0
     dt 0,1,0,1,1,1,0,1,1,1
     dt 1,1,1,1
Edit:04/21/16 corrected error in table line2, positions 5&6 were reversed. jpa
 

Attachments

Last edited:

JohnInTX

Joined Jun 26, 2012
4,787
@jpanhalt Nice work.
I see you are using enhanced midrange. Be sure to check the errata for your particular chip - some of them don't execute BRW or BRA correctly when interrupts are used. I know, right?
 

Thread Starter

jpanhalt

Joined Jan 18, 2008
11,087
UPDATE#2
A problem with the above method (post #6) is that the time lag introduced is not symmetrical when going high to low versus low to high. I knew that when first testing it, but then decided it was not a good idea. The obvious fix was to take 7 measurements, which gives a larger table (n=128). There was a transposition in the original table, which has been fixed and noted.

Cold and rainy today, so maybe I will see some atmospheric disturbances to see how well the filter works. In the meantime, I had nothing to do, so I re-wrote code for the Kernigham algorithm for population count into Assembly. It is not faster than a table, but takes up less space. In the rare event someone may be interested:

Code:
;Kernigham algorithm for population count
;Source: http://www.geeksforgeeks.org/count-set-bits-in-an-integer/

     list      p=12F1840
     #include  <p12F1840.inc>
     radix     dec

     cblock 0x20
     integer
     popcount
     endc
 
     clrf      popcount
     movlw     13             ;enter integer to test
     movwf     integer

Start
     movf      integer,w      ;check for n=0
     btfsc     STATUS,2
     goto      Done
Loop
     incf      popcount
     decf      integer,w
     andwf     integer,f
     btfss     STATUS,2
     goto      Loop 
Done
     NOP                      ;result in popcount
     end
John
 
Last edited:

Thread Starter

jpanhalt

Joined Jan 18, 2008
11,087
UPDATE #3

Finally, thunderstorms. There is a storm approaching Fort Wayne, IN (about 200 miles West on a line to Fort Collins, CO from Cleveland, OH), and I am picking up some interference.
upload_2016-4-21_18-46-6.png

Using the 7-bit filter, delays of 2, 4, and 10 msecs helped a little, but the signal from WWVB was still too corrupted to be read by my algorithm. With sampling at 20 msecs, better results are obtained.

Here is some noise that was cleaned up nicely:

upload_2016-4-21_18-51-37.png

But a really big strike doesn't do so well:

upload_2016-4-21_18-53-28.png

My conclusion is that this simple filter works well enough if all you need to do is to capture time to the nearest second and it is not a big storm. On the other hand, if you want to measure a few milliseconds drift of the RTC, you had better wait for clear weather.

John

PS: I used the TEK 210 scope because it doesn't have a frustrating pause between sweeps.
 

NorthGuy

Joined Jun 28, 2014
611
I think FIR is not the best idea for WWVB.

The general goal is to synchronize the down transition at the beginning of every second. Once you get it synchronized, it's easy to predict and only needs gentle adjustments if your PIC's clock strats wandering away. To make adjustments, you can do heavy sampling at the beginning of the second, and if you detect that the transition wandered in some direction and is consistently off, you adjust your clock to WWVB. This synchronization puts you in sync with real time on sub-second basis and is the most important.

Once you have it synchronized, you only need to sample in the interval between 0.2 and 0.5s from the beginning of the second (and possibly 0.5 to 0.8 if you don't know yet where the minute starts). If you make 100 samples from 0.25 to 0.45 and then decide the bit by the majority of them (1 if you get 50 ones or more, 0 if you get less), it'll be much more reliable than detecting edges.

Since for most bits you already know what you're about to receive, it's easy to spot errors when your detection goes wrong.
 

Thread Starter

jpanhalt

Joined Jan 18, 2008
11,087
@NorthGuy
Thank you for the comment. This project has a history, and it is completely understandable that you may not have seen or remembered those details. In March, I posted how I was determining the RTC module's drift: http://forum.allaboutcircuits.com/threads/learning-rigol-ds1054z-quick5.122468/#post-982795

That was very similar to the process you describe, except at that time, I counted edges in the WWVB signal instead of a less noise prone signal. Of course it worked when I had a clean signal. When there was noise, it did not. So, I just changed to counting the edges on the 1 Hz square wave that the RTC can be configured to provide. So long as the actual zero-second signal from WWVB is not too noisy, it works as expected. Here is an example with the auto cursor enabled to show the current out of sync relationship between the RTC and WWVB:
RTC SQW sync.png
Yellow = raw WWVB; cyan = nearest rising edge of 1 Hz RTC synchronized to zero-second; and magenta = RTC 1Hz square wave. Removing that error is easily done. I just haven't done that.

The vast majority of consumer "atomic" clocks and watches only synchronize to WWVB about once a day -- usually late at night -- when interference is low. However, I was intrigued about reading the signal when there was interference, and the digital filter was just a flight/tangent in that direction. I was happy with the result.

Regards, John
 
Top