A robust decoding and reporting system for WSPR and FST4W
The following are working definitions of data fields in our Timescale spots and wsprdaemon_spots_s tables.
10 June 2020 Include information from Steve Franke to Rob Robinett for WSJT-x V2.2.
23 October 2020 Add information on the 'code' field as used in WSJT-x V2.3 onwards.
24 May 2020 Bring up to date with additional variables and names.
There's a separate for wsprdaemon_noise_s data fields.
Variables uploaded to, or derived by, wsprnet.org and also uploaded to wsprdaemon_spots_s
WSPR field name definitions
date and time
Drift of the transmitted signal in Hz over the duration of the WSPR message seen by the receiver (which may also drift).
Frequency in MHz as seen at the receiver by adding the measured audio frequency to the 'dial' frequency for the selected band. Frequency is reported to seven significant digits, i.e. 0.1 Hz, to wsprdaemon_spots_s.
Distance in km calculated from the receiver and transmitter grid squares. Accuracy will be best with two 6-character locators.
Signal to Noise ratio calculated in wsprd from the 30th percentile of the sorted FFT coefficients scaled to a 2.5 kHz bandwidth.
Callsign (or other designator if SWL) of the reporting station.
4 or 6 character Maidenhead grid square. Map available .
Callsign of the transmit station. If the sender is using a WSPR Type 3 message then the entry will look like <AJ8S/1>.
Power reported by the transmitting station. 30 dBm = 1 Watt.
4 or 6 character Maidenhead grid square. Map available here.
Azimuth in degrees of the receiver as seen at the transmitter assuming a great circle short path. Clockwise from north. This is the azimuth reported by wsprnet.
A mode designator code: 1 is 'standard' WSPR2 and the new mode FST4W-120, 2 is WSPR15 and FST4W-900, 4 is FST4W-300 and 8 is FST4W-1800. Note: This field is superceeded by 'mode' in wsprdaemon_spots_s, see below.
A user-specified designator for a receiver, set in wsprdaemon.conf.
A count of the number of times the KiwiSDR sets its overload flag in a two-minute interval.
Band in metres for 2200 - 2 metres. When wsprdaemon can report spots from receivers at 70cm and 23cm those will be reported as 70 and 23.
Noise estimate from the wsprdaemon FFT algorithm using the c2 decimated samples file produced by wsprd. Units are dBm in 1Hz, however the absolute value will depend on the offset calibration provided at the receiver (same goes for rms_noise).
Noise estimate from the wsprdaemon RMS algorithm, essentially the RMS value of the quietest 50ms within the gap between WSPR transmissions.
Azimuth in degrees of the incoming signal at the receiver assuming a great circle short path from the transmitter. Clockwise from north. This is calculated by wsprdaemon; only tx_az is reported by wsprnet.
Latitude in degrees of the receiver calculated from the rx_grid. Negative is south. These numeric latitude and longitude fields allow for numeric SELECT statements in postgreSQL queries.
Longitude in degrees of the receiver calculated from the rx_grid. Negative is west.
Latitude in degrees of the transmitter calculated from the tx_grid. Negative is south.
Longitude in degrees of the transmitter calculated from the tx_grid. Negative is west.
Latitude in degrees of the vertex of the great circle path between receiver and transmitter. The vertex is the most northerly, or southerly, point on the path. There are, of course, instances where the vertex is at the receiver or transmitter. This is calculated by wsprdaemon and can be useful when studying paths that are near to or within the polar Auroral Ovals.
Longitude in degrees of the vertex of the great circle path between receiver and transmitter.
Instead of the field 'code' in the spots table 'mode' has values as follows: WSPR-2=2; FST4W-120=3; FST4W-300=6; WSPR-15=15; FST4W-900=16; FST4W-1800=31. Note that WsprDaemon distinguishes between WSPR-2 and FST4W-120.
Derived and other variables calculated by wsprdaemon and uploaded to wsprdaemon_spots_s
Variables available from wsprd, not uploaded to wsprnet but uploaded to wsprdaemon
The following variables are calculated within wsprd but are not reported to wsprnet. They have been added to the wsprdaemon data set for completeness and in case they may be of use in some studies, especially given our full relational database facility. The interpretation below is that of G Griffiths, G3ZIL, derived from reading the wsprd.c source code and related material. There may well be errors of interpretation or understanding. We would welcome posts on our Forum with corrections or further/clearer explanations. Curret as of wsjtx 2.2rc1.
A parameter controlling the detection of individual symbols in the wsprd demodulator. Allowable values are 1, 2, 3, units are symbols. A value of 1 signifies that the first try using non-coherent detection of individual symbols was successful (sufficient), this is equivalent to the original wspr demodulator. Blocksizes of 2 and above means that that many symbols are decoded at once; from the source code, "Longer block lengths require longer channel coherence time". Most of the time blocksize will be 1, as an example, of 90271 spots at KD2OM 88214 were blocksize 1, 1575 at 2 and 482 at 3.
This is the time difference between the actual start of the audio signal presented to wsprd and 2 seconds past an even minute as perceived by the clock at the receiver. wsprdaemon records the full 10ms resolution value. There are (possibly at least) four main causes for a non-zero reading:
1. A time offset from UTC in the clock at the transmitter.
2. A time offset from UTC in the clock at the receiver.
3. A delay (latency) at the transmitter between the clock-commanded start of transmission and the actual transmission.
4. A delay (latency) at the receiver between the clock-commanded start of reception and the actual audio file start.
This is the number of cycles taken for the Fano (default) or Jelinek (if selected) decoder to produce an output. The default maximum number of cycles is 10,000, but can be set with the -C option on calling wsprd. In practice the most common value for decode_cycles (mode) is 1; for example, decode_cycles was 1 for 10212 out of 13285 spots for KD2OM on 40m, which is about 77%. This field may also be refered to as Fano iterations-per-bit.
Applied as a fine adjustment to the well-known dt time shift value. Centred on zero, values are in steps of 8 between -64 and +64. and are tried in the sequence 0, -8, 8, 16 etc. For 91092 spots at KD2OM 85645 i.e. 94% jitter was 0, 1.5% were at -8 and 1% at +8, and all allowed values to +/-64 were present. While no unit is given, it is certainly time, and quite likely to be the sampling interval, that is 1/375Hz or 0.26667 seconds, making each step of 8 equivalent to 0.021333 seconds.
This is an output from the Fano (default) or Jelinek (if selected in the call to wsprd.c) decoder. In Information Theory, metric is a measure of the "closeness of a path to the received sequence". The distribution of metric for 92513 spots at KD2OM is shown below. There is a broad asymmetric distribution at about 570. The singular peak at 810 is because that value is when the Fano (or Jelinek) algorithm has failed. In that case, the Ordered Statistic Decoder (OSD) is executed. It will come up with its decode, and if accepted, the osd_decode flag (see below) will be set to 1. This happened (osd_decode flag set to 1) in 98.8% of instances when metric was 810 in this test case with KD2OM spots. If FST4W is enabled (v3.0 onward, this field becomes spectral spreading in milliHz)
where the summations are over k=0 to 161, as WSPR uses a 162-bit sync vector pr3 comprising a pseudorandom sequence of 0s and 1s. However, note that wsprdaemon records the raw value of sync_quality times 100 as an integer.
p0[k] to p3[k] are assumed to be the amplitudes of the four tones - amplitude because they are the square root of values with the same variable name. Below left is a histogram of sync_quality from 38,487 spots on 40m at G3ZIL 5-10 June 2020, and below right a scatterplot of sync_quality against SNR for the same data set, with non-parametric density contours at 10% intervals.
Flag, either 0 or 1. If 0 then either the Fano (default) or the stack (Jelinek, only if wsprd is called with option -J ) decoding algorithms have been used to decode. These algorithms can end without a decode being produced. If 1 then the Ordered Statistics Decoder (OSD) has been used. This decoder will always produce a decode - but of course it can be wrong. Therefore, wsprd only accepts an OSD decode if the tx_call it produces is already in the hash table having been decoded previously by the Fano or Jelinek algorithms.
Our conjecture: A measure of how well the incoming sync symbol sequence is synchronised to the sync vector sequence timed by the receiver's clock. The raw variable is on a scale of 0 to 1 (but was recorded as the integer of 10 times the raw value in V2.1), hence care needs to be taken if comparing across versions. From V2.2 the overall equation is:
A flag determined by user set options and the number of passes required to effect a decode. If wsprd is called with option -s (which it is not in wsprdaemon) this is the single pass (now very old) mode, so ipass can never be greater than 1, but (I think) can still be 1 if only a single pass was needed. If option -B was set (which it is not in wsprdaemon) then block demodulation is disables, only single-symbol noncoherent demodulation is used, and npass can take the values 1 or 2. Otherwise, npass may take a value of up to 3.
A count of the hard errors from the Fano (default) or the stack (Jelinek, only if wsprd is called with option -J ) decoding algorithm. Not yet clear what it means if the Ordered Statistics Decoder (OSD) has been used.
Graphs and text by Gwyn Griffiths G3ZIL