This article demonstrates detailed message analysis using SNIP’s RTCM3 Decoder dialog to determine what messages are present in a given connection stream.
Background
The primary reasons to select one stream over another are typically that it has the message content you seek and that it is suitably close to where you are operating. But how can you determine which streams have the message contents you require?
There are four standard ways to determine the message contents and the rates at which each is being sent for any given remote caster stream. They are:
- Use (trust?) what the Caster Table tells you about the stream.
- Use (trust?) what the Stream says about itself in Message Type 1013.
- Connect and read it yourself over a few minutes’ time using the decoder tables.
- Connect and read it yourself over a few minutes’ time using the console filters.
Of these, only the last two are entirely trustworthy, but errors and omissions in the other two are often simply annoying and are not fatal for using the subject stream. We consider each in turn here.
The Caster Table
The Caster Table was originally intended to be a machine generated table that could be used by Rover devices to select what streams to connect to. Information about each stream is represented by a single row of data. In practical fact the table is created by a combination of machine and human data entry and often has obvious errors in its data fields. As it is “just a guide” it does not affect the actual navigation results achieved. But as a SNIP owner/operator we urge you to take the time to make your own entries as correct as possible.
Hint: A good briefing on how to read a Caster Table is provided by rtcm-ntrip.org at this page and (more to the task at hand, the stream entries) at this page. For the actual RTCM SC-104 NTRIP standard, look here.
Hint: You can see the current caster table which your SNIP installation is sending in the console by pressing the Table button on the Casters and Clients tab. Alternatively you can see the caster table of most NTRIP Casters by typing the URL into a browser along with the port number, a string such as: www.igs-ip.net:2101
Two fields are of importance in this discussion–the 3rd and 4th entries in each row. These are often called the FORMAT and the FORMAT-DETAILS.
Note: One further sidebar remark: in the Caster Table the field delimiter is “;” so a semicolon may be used within a field for further element demarcation.
The first of these, the format, defines the overall protocol of the data. The precise list and spelling of valid strings to be used are not defined by the RTCM SC104 standard, so small variations do exist. Strings like “RTCM 3.0”, “RTCM 3.1”, or “RTCM3” all mean that RTCM revision 3.x message traffic will be found. Other common terms like “Raw” can be used to mean anything. In SNIP we add terms like “uBlox Raw” when it is helpful.
The next entry, the format-details, then defines what messages of this protocol are to be found. Here are some representative entries taken from actual Caster Tables to show the variations found.
- 1(1),2(30),3(10),18(1),19(1),22(10),23(10),24(10),31(1),32(10)
- “”
- 1004(1),1006(10),1008(10),1012(1),1033(10)
- 1004(1),1005/1007(5),PBS(10)
- 1004, 1006, 1012
- 1019(5),1020(5),1044(5),1046(5)
- 1004(1),1006(10),1008(10),1012(1),1013(10),1019(30),1020(30),1033(10)
Consider these entries a bit further. Entry#1 shows a caster sending 2.x type messages. Most stations sending this type of data send only message types 18 and 19. Entry#2 shows a caster entry where no message details are given at all. This practice is fairly common even among larger state-run networks. Entry#3 shows a typical entry where the message types and their rates are well laid out. Entry#4 shows a typical US West Coast entry where an additional non-std message “PBS” (Plate Boundary Observation) is claimed to be sent every 10th second (at this time no such message is sent). Also note the combination of two message types in this list (1005/1007). Entry#5 shows the list of messages sent but no time rates, a less common but valid use of the field (you will also find this pattern used when messages are sent at >1Hz). Entry#6 shows a stream sending only orbital data. Entry#7 shows a stream sending observational measurements and also sending orbital data, which is fairly uncommon.
From this data it is possible to select the best stream to be used.
It should also be noted in closing that many Caster Table entries do not provide correct Lat-Long values. Some virtual reference stations leave this data field empty (defaulting to 0,0), while other stations fail to recall that North America is located in the range of negative Longitude values. Many NTRIP Client tools (such as RTKLIB’s NTRIP Browser tool) allow you to display the above in a more readable table format and also to plot the reported locations on a base map.
The Contents of Message Type 1013
RTCM3 message type 1013 provides a way for the data stream to describe what messages and rates are being sent. Because the stream itself builds this message (technically NTRIP Server element does it), it does not suffer from the same problems of incorrect or missing data which the Caster Table often has.
The 1013 message is commonly used for two purposes. First, its header provides a neat and easy way to determine the offset between GPS and UTC time, 17 seconds at the time of this writing. It also provides the current Modified Julian Date (MJD) which is an interesting way to recover the current GPS week and convert to common time systems. Some operators send and use this message only for these purposes. [As a rule RTCM uses the current TOW as its working time stamp, so in order to represent an RTCM stream as a final absolute time, additional data is needed.]
Second, the 1013 message also allows sending information regarding other message content in the stream. For up to 31 additional message types, the 1013 message allows sending the period between subsequent messages (in 1/10th second units) and a synchronicity bit (used to indicate that the stated schedule time will not vary based on other traffic).
Here are some representative 1013 message contents, using SNIP’s RTCM3 Decoder dialog to obtain the data.
Decoded 1013 Type message contents for two streams,
One with message details, one without
When present (as many streams do not send it at all, and some streams do not fill out the 2nd part) the 1013 message represents the best way to determine the messages present and their sending rate
Reading the Stream Itself, decoder method
One definitive way to determine the stream’s content is by using SNIP’s RTCM3 Decoder dialog to obtain the data for a period of time (say a few minutes) and then simply observe how many messages of each type have been recorded. Using the same caster streams above, observe the red highlighted count values.
Summaries of Decoded Message Counts for two streams
Reading the Stream Itself, the filter method
Another definitive way to determine the stream’s content and the rate at which it occurs is to use the console logs time stamps in combination with a tightly selected filter to display the data. Proceed as follows:
Use the Parsed message setting and enable the “Show Message Types” control. Toggle the console display format with “Toggle the Label Style Used” (under the Types combo menu) until the time stamps are shown. Once in this mode, wait until two or more of the desired messages have appeared.
Disable the “Show Message Types” control to prevent further entries. Now Filter the console to display only the data stream of interest. First show “none” – then show only the one stream you desire. Scroll the console back (using the Refresh and Refresh-All buttons if needed) and read the times at which the message appeared. Subtract.
As a practical example, here are three entries for the batches of type 1019 message (GPS orbits) taken from the console log for the stream mountPt at: euref-ip.net : 2101 / ZIM20
[Thu 16:50:04]:[R04]:[ZIM20]: MountPt ZIM20 [R04], 18 messages decoded, of 1,643 so far. [Thu 16:50:04]:[R04]:[ZIM20]: RTCM type 1019 (067 bytes), the 148th message of this type found so far. [Thu 16:51:04]:[R04]:[ZIM20]: MountPt ZIM20 [R04], 18 messages decoded, of 1,822 so far. [Thu 16:51:04]:[R04]:[ZIM20]: RTCM type 1019 (067 bytes), the 165th message of this type found so far. [Thu 16:52:04]:[R04]:[ZIM20]: MountPt ZIM20 [R04], 18 messages decoded, of 2,001 so far. [Thu 16:52:04]:[R04]:[ZIM20]: RTCM type 1019 (067 bytes), the 182nd message of this type found so far.
As the above time stamps clearly show, the type 1019 messages are being sent once per minute on this stream. One can also deduce from the running message counts for this type that 17 messages are being sent in each batch, presumably because 17 SVs are in view in Zimmerwald , Switzerland where this reference station is located when this data was acquired.
This article has shown four basic ways to determine the contents of a stream and the rates at which each message type is being sent.
Note: The Universal Decoder can also be used to count basic RTCM3 message types (as well as other message types which may be interspersed within it). A planned improvement to the SNIP caster table logic is to use this ability to periodically re-evaluate its streams as a background task and from these measured counts update the caster table entries to ensure accuracy. This is currently targeted for a midsummer planned improvement.