The Horus Binary v2 format expands the payload ID space from 8 to 16-bits (for 65535 possible payload IDs), and allows for 9 bytes of user-customisable payload data. How these 9 bytes are used is up to the payload designer, but needs to be described in a way that allows the Horus Binary decoder to interpret the bytes, and produce a UKHAS-compliant telemetry sentence suitable for uploading to the Habitat database (and hence displayed on the HabHub tracker).
Horus Binary v2 Packet Format Re-cap
The Horus Binary v2 packet format (prior to forward-error correction) is as follows:
| Byte No. | Data Type | Size (bytes) | Description |
|---|---|---|---|
| 0 | uint16 | 2 | Payload ID (0-65535) |
| 2 | uint16 | 2 | Sequence Number |
| 4 | uint8 | 1 | Time-of-day (Hours) |
| 5 | uint8 | 1 | Time-of-day (Minutes) |
| 6 | uint8 | 1 | Time-of-day (Seconds) |
| 7 | float | 4 | Latitude |
| 11 | float | 4 | Longitude |
| 15 | uint16 | 2 | Altitude (m) |
| 17 | uint8 | 1 | Speed (kph) |
| 18 | uint8 | 1 | Satellites |
| 19 | int8 | 1 | Temperature (deg C) |
| 20 | uint8 | 1 | Battery Voltage |
| 21 | ??? | 9 | Custom data |
| 30 | uint16 | 2 | CRC16-CCITT Checksum |
Note that while the majority of the packet format is fixed, there are 9 bytes available to the user to include data from other sensors. 9 bytes might not sound like a lot, but if you're conservative with your data types, you can fit quite a lot in here!
Interpreting the Custom Data Section
The information necessary for interpreting the custom data section is contained with custom_field_list.json. Each payload making use of the custom data area needs to have an entry in this file (matching the callsign allocated in payload_id_list.txt), otherwise the custom data area is ignored when packet parsing is performed.
An example entry in this file is shown below:
"4FSKTEST-V2": {
"comment": "Default custom fields for RS41ng",
"struct": "<hhBHxx",
"fields": [
["ascent_rate", "divide_by_100"],
["ext_temperature", "divide_by_10"],
["ext_humidity", "none"],
["ext_pressure", "divide_by_10"]
]
},
The entry is named with the payload's callsign (in this case 4FSKTEST-V2), which must match the entry in payload_id_list.txt.
If no matching callsign entry is found, the metadata for the 4FSKTEST-V2 callsign will be used, which contains the default custom telemetry fields transmitted by the RS41ng firmware. If you are happy with these fields, then you do not need to take any further action.
struct
The struct entry tells the decoder how to interpret the 9 bytes, and is a Python struct 'format string'. The text <BbBfH might look a bit arcane, but is a shorthand way of describing packed binary data types.
The first character indicates the endianness of the multi-byte data types used (e.g. big-endian = most-significant or little-endian = least-significant byte sent first). < represents little-endian, and > represents big-endian.
Some common data types that we would expect within high-altitude balloon telemetry are represented by the following characters:
B- Unsigned 8-bit integer (0 - 255)b- Signed 8-bit integer (-128 - 127)H- Unsigned 16-bit integer (0-65535)h- Signed 16-bit integer (-32768 - 32767)f- 32-bit Floating-point number (printed out to 6 decimal places)x- A pad byte - not interpreted.
For the 4FSKTEST-V2 example, the string is indicating that the 9 bytes contain:
<- Little-endian ordered data (for multi-byte data types - common on most modern platforms)h- Bytes 1-2 are a signed 16-bit integer (-32768 - 32767)h- Bytes 3-4 are a signed 16-bit integer (-32768 - 32767)B- Byte 5 is a unsigned 8-bit integer (0 - 255).H- Bytes 6-7 are a unsigned 16-bit integer (0-65535)xx- Bytes 8-9 are unused.
So as an example, if the custom byte area contained the data 12014AFE0048040000, this would be interpreted as values (274, -438, 0, 1096). These values are then further adjusted using the field post-processing described below.
fields
While the struct entry tells us how to split the bytes into different fields, we would like to provide some information as to what each of the fields are, and potentially do some post-processing of the field data.
For each field that is decoded by the struct definition, we need to have an entry in the fields section. Each entry consists of a field name (e.g. 'external_temperature'), and a value with indicates if any additional post-processing is done to the data. The following values are supported:
none- No additional post-processing of the value. The value is converted to a string using standard Python type conversion (e.g.str(value))battery_5v_byte- Interpret an unsigned 8-bit integer as a battery voltage, where 0 = 0v, 255 = 5.0V, with linear steps in between.divide_by_10- Interpret an integer (signed or unsigned, 8-bit or larger) as a fixed-point value, and divide it by 10. (e.g. 123 -> 12.3)divide_by_100- Interpret an integer (signed or unsigned, 8-bit or larger) as a fixed-point value, and divide it by 100. (e.g. 123 -> 1.23)
Additional post-processing functions can be added if necessary - these would need to be added to delegates.py - please let me know what you need!
UKHAS Sentence Generation
Each of the defined fields are appended onto the UKHAS-compliant sentence generated by the decoder. A sentence without any custom fields would look something like this:
$$HORUS-V2,700,06:42:14,-34.00000,138.00000,100,0,10,40,3.10*CRC16
This is the same format produced when decoding a Horus Binary v1 packet.
With the addition of the custom fields in the example above, it might look like:
$$HORUS-V2,630,01:29:44,-34.35389,139.96246,16244,66,10,-9,1.31,2.74,-43.8,0,109.6*CRC16
You will need to define an appropriate Habitat payload document for the field names to show up correctly on the HabHub tracker.
Testing Packet Parsing
Once you have an example data packet (represented as hexadecimal - e.g. copied out of the 'Latest Packet (Raw)' field in horus-gui), you can test decoding of it using some command-line utilities provided with horusdemodlib.
First up, make your modifications to the payload_id_list.txt (if necessary - ideally you would have already had a payload ID allocated) and custom_field_list.json file.
Assuming you have the python horusdemodlib library installed, you can run:
$ python -m horusdemodlib.decoder --decode 00015F000C223800000000000000000000000000000152069E3FC87BD20429BE
2021-09-11 22:16:48,099 INFO: Using existing payload/custom-field files.
Decoded UKHAS String: $$4FSKTEST-V2,95,12:34:56,0.00000,0.00000,0,0,0,0,0.00,1,1.234568,3.92,12.3,12.34*BBDB
Replace the hex with your payload. If all goes well, you will get the UKHAS string printed to the terminal like in the example above.
You can also run up horus-gui in a mode where it does not attempt to download the latest Payload ID List and Custom Field List from github, by using:
$ python horus-gui.py --payload-id-list my_payload_id_list.txt --custom-field-list my_custom_field_list.json
Requesting an Update to horusdemodlib
Once you have a working custom field description section, please raise an issue on this repository with the information. It will then get reviewed (I'm usually pretty quick with this!) and added into the repository. These updates will then be automatically downloaded by all horus-gui and horusdemodlib users when they next start the software.