Status and Streaming

[emiltin]

When there's a lot of status data to transmit, and it changes often, there is a need to ensure that the transfer is efficient and does not incur too much overhead.

In RSMP 3, this was the background for introducing status messages like S0001 that provides the status of all signal groups, in a single message:
https://rsmp-nordic.github.io/rsmp_sxl_traffic_lights/1.2.1/sxl_traffic_light_controller.html#s0001

For RSMP 4, the component path is included in the topic path:

status/sensor/3/45fe/dl/1   

dl/1 address detector logic 1. The payload would include the status for that component, eg:

{"count": 5}

We could address all detector logics using dl. but would need a way to structure the payload data. e.g:

{
  "counts": {
    "1": 5,
    "2": 20 
  }
}

Streaming

The basic concept of streaming is just that you get data bit by bit in real-time, as opposed to downloading a complete data set. In this sense, all RSMP data is streaming.

But there are some concepts other types of streaming, like video and audio, that we could be inspired by. A keyframe in video streaming include the full picture. Between keyframes, smaller delta-updates are send. Keyframes makes sure that the stream does not corrupt over time, and also enables random access, fast-forward, etc.

In RMSP we could use this concept to send keyframe that includes the full status, and then delta updates that only include changes. The keyframe might be retained, while delta would not. That way, when you connect you get the latest full status, and after that you receive delta. We might need to number delta updates, so nodes an check if deltas are missing/skipped.

One way to look at this is that our traditional status messages are the keyframes. We could then introduce a new delta message that only includes changes. Example:

status/tlc/1/sg {"v":"aaaaaaaaaaaaaaaaa"}    # all groups are in state "a"
delta/tlc/1/sg/5 {"v":"b"}                              # group 5 is now in state "b"
delta/tlc/1/sg/1,6,9 {"v":{5:"bbb"}            # group 1,6 and 9 are now in state "b"
delta/tlc/1/sg/5 {"v": "c" }                    # group 5 is now in state "c"
status/tlc/1/sg {"v":"ccccccccccccccccc"}        # all groups are in state "c"
delta/tlc/1/sg/5-8 {"v":{5:"bbb"}           # group 5,6,7 and 8 are now in state "b"

Retaining values

It's important that status can be retained, so you get quickly get the latests know status when you you connect to the broker, even for devices that are currently offline.

If we have overlapping status messages that cover the same data, we need to ensure that they are received in the correct order, but perhaps it's simple to ensure we don't have overlapping messages. Overlapping messages can occur if a single message can include data from multiple components.

[emiltin]

A simplification would be to allow consecutive series to be send while still just sending the intial index, instead of using ranges:

Instead of:

delta/tlc/1/sg/5-7         payload: "abc"                        # 5=a, 6=b, 7=c

You could just use:

delta/tlc/1/sg/5         payload: "abc"                        # 5=a, 6=b, 7=c

It could be combine with lists:

delta/tlc/1/sg/5,10      payload: ["ab","cd"]                  # 5=a, 6=b, 10=c,11=d

If we want to send a single string with a compact delta with only the groups that changed, we could use escape codes. For examp,e, (n) could mean skip n groups, while {n} could mean repeat the previous status n times.

"(7)A{5}B"

Explanation:
(7): Groups 1-7 did not change
A{5}: Group 8 changed to A and so did the following 5 groups (9-13).
B: Group 14 changed to B.