Replies: 12 comments 11 replies
-
DiscoveryHow do a supervisor discover new devices? ConnectIn RSMP 3, the device connect to the supervisor. With MQTT the device still connect to a central server, but it's the broker, rather than the supervisor. Supervisor can subscribe to For example, let's assume a device with 65a3 connects. It will then publish to the topic graph LR;
A[Device 65a3]-->|hello/65a3| Broker;
Broker-->|hello/65a3| Supervisor;
DisconnectMQTT has something called Last Will and Testament (LWT), where the broker sends a predefined message, if a client disconnects: You can choose the keep-alive-time, etc. We could use this to make sure a device publishes to graph LR;
A[Device 65a3];
Broker-->|died/65a3| Supervisor;
If the supervisor is disconnected. the broker can retain messages send be devices. I think this means discovery and disconnects should be handled transparently, even if the supervisor goes offline, or devices are connected before the supervisor. |
Beta Was this translation helpful? Give feedback.
-
|
A graceful disconnect (e.g. if the device chooses to power down to to a low battery, or a manual shutdown) can be handled by the device posting to a 'bye' channel: graph LR;
A[Device 65a3]-->|bye/65a3| Broker
Broker-->|bye/65a3| Supervisor;
|
Beta Was this translation helpful? Give feedback.
-
|
If hello, bye and died are retained, you get might get more than one of these when the supervisor reconnects. It's probably better to merge hello/bye/died into single Device comes online, devices sends state: graph LR;
A[Device 65a3]-->|state/65a3: 1| Broker;
Broker-->|hello/65a3: 1| Supervisor;
Device goes offline, and the last will is send by the broker: graph LR;
A[Device 65a3]
Broker-->|state/65a3: 0| Supervisor;
|
Beta Was this translation helpful? Give feedback.
-
HandshakeEverything related to low-level connection is handled by MQTT. But we will still have to handle things like:
MQTT allows you to include payload (any data) as part of the connect message. This could perhaps by used for some of the above, although large data should maybe be split info other messages/topics. When connecting, a Keep Alive interval is specified in seconds. When there is no message to send, the client will periodically send a heartbeat message to the broker according to the value of Keep Alive to ensure that the broker will not disconnect the connection. This would be handled by MQTT library used by in device. After the connection is established successfully, if the broker does not receive any packets from the client within 1.5 times of Keep Alive, it will consider that there is a problem with the connection with the client, and the broker will disconnect from the client. The Last Will is also send as part of the connect message. |
Beta Was this translation helpful? Give feedback.
-
CommandThe supervisor must be able to send commands to specific devices. All devices subscribe to a command topic, which the supervisor can publish to. Depending on how we design it, this would also make it easy to send the same command to all device, if there's a use for this. Device subscribe to The payload would contain the parameters, e.g. in JSON format. For example, a traffic light with id 45fe might subscribe to all command for all components using either single level wildcards with: Or using a multi-level wildcard: Now the supervisor can send commands to this particular traffic light and component by publishing to Suppose the traffic light has these components:
And let's suppose the traffic light supports these modules:
We could also choose to be selective about command modules we listen to by subscribing to specific components and modules: Change signal plan
We send the command to the The payload would include the arguments, like what plan to switch to. Set detector sensibility
We send the command to the The payload would include the arguments, like the sensibility value. Sending to many devicesA traffic light with id 45fe could subscribe to a topic with the id 'all': Or a bit more selectively: The supervisor can publish to this topic to e.g. change the signal plan on all traffic lights at the same: Or change the detector sensibility on all detector logics in a particular traffic lights: Or change the detector sensibility on all detector logics in all traffic lights: |
Beta Was this translation helpful? Give feedback.
-
|
It's worth noting device ids must be unique, otherwise you cannot address them individually. This is probably assumed in any case. |
Beta Was this translation helpful? Give feedback.
-
|
You could also send all command to the same topic, e.g. |
Beta Was this translation helpful? Give feedback.
-
|
How would command responses be handled? We would use the request-response pattern, which is based on Response Topics. When you send a command, you pass a topic that would want to response to be published to. A supervisor sending a command could pass the response topic: When a supervisor with id 22ba changing plan with All supervisor can receive response if the just subscribe to |
Beta Was this translation helpful? Give feedback.
-
StatusDevices could publish status to: Where: For example, suppose a traffic sensor with the id 12d9 sends traffic counts once per minut, using a component 'dl1', a status called 'volume' in a module called 'traffic': The payload would contain the actual data. Is anyone listening?One of the ideas of MQTT is that clients don't have to know anything about each other. They just need the address of the broker, and some topic to publish/subscribe to. This means that a sensor, e.g. a thermometer, starts publishing data as soon as it connects to the broker, regardless of whether anyone is listening. A benefit of this is that the broker can be set up to persists data to a database, regardless of which clients (supervisors) are connected. This could also be an issue if the device can deliver the data in several formats, aggregation intervals, etc. Instead of having to publish in all formats, only what's asked by supervisor should be send. In RSMP 3 you can set the status interval, either a fixed interval, or send on change. Some settings might result if high data bandwidth, like signal group status for all groups every second, or some traffic sensor that has a lot of very frequent changes. How could this be handled in MQTT? It should be possible to use topics for building our own subscription mechanism on top of MQTT if we need to. Subscriber count: Devices could subscribe to a 'subscribe' channel, and supervisor would post to them when they want data. Timed: Supervisors could also inform a device that it would like data to be send for some interval. A supervisor would have to ask for more before the current interval times out. If the supervisor dies, the device will then stop sending data at some point. Update intervalsIf we want a supervisor to be able to subscribe to data at arbitrary intervals, device could publish to topics that include the update interval in the topic name e.g an interval of once per minute: A device could perhaps offer some fixed interval that you can use, e.g. 1s, 10s, 1m, 10m, 1h, 24h SubscriptionsIf we need to provide different data stream to each supervisor, another approach would be for each supervisor to subscribe to a topic that include the id of the supervisor. The supervisor informs the device about what data it wants, and the device then starts publishing to a topic that includes the id of the supervisor, which only that particular supervisor reads. Example: A traffic sensor with the id 6b41 can deliver traffic speeds and other metrics. The device subscribes to: These topics are used to receive status subscribe/unsubscribe requests. A supervisor with id 299c wants to receive traffic volumes once per 10s. The device receives the request, and starts publishing every 10s to When the supervisor does not want to receive data anymore, it send a message to If the supervisor dies ungracefully, a Last Will can be send to
|
Beta Was this translation helpful? Give feedback.
-
|
A question is whether a device should be able to provide data in several different aggregations at the same time (to different supervisor) or just one. Or whether we want to move to model where devices send just raw data, and aggregation is handled later in the chain. |
Beta Was this translation helpful? Give feedback.
-
AlarmsDevices could publish to Where: Supervisors could subscribe to all alarm from all devices using: Or if you only want certain modules of alarms: For example, a traffic light bb35 that has a hardware error might publish to: |
Beta Was this translation helpful? Give feedback.
-
|
As long as each alarm is on a separate topic (including the alarm code), a retained message can be kept for each alarm. When a supervisor connects it will then receive a message for each alarm, with the latest status. |
Beta Was this translation helpful? Give feedback.
-
|
The alarm status (active, inactive, etc) would be part of the payload. |
Beta Was this translation helpful? Give feedback.
-
|
Alarms can be retained, so that a supervisor that goes offline and then reconnects receives that latest status of all alarms immediately. Or we can think if letting devices know when a supervisor comes online (a kind of inverse discovery), so device can then send something to that supervisor. |
Beta Was this translation helpful? Give feedback.
-
|
We need to look at how alarm states and blocking would be handled. |
Beta Was this translation helpful? Give feedback.
-
MQTT Request-Response PatternFrom https://www.hivemq.com/blog/mqtt5-essentials-part9-request-response-pattern/:
Response TopicWhen you publish, you can provide a response topic, which the receiver can use to send a response to. Correlation DataCorrelation data helps you identify which request a later received response relates to. If you send several commands, and receive responses, you can tell which command a response relates to. |
Beta Was this translation helpful? Give feedback.
-
Topic listA summary of topics |
Beta Was this translation helpful? Give feedback.
-
|
How are MQTT client ids used? Would we use them? |
Beta Was this translation helpful? Give feedback.
-
|
You can choose the client id when you connect to start the mqtt libary and connect to the broker. This can be used by the broker when authorizing which topics the client is allowed to publish to. But this client id is not send as part of the message, so when you receive a message, the client id of the sender is not included. This is keeps the header short, and decouples clients. However, you can add the sender to the payload if you need it, or you can rely on topic that include unique ids. For example a supervisor_742c that send to tlc_582a, can specify that responses are send to:
Anything received on this topic can be assumed to be from tlc_582a. |
Beta Was this translation helpful? Give feedback.
-
Retained MessageThe last message published to a topic can be stored by the broker. As soon as someone subscribes to the topic, they received the message immediately: This could be used for e.g. AggregatedStatus. |
Beta Was this translation helpful? Give feedback.
-
Aggregated StatusWe would perhaps consider this a status like any other, just defined to indicate the overall status of all components. It would be linked to the main component. Eg:
An option would also be to split each aggregated status part into topics: |
Beta Was this translation helpful? Give feedback.
-
Message PrioritiesMQTT does not have any way to prioritize the delivery of some message over others. All message are real-time, and queues on the broker are FIFO. So if we want to send huge amounts of data, but still want to provide live updates during the transmission, we should split the data delivery into smaller messages, so that live updates can be send in between. This is similar to we would handle it with RSMP 3 on TCP. QUICQUIC allows multiple data stream which can be prioritized, as far as I understand, see https://http3-explained.haxx.se/en/quic/quic-streams#stream-prioritization. EMQX 5 supports QUIC, and according to https://www.emqx.com/en/blog/mqtt-over-quic, this allow for:
Also: Combining QUIC features with IoT scenarios, we have planned many features for MQTT over QUIC, such as topic prioritization by differentiating control channels, non-reliable real-time streaming for high-frequency data transfer scenarios, and flexible topic and data channel (Stream) mapping to reduce interference between topics. These will may be presented in future releases depending on the feedback from the community and our customers. |
Beta Was this translation helpful? Give feedback.
-
Site-to-site communicationSite-to-site communication is possible with the current RSMP 3, and has been used in a few cases to coordinate traffic lights, for green wave corridors. Sites have a direct connection to each other. One is the master and sends commands to active inputs/outputs on the other sites. Or a site subscribes to a status from another site. With MQTT all sites can easily communication through the broker. MQTT itself has no concept of sites and supervisors - it's all just clients. A site could send a command to another sites, just like a supervisor does. Things be note:
|
Beta Was this translation helpful? Give feedback.
-
MQTT
MQTT is a popular protocol for IoT devices. It uses a publish-subscribe model with a broker:
https://mqtt.org/
It uses topics as a flexible way to route and filter message, see e.g:
https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/
Here I want to look at how existing RSMP use cases would be handled with MQTT
Broker
In MQTT, clients all connect to the broker, which routes messages according to which topics are used when subscribing and publishing:
graph LR; A[Client]---Broker; B[Client]---Broker; Broker---C[Client]; Broker---D[Client];Clients can communicate with all other clients, as long as they agree on which topics to use.
There is no inherent idea of a client vs. supervisor side in MQTT, it's all just clients. But for RSMP we would still use the concept of devices and supervisors:
graph LR; A[Device]---Broker; B[Device]---Broker; Broker---C[Supervisor]; Broker---D[Supervisor];Note that devices can talk to each other, and supervisors can also talk to each other (depending on how to design topic).
Topics
From https://www.hivemq.com/blog/mqtt-essentials-part-4-mqtt-publish-subscribe-unsubscribe/
MQTT uses the topic name as a fundamental concept. It structures this name hierarchically using forward slashes as delimiters and creates a simple string. It’s similar to a URL path but without the protocol and domain components. MQTT topics are used to label messages and provide a way for clients to subscribe to specific messages.
For example, a device that measures temperature might publish its readings to the topic "sensors/temperature/livingroom". A client interested in these readings can subscribe to this topic and receive updates as they’re published.
MQTT provides two types of wildcards to use with topic subscriptions:
Beta Was this translation helpful? Give feedback.
All reactions