From 6a6f054da01d33b6199d3174e432dca49105aa54 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 9 Mar 2023 11:36:32 -0800 Subject: [PATCH 01/37] Initial setup --- docs/articles/client-server.md | 9 +++++++++ docs/articles/toc.yml | 4 +++- 2 files changed, 12 insertions(+), 1 deletion(-) create mode 100644 docs/articles/client-server.md diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md new file mode 100644 index 0000000..bacb6df --- /dev/null +++ b/docs/articles/client-server.md @@ -0,0 +1,9 @@ +--- +uid: client-server +title: "Client-Server" +--- + +Client-Server Tutorial +====================== + +# Introduction \ No newline at end of file diff --git a/docs/articles/toc.yml b/docs/articles/toc.yml index a6625ae..fddbdba 100644 --- a/docs/articles/toc.yml +++ b/docs/articles/toc.yml @@ -7,4 +7,6 @@ - href: push-pull.md - href: proxy.md - name: Recipes -- href: recipes.md \ No newline at end of file + href: recipes.md +- name: Client-Server Tutorial + href: client-server.md \ No newline at end of file From 04b683cd3c2e948dbab17d01072c0b09b8b89c14 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 9 Mar 2023 17:27:48 -0800 Subject: [PATCH 02/37] modify docgf.json for API build --- docs/articles/client-server.md | 32 +++++++++++++++++++++++++++++++- docs/docfx.json | 2 +- 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index bacb6df..66b5de2 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -6,4 +6,34 @@ title: "Client-Server" Client-Server Tutorial ====================== -# Introduction \ No newline at end of file +The Bonsai.ZeroMQ package allows us to harness the powerful [ZeroMQ](https://zeromq.org/) library to build networked applications in Bonsai. Applications could include: +- Interfacing with remote experimental rigs via network messages +- Performing distributed work across pools of machines (e.g. for computationally expensive deep-learning inference) +- Streaming video data between clients across a network +- **Real-time interaction between clients in a multiplayer game** + +In this article, we will use Bonsai.ZeroMQ to explore this final example and build a basic client-server architecture similar to one that might be used in a multiplayer game. + +## Network design +The basic network architecture we want to achieve will be composed of a number of clients sending their state to a server, which then updates the other connected clients with that clients’ state. This is comparable to a multiplayer game in which client players move through the game world and must synchronise that movement via a dedicated server so that all players see each other in their ‘true’ position in the world. + +```mermaid +sequenceDiagram + actor Client1 + actor Client2 + actor Client3 + participant Server + Client1->>Server: MOVE + Server->>Client2: SYNC + Server->>Client3: SYNC +``` + +An important requirement to point out here is that our server should be choosy about which clients it broadcasts information to. If client 1 updates the server with its current state, that information needs to be sent to all other connected clients, but there is no need to send it back to client 1 as it already knows its own state and this feedback message would be redundant. + +ZeroMQ provides a number of socket types that could be used to achieve something approaching this architecture. The Router / Dealer socket pair acting as Server / Client has a couple of advantages for this design: +- Routers assign a unique address for each connected client allowing clients in turn to be addressed individually +- Messages can be passed between Router / Dealer sockets without the requirement that a reply is received before the next message is sent, as is the case with the Request / Response socket pair. + +## Basic client +To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project, add a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node. In the node properties, set Host: localhost, `Port`: 5557, `SocketConnection`: Connect, `SocketProtocol`: TCP. + diff --git a/docs/docfx.json b/docs/docfx.json index 7caceec..e86b638 100644 --- a/docs/docfx.json +++ b/docs/docfx.json @@ -4,7 +4,7 @@ "src": [ { "files": [ - "Bonsai.ZeroMQ/*.csproj" + "Bonsai.ZeroMQ/Bonsai.ZeroMQ.csproj" ], "src": "../src" } From 4e9aa4c58fc475123b043b56230202a144d508c3 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 9 Mar 2023 17:42:17 -0800 Subject: [PATCH 03/37] Basic client text --- docs/articles/client-server.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 66b5de2..d060379 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -35,5 +35,9 @@ ZeroMQ provides a number of socket types that could be used to achieve something - Messages can be passed between Router / Dealer sockets without the requirement that a reply is received before the next message is sent, as is the case with the Request / Response socket pair. ## Basic client -To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project, add a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node. In the node properties, set Host: localhost, `Port`: 5557, `SocketConnection`: Connect, `SocketProtocol`: TCP. +To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project, add a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node. In the node properties, set `Host`: localhost, `Port`: 5557, `SocketConnection`: Connect, `SocketProtocol`: TCP. + +In Bonsai.ZeroMQ, the **`Dealer`** can have two functions based on its inputs. On its own, as above, the **`Dealer`** node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our **`Dealer`** to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. + +If we add inputs to the **`Dealer`**, it will act as both a sender and receiver of messages on the network. Before the **`Dealer`** node add a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown), [**`String`**](xref:Bonsai.Expressions.String), and [**`Format (Osc)`**](xref:Bonsai.Osc.Format) node in sequence. From 4d9f548587cf6a645b4727c0a28dc8278b58c383 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 9 Mar 2023 18:07:06 -0800 Subject: [PATCH 04/37] Add example workflow --- docs/articles/client-server.md | 5 +- docs/docfx.json | 17 +++++-- docs/workflows/dealer-basic-input.bonsai | 31 ++++++++++++ .../dealer-basic-input.bonsai.layout | 49 +++++++++++++++++++ 4 files changed, 98 insertions(+), 4 deletions(-) create mode 100644 docs/workflows/dealer-basic-input.bonsai create mode 100644 docs/workflows/dealer-basic-input.bonsai.layout diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index d060379..4523a78 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -39,5 +39,8 @@ To begin with, we’ll create a simple client that sends basic messages on a net In Bonsai.ZeroMQ, the **`Dealer`** can have two functions based on its inputs. On its own, as above, the **`Dealer`** node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our **`Dealer`** to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. -If we add inputs to the **`Dealer`**, it will act as both a sender and receiver of messages on the network. Before the **`Dealer`** node add a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown), [**`String`**](xref:Bonsai.Expressions.String), and [**`Format (Osc)`**](xref:Bonsai.Osc.Format) node in sequence. +If we add inputs to the **`Dealer`**, it will act as both a sender and receiver of messages on the network. Before the **`Dealer`** node add a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) and [**`String`**](xref:Bonsai.Expressions.String) node in sequence as input to the **`Dealer`**. +:::workflow +![Basic Dealer input](~/workflows/dealer-basic-input.bonsai) +::: \ No newline at end of file diff --git a/docs/docfx.json b/docs/docfx.json index e86b638..8b67570 100644 --- a/docs/docfx.json +++ b/docs/docfx.json @@ -6,7 +6,16 @@ "files": [ "Bonsai.ZeroMQ/Bonsai.ZeroMQ.csproj" ], - "src": "../src" + "src": "../src", + "exclude": [ + "**/bin/**", + "**/obj/**", + "**/**.Tests/**", + "**/Bonsai.NuGet**/**", + "**/Bonsai.Configuration/**", + "**/Bonsai.StarterPack**/**", + "**/Bonsai.Templates/**" + ] } ], "dest": "api", @@ -66,7 +75,7 @@ "_gitContribute": { "repo": "https://github.com/bonsai-rx/zeromq", "branch": "main", - "apiSpecFolder": "docs/apidoc" + "apiSpecFolder": "apidoc" } }, "output": "_site", @@ -84,7 +93,9 @@ ] }, "xref": [ - "https://bonsai-rx.org/docs/xrefmap.yml", + "https://horizongir.github.io/opencv.net/xrefmap.yml", + "https://horizongir.github.io/ZedGraph/xrefmap.yml", + "https://horizongir.github.io/opentk/xrefmap.yml", "https://horizongir.github.io/reactive/xrefmap.yml" ] } diff --git a/docs/workflows/dealer-basic-input.bonsai b/docs/workflows/dealer-basic-input.bonsai new file mode 100644 index 0000000..46d1e6a --- /dev/null +++ b/docs/workflows/dealer-basic-input.bonsai @@ -0,0 +1,31 @@ + + + + + + + A + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/dealer-basic-input.bonsai.layout b/docs/workflows/dealer-basic-input.bonsai.layout new file mode 100644 index 0000000..f87e23c --- /dev/null +++ b/docs/workflows/dealer-basic-input.bonsai.layout @@ -0,0 +1,49 @@ + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 541 + 280 + + + 334 + 356 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + true + + 543 + 119 + + + 334 + 156 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + \ No newline at end of file From 3ba8a5c98be35c5c02cc114d604a626096c7f187 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Mar 2023 12:15:50 +0000 Subject: [PATCH 05/37] Clients and router message parsing --- docs/articles/client-server.md | 25 ++- docs/docfx.json | 18 +- docs/workflows/dealer-basic-input.bonsai | 2 +- docs/workflows/dealer-basic-input.svg | 3 + docs/workflows/multiple-clients.bonsai | 67 ++++++ docs/workflows/multiple-clients.bonsai.layout | 113 ++++++++++ docs/workflows/multiple-clients.svg | 3 + docs/workflows/router-message-parsing.bonsai | 94 ++++++++ .../router-message-parsing.bonsai.layout | 206 ++++++++++++++++++ docs/workflows/router-message-parsing.svg | 3 + 10 files changed, 516 insertions(+), 18 deletions(-) create mode 100644 docs/workflows/dealer-basic-input.svg create mode 100644 docs/workflows/multiple-clients.bonsai create mode 100644 docs/workflows/multiple-clients.bonsai.layout create mode 100644 docs/workflows/multiple-clients.svg create mode 100644 docs/workflows/router-message-parsing.bonsai create mode 100644 docs/workflows/router-message-parsing.bonsai.layout create mode 100644 docs/workflows/router-message-parsing.svg diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 4523a78..415d4eb 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -35,7 +35,7 @@ ZeroMQ provides a number of socket types that could be used to achieve something - Messages can be passed between Router / Dealer sockets without the requirement that a reply is received before the next message is sent, as is the case with the Request / Response socket pair. ## Basic client -To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project, add a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node. In the node properties, set `Host`: localhost, `Port`: 5557, `SocketConnection`: Connect, `SocketProtocol`: TCP. +To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project, add a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node. In the `ConnectionString` property, set `Address`: localhost:5557, `Action`: Connect, `Protocol`: TCP. In Bonsai.ZeroMQ, the **`Dealer`** can have two functions based on its inputs. On its own, as above, the **`Dealer`** node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our **`Dealer`** to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. @@ -43,4 +43,25 @@ If we add inputs to the **`Dealer`**, it will act as both a sender and receiver :::workflow ![Basic Dealer input](~/workflows/dealer-basic-input.bonsai) -::: \ No newline at end of file +::: + +In the node properties, set the **`KeyDown`** `Filter` to the ‘1’ key and set the **`String`** `Value` to ‘Client1’. If we run the Bonsai project now, the **`Dealer`** will continue listening for incoming messages on the network, but every time the ‘1’ key is pressed a message containing the string ‘Client1’ will be sent from the socket. + +Copy and paste this client structure a couple of times and change the **`KeyDown`** and **`String`** properties accordingly on each (2, ‘Client2’; 3, ‘Client3’) so that we have 3 total clients that send messages according to different key presses. + +:::workflow +![Multiple clients](~/workflows/multiple-clients.bonsai) +::: + +> For the purposes of this article we are creating all of our clients and our server on the same Bonsai project and same machine for ease of demonstration. In a working example, each client and server could be running in separate Bonsai instances on different machines on a network. In this case, localhost would be replaced with the server machine’s IP address. + +## Basic server +Now that we have our client pool set up and sending messages, let’s implement a server to listen for those messages. Add a [**`Router`**](xref:Bonsai.ZeroMQ.Router) node to the project and set its properties to match the **`Dealer`** sockets we already added so that it is running on the same network. As the **`Router`** is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. + +As with the **`Dealer`** node, a **`Router`** node without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the **`Router`** node, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the **`Router`**. Expanding the output the the **`Router`**, we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. To make sense of the message, let's expose the `Buffer` `byte[]` of the `First` frame. Add an **`Index (Expressions)`** node the the first frame buffer and set its `Value` to 1 to access the unique address ID. Add a [**`ConvertToString`**](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. + +:::workflow +![Router message parsing](~/workflows/router-message-parsing.bonsai) +::: + +Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the **`Index`** node output and a corresponding `string` in the **`ConvertToString`** node output. \ No newline at end of file diff --git a/docs/docfx.json b/docs/docfx.json index 8b67570..9681da5 100644 --- a/docs/docfx.json +++ b/docs/docfx.json @@ -6,16 +6,7 @@ "files": [ "Bonsai.ZeroMQ/Bonsai.ZeroMQ.csproj" ], - "src": "../src", - "exclude": [ - "**/bin/**", - "**/obj/**", - "**/**.Tests/**", - "**/Bonsai.NuGet**/**", - "**/Bonsai.Configuration/**", - "**/Bonsai.StarterPack**/**", - "**/Bonsai.Templates/**" - ] + "src": "../src" } ], "dest": "api", @@ -73,9 +64,8 @@ "_enableNewTab": true, "_enableSearch": true, "_gitContribute": { - "repo": "https://github.com/bonsai-rx/zeromq", "branch": "main", - "apiSpecFolder": "apidoc" + "apiSpecFolder": "docs/apidoc" } }, "output": "_site", @@ -93,9 +83,7 @@ ] }, "xref": [ - "https://horizongir.github.io/opencv.net/xrefmap.yml", - "https://horizongir.github.io/ZedGraph/xrefmap.yml", - "https://horizongir.github.io/opentk/xrefmap.yml", + "https://bonsai-rx.org/docs/xrefmap.yml", "https://horizongir.github.io/reactive/xrefmap.yml" ] } diff --git a/docs/workflows/dealer-basic-input.bonsai b/docs/workflows/dealer-basic-input.bonsai index 46d1e6a..80ca89e 100644 --- a/docs/workflows/dealer-basic-input.bonsai +++ b/docs/workflows/dealer-basic-input.bonsai @@ -8,7 +8,7 @@ - A + D1 false diff --git a/docs/workflows/dealer-basic-input.svg b/docs/workflows/dealer-basic-input.svg new file mode 100644 index 0000000..31094d4 --- /dev/null +++ b/docs/workflows/dealer-basic-input.svg @@ -0,0 +1,3 @@ + +]>DealerStringKeyDown \ No newline at end of file diff --git a/docs/workflows/multiple-clients.bonsai b/docs/workflows/multiple-clients.bonsai new file mode 100644 index 0000000..3b54fef --- /dev/null +++ b/docs/workflows/multiple-clients.bonsai @@ -0,0 +1,67 @@ + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/multiple-clients.bonsai.layout b/docs/workflows/multiple-clients.bonsai.layout new file mode 100644 index 0000000..1cd0cc8 --- /dev/null +++ b/docs/workflows/multiple-clients.bonsai.layout @@ -0,0 +1,113 @@ + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/multiple-clients.svg b/docs/workflows/multiple-clients.svg new file mode 100644 index 0000000..2471822 --- /dev/null +++ b/docs/workflows/multiple-clients.svg @@ -0,0 +1,3 @@ + +]>DealerDealerDealerStringStringStringKeyDownKeyDownKeyDown \ No newline at end of file diff --git a/docs/workflows/router-message-parsing.bonsai b/docs/workflows/router-message-parsing.bonsai new file mode 100644 index 0000000..8ee8daf --- /dev/null +++ b/docs/workflows/router-message-parsing.bonsai @@ -0,0 +1,94 @@ + + + + + + + @tcp://localhost:5557 + + + + Request.First + + + Buffer + + + + 1 + + + + Request.Last + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/router-message-parsing.bonsai.layout b/docs/workflows/router-message-parsing.bonsai.layout new file mode 100644 index 0000000..eb127c3 --- /dev/null +++ b/docs/workflows/router-message-parsing.bonsai.layout @@ -0,0 +1,206 @@ + + + + false + + 144 + 156 + + + 334 + 320 + + Normal + + + true + + 120 + 130 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 458 + 308 + + + 413 + 278 + + Normal + Bonsai.Design.Visualizers.TimeSeriesVisualizer + + + 640 + 200 + 202.5 + true + + + + + true + + 457 + 130 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + true + + 472 + 189 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/router-message-parsing.svg b/docs/workflows/router-message-parsing.svg new file mode 100644 index 0000000..a64db56 --- /dev/null +++ b/docs/workflows/router-message-parsing.svg @@ -0,0 +1,3 @@ + +]>IndexConvertToStringDealerDealerDealerBufferRequest.LastStringStringStringRequest.FirstKeyDownKeyDownKeyDownRouter \ No newline at end of file From 9b742e078953c77e0697ca02b0d5f13a78369f1f Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Mar 2023 12:30:52 +0000 Subject: [PATCH 06/37] Client address tracking --- docs/articles/client-server.md | 23 +- docs/workflows/address-kvp.bonsai | 100 +++++++ docs/workflows/address-kvp.bonsai.layout | 222 ++++++++++++++++ docs/workflows/address-kvp.svg | 3 + docs/workflows/address-replay-subject.bonsai | 109 ++++++++ .../address-replay-subject.bonsai.layout | 250 ++++++++++++++++++ docs/workflows/address-replay-subject.svg | 3 + docs/workflows/unique-kvp.bonsai | 104 ++++++++ docs/workflows/unique-kvp.bonsai.layout | 238 +++++++++++++++++ docs/workflows/unique-kvp.svg | 3 + 10 files changed, 1054 insertions(+), 1 deletion(-) create mode 100644 docs/workflows/address-kvp.bonsai create mode 100644 docs/workflows/address-kvp.bonsai.layout create mode 100644 docs/workflows/address-kvp.svg create mode 100644 docs/workflows/address-replay-subject.bonsai create mode 100644 docs/workflows/address-replay-subject.bonsai.layout create mode 100644 docs/workflows/address-replay-subject.svg create mode 100644 docs/workflows/unique-kvp.bonsai create mode 100644 docs/workflows/unique-kvp.bonsai.layout create mode 100644 docs/workflows/unique-kvp.svg diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 415d4eb..62a2dbd 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -64,4 +64,25 @@ As with the **`Dealer`** node, a **`Router`** node without any input will simply ![Router message parsing](~/workflows/router-message-parsing.bonsai) ::: -Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the **`Index`** node output and a corresponding `string` in the **`ConvertToString`** node output. \ No newline at end of file +Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the **`Index`** node output and a corresponding `string` in the **`ConvertToString`** node output. + +## Client address tracking +So far our network is rather one-sided. We can send client messages to the server which can in turn receive and parse them, but currently nothing is relayed back to the clients. The first goal for server feedbasck is that any time a client message is received, the server sends this message back to all connected clients. To do this, we first need a way of keeping track of all active clients. Add a [**`Zip`**](xref:Bonsai.Reactive.Zip) node to the **`Index`** node and connect the `byte[]` **`Buffer`** as the second input. + +:::workflow +![Address key-value pair](~/workflows/address-kvp.bonsai) +::: + +Every time the **`Router`** receives a message, the **`Zip`** will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. Next, add a [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) node after the **`Zip`** and set the `KeySelector` property to the `byte` value (`Item1`). + +:::workflow +![Unique key-value pair](~/workflows/unique-kvp.bonsai) +::: + +The **`DistinctBy`** node filters the output of **`Zip`** according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by **`Zip`**. The output of **`DistinctBy`** will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. Add a [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) node after **`DistinctBy`** and name it ‘ClientAddresses’. + +:::workflow +![Address ReplaySubject](~/workflows/address-replay-subject.bonsai) +::: + +A **`ReplaySubject`** has the useful feature that it stores its input sequence and replays those values to any current or future subscribers. The effect in this case is that anything that subscribes to **`ClientAddresses`** will receive all the unique client addresses encountered by the server so far. \ No newline at end of file diff --git a/docs/workflows/address-kvp.bonsai b/docs/workflows/address-kvp.bonsai new file mode 100644 index 0000000..5b2322d --- /dev/null +++ b/docs/workflows/address-kvp.bonsai @@ -0,0 +1,100 @@ + + + + + + + @tcp://localhost:5557 + + + + Request.First + + + Buffer + + + + 1 + + + + + + + Request.Last + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/address-kvp.bonsai.layout b/docs/workflows/address-kvp.bonsai.layout new file mode 100644 index 0000000..9dda498 --- /dev/null +++ b/docs/workflows/address-kvp.bonsai.layout @@ -0,0 +1,222 @@ + + + + false + + 144 + 156 + + + 334 + 320 + + Normal + + + true + + 120 + 130 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 458 + 308 + + + 413 + 278 + + Normal + Bonsai.Design.Visualizers.TimeSeriesVisualizer + + + 640 + 181 + 183.5 + true + + + + + true + + 149 + 468 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + true + + 457 + 130 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + true + + 472 + 189 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/address-kvp.svg b/docs/workflows/address-kvp.svg new file mode 100644 index 0000000..b4439a1 --- /dev/null +++ b/docs/workflows/address-kvp.svg @@ -0,0 +1,3 @@ + +]>ZipConvertToStringDealerDealerDealerIndexRequest.LastStringStringStringBufferKeyDownKeyDownKeyDownRequest.FirstRouter \ No newline at end of file diff --git a/docs/workflows/address-replay-subject.bonsai b/docs/workflows/address-replay-subject.bonsai new file mode 100644 index 0000000..b7b67c1 --- /dev/null +++ b/docs/workflows/address-replay-subject.bonsai @@ -0,0 +1,109 @@ + + + + + + + @tcp://localhost:5557 + + + + Request.First + + + Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + Request.Last + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/address-replay-subject.bonsai.layout b/docs/workflows/address-replay-subject.bonsai.layout new file mode 100644 index 0000000..f49d2ac --- /dev/null +++ b/docs/workflows/address-replay-subject.bonsai.layout @@ -0,0 +1,250 @@ + + + + false + + 144 + 156 + + + 334 + 320 + + Normal + + + false + + 120 + 130 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 458 + 308 + + + 413 + 278 + + Normal + Bonsai.Design.Visualizers.TimeSeriesVisualizer + + + 640 + 56 + 58.5 + true + + + + + true + + 149 + 468 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 24 + 26 + + + 334 + 500 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 457 + 130 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 472 + 189 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/address-replay-subject.svg b/docs/workflows/address-replay-subject.svg new file mode 100644 index 0000000..fe1a5d0 --- /dev/null +++ b/docs/workflows/address-replay-subject.svg @@ -0,0 +1,3 @@ + +]>ClientAddressesConvertToStringDealerDealerDealerDistinctByRequest.LastStringStringStringZipKeyDownKeyDownKeyDownIndexBufferRequest.FirstRouter \ No newline at end of file diff --git a/docs/workflows/unique-kvp.bonsai b/docs/workflows/unique-kvp.bonsai new file mode 100644 index 0000000..8dfc222 --- /dev/null +++ b/docs/workflows/unique-kvp.bonsai @@ -0,0 +1,104 @@ + + + + + + + @tcp://localhost:5557 + + + + Request.First + + + Buffer + + + + 1 + + + + + + + Item1 + + + Request.Last + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/unique-kvp.bonsai.layout b/docs/workflows/unique-kvp.bonsai.layout new file mode 100644 index 0000000..bc43f5c --- /dev/null +++ b/docs/workflows/unique-kvp.bonsai.layout @@ -0,0 +1,238 @@ + + + + false + + 144 + 156 + + + 334 + 320 + + Normal + + + false + + 120 + 130 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 458 + 308 + + + 413 + 278 + + Normal + Bonsai.Design.Visualizers.TimeSeriesVisualizer + + + 640 + 56 + 58.5 + true + + + + + true + + 149 + 468 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 24 + 26 + + + 334 + 500 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 457 + 130 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 472 + 189 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/unique-kvp.svg b/docs/workflows/unique-kvp.svg new file mode 100644 index 0000000..3fdb6d4 --- /dev/null +++ b/docs/workflows/unique-kvp.svg @@ -0,0 +1,3 @@ + +]>DistinctByConvertToStringDealerDealerDealerZipRequest.LastStringStringStringIndexKeyDownKeyDownKeyDownBufferRequest.FirstRouter \ No newline at end of file From d4882fdb36f87784936ba67aba63de202e7591a7 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Mar 2023 14:46:46 +0000 Subject: [PATCH 07/37] Server-->client and SelectMany detour --- docs/articles/client-server.md | 44 ++- docs/workflows/select-many-detour.bonsai | 190 +++++++++++++ .../select-many-detour.bonsai.layout | 249 +++++++++++++++++ docs/workflows/select-many-detour.svg | 3 + docs/workflows/server-basic-response.bonsai | 129 +++++++++ .../server-basic-response.bonsai.layout | 262 ++++++++++++++++++ docs/workflows/server-basic-response.svg | 3 + docs/workflows/server-message-format.bonsai | 129 +++++++++ .../server-message-format.bonsai.layout | 262 ++++++++++++++++++ .../workflows/server-message-multicast.bonsai | 157 +++++++++++ .../server-message-multicast.bonsai.layout | 233 ++++++++++++++++ docs/workflows/server-message-multicast.svg | 3 + 12 files changed, 1663 insertions(+), 1 deletion(-) create mode 100644 docs/workflows/select-many-detour.bonsai create mode 100644 docs/workflows/select-many-detour.bonsai.layout create mode 100644 docs/workflows/select-many-detour.svg create mode 100644 docs/workflows/server-basic-response.bonsai create mode 100644 docs/workflows/server-basic-response.bonsai.layout create mode 100644 docs/workflows/server-basic-response.svg create mode 100644 docs/workflows/server-message-format.bonsai create mode 100644 docs/workflows/server-message-format.bonsai.layout create mode 100644 docs/workflows/server-message-multicast.bonsai create mode 100644 docs/workflows/server-message-multicast.bonsai.layout create mode 100644 docs/workflows/server-message-multicast.svg diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 62a2dbd..9877865 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -85,4 +85,46 @@ The **`DistinctBy`** node filters the output of **`Zip`** according to the uniqu ![Address ReplaySubject](~/workflows/address-replay-subject.bonsai) ::: -A **`ReplaySubject`** has the useful feature that it stores its input sequence and replays those values to any current or future subscribers. The effect in this case is that anything that subscribes to **`ClientAddresses`** will receive all the unique client addresses encountered by the server so far. \ No newline at end of file +A **`ReplaySubject`** has the useful feature that it stores its input sequence and replays those values to any current or future subscribers. The effect in this case is that anything that subscribes to **`ClientAddresses`** will receive all the unique client addresses encountered by the server so far. + +## Server --> client communication +Eventually, we will use these unique client addresses to route server messages back to specific client. For now, we'll implement a more basic approach where the server just sends messages back to the client that originally sent them. The Bonsai ZeroMQ library provides a convenient node for this task in the form of [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse). Add this after the **`Router`** in a separate branch, and inside (double-click on **`SendResponse`**) add a **`String`** node with a generic response value like `ServerResponse` after the **`Source`** node. + +:::workflow +![Basic server response](~/workflows/server-basic-response.bonsai) +::: + +The **`SendResponse`** node has a couple of interesting properties which may not be immediately obvious from this simple example. First, this node always transmits its response back to the ZeroMQ socket that initiated the request (in this case one of our **`Dealer`** clients) and we therefore do not need to specify an address in its processing logic. Second, the internal flow of **`SendResponse`** logic is computed asynchronously. This is very useful for responses that require more intensive computation and allows a **`Router`** to deal with frequent incoming **`Dealer`** requests efficiently. + +> Imagine, for example, that our Dealer sockets were sending video snippets to a Router server that is tasked with doing some processing of the video and returning the results back to the Dealers. If the responses were not computed in an asynchronous manner we would start to incur a bottleneck on the router if there were many connected Dealers or frequent Dealer requests. + +Running this workflow, you should see a 'bounceback' where any **`Dealer`** client that sends a message receives a reply from the **`Router`** server. However, in order to address these messages to specific other clients we need to take a slightly different approach. Delete the **`SendResponse`** and **`ConvertToString`** branches and replace with a branch that generates a bounceback message without using the **`SendResponse`** node: + +:::workflow +![Server message multicast](~/workflows/server-message-multicast.bonsai) +::: + +We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the **`SendResponse`** node in this implementation we need to pass messages directly into the **`Router`**. To do this we generate a **`BehaviorSubject`** source with a `NetMQMessage` output type and connect it to the **`Router`** (can implement this by creating a **`ToMessage`** node, right-clicking it and creating a **`BehaviorSubject`** source). This will change the output type of the **`Router`** node from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. + + We want the **`Router`** to generate a reply message every time it receives a request from a **`Dealer`**. Since we are now building this message ourselves instead of using **`SendResponse`**, we branch off the **`Router`** with a **`SelectMany`** node. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using **`Index`** to grab the middle empty delimiter frame and creating a new **`String`** which we convert to a `NetMQFrame` for the message content. We **`Merge`** these component frames back together and use a **`Take`** node (with count = 3) followed by **`ToMessage`**. The **`Take`** node is particularly important here as 1) **`ToMessage`** will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the **`SelectMany`**. Finally, we use a **`MulticastSubject`** to send our completed message to the **`Router`**. + + If we run the workflow now, we should see the same behavior as before (server bounces message back to initiating client). + + ## SelectMany detour + Now our network has a complete loop of client --> server --> client communication, but only the client that sends a message receives anything back from the server. Instead we’d like all clients to know when any of the clients sends a message. We already have access to the connected clients from **`ClientAddresses`**, and we know how to package data and send it back to clients via the **`Router`**. In an imperative language we would do something like: + + ``` +foreach (var client in Clients) { + Router.SendMessage(client.Address, Message); +} +``` + +using a loop to send the message back to each client in turn. In a reactive / observable sequence based framework we have to think about this a bit differently. The solution is to use a **`SelectMany`** operator and it is worth taking a detour here to understand its use in some detail since we have already used it to generate our bounceback message and will apply it again for addressing multiple clients. + +The **`SelectMany`** operator can be a tricky one to understand. Lee Campbell’s excellent [Introduction to Rx](http://introtorx.com/Content/v1.0.10621.0/08_Transformation.html#SelectMany) book does a good job of summarising its utility, suggesting we think of it as “from one, select many” or “from one, select zero or more”. In our case, we can think of **`SelectMany`** as a way to repeat some processing logic several times and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. + +Create a **`KeyDown`** node followed by a **`SelectMany`**. Set the `Filter` for the **`KeyDown`** to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the **`SelectMany`** node add a **`SubscribeSubject`** and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a **`TakeUntil`** node after the **`SubscribeSubject`** and connect the output of **`TakeUntil`** to the **`WorkflowOutput`** (disconnecting the `Source` node). Finally, create a **`KeyUp`** node and connect it to **`TakeUntil`**. Set the key `Filter` for **`KeyUp`** to the same as previously created **`KeyDown`** node outside the **`SelectMany`**. + +:::workflow +![SelectMany detour](~/workflows/select-many-detour.bonsai) +::: \ No newline at end of file diff --git a/docs/workflows/select-many-detour.bonsai b/docs/workflows/select-many-detour.bonsai new file mode 100644 index 0000000..dc577a6 --- /dev/null +++ b/docs/workflows/select-many-detour.bonsai @@ -0,0 +1,190 @@ + + + + + + RouterMessages + + + + @tcp://localhost:5557 + + + + First.Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + BounceBack + + + + Source1 + + + First + + + + 1 + + + + + ServerResponse + + + + + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + A + false + + + + + + + Source1 + + + ClientAddresses + + + + A + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/select-many-detour.bonsai.layout b/docs/workflows/select-many-detour.bonsai.layout new file mode 100644 index 0000000..6d742ea --- /dev/null +++ b/docs/workflows/select-many-detour.bonsai.layout @@ -0,0 +1,249 @@ + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 703 + 201 + + + 334 + 320 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 431 + 208 + + + 646 + 486 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 760 + 256 + + + 334 + 195 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + \ No newline at end of file diff --git a/docs/workflows/select-many-detour.svg b/docs/workflows/select-many-detour.svg new file mode 100644 index 0000000..932a5c7 --- /dev/null +++ b/docs/workflows/select-many-detour.svg @@ -0,0 +1,3 @@ + +]>ClientAddressesBounceBackDealerDealerDealerSelectManyDistinctByStringStringStringKeyDownZipKeyDownKeyDownKeyDownIndexFirst.BufferRouterRouterMessages \ No newline at end of file diff --git a/docs/workflows/server-basic-response.bonsai b/docs/workflows/server-basic-response.bonsai new file mode 100644 index 0000000..68d7dfc --- /dev/null +++ b/docs/workflows/server-basic-response.bonsai @@ -0,0 +1,129 @@ + + + + + + + @tcp://localhost:5557 + + + + Request.First + + + Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + Request.Last + + + + + + + + + Source1 + + + + ServerResponse + + + + + + + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/server-basic-response.bonsai.layout b/docs/workflows/server-basic-response.bonsai.layout new file mode 100644 index 0000000..d45fa24 --- /dev/null +++ b/docs/workflows/server-basic-response.bonsai.layout @@ -0,0 +1,262 @@ + + + + false + + 144 + 156 + + + 334 + 320 + + Normal + + + false + + 120 + 130 + + + 334 + 64 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 753 + 366 + + + 413 + 278 + + Normal + Bonsai.Design.Visualizers.TimeSeriesVisualizer + + + 640 + 38 + 40.5 + true + + + + + true + + 149 + 468 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 24 + 26 + + + 334 + 500 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 457 + 130 + + + 334 + 64 + + Normal + + + false + + 472 + 189 + + + 334 + 64 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 808 + 210 + + + 314 + 238 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 96 + 104 + + + 334 + 184 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/server-basic-response.svg b/docs/workflows/server-basic-response.svg new file mode 100644 index 0000000..aa56963 --- /dev/null +++ b/docs/workflows/server-basic-response.svg @@ -0,0 +1,3 @@ + +]>ClientAddressesConvertToStringSendResponseDealerDealerDealerDistinctByRequest.LastStringStringStringZipKeyDownKeyDownKeyDownIndexBufferRequest.FirstRouter \ No newline at end of file diff --git a/docs/workflows/server-message-format.bonsai b/docs/workflows/server-message-format.bonsai new file mode 100644 index 0000000..68d7dfc --- /dev/null +++ b/docs/workflows/server-message-format.bonsai @@ -0,0 +1,129 @@ + + + + + + + @tcp://localhost:5557 + + + + Request.First + + + Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + Request.Last + + + + + + + + + Source1 + + + + ServerResponse + + + + + + + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/server-message-format.bonsai.layout b/docs/workflows/server-message-format.bonsai.layout new file mode 100644 index 0000000..d45fa24 --- /dev/null +++ b/docs/workflows/server-message-format.bonsai.layout @@ -0,0 +1,262 @@ + + + + false + + 144 + 156 + + + 334 + 320 + + Normal + + + false + + 120 + 130 + + + 334 + 64 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 753 + 366 + + + 413 + 278 + + Normal + Bonsai.Design.Visualizers.TimeSeriesVisualizer + + + 640 + 38 + 40.5 + true + + + + + true + + 149 + 468 + + + 334 + 64 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 24 + 26 + + + 334 + 500 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 457 + 130 + + + 334 + 64 + + Normal + + + false + + 472 + 189 + + + 334 + 64 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 808 + 210 + + + 314 + 238 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 96 + 104 + + + 334 + 184 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/server-message-multicast.bonsai b/docs/workflows/server-message-multicast.bonsai new file mode 100644 index 0000000..80739bb --- /dev/null +++ b/docs/workflows/server-message-multicast.bonsai @@ -0,0 +1,157 @@ + + + + + + RouterMessages + + + + @tcp://localhost:5557 + + + + First.Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + BounceBack + + + + Source1 + + + First + + + + 1 + + + + + ServerResponse + + + + + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/server-message-multicast.bonsai.layout b/docs/workflows/server-message-multicast.bonsai.layout new file mode 100644 index 0000000..5092152 --- /dev/null +++ b/docs/workflows/server-message-multicast.bonsai.layout @@ -0,0 +1,233 @@ + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 703 + 201 + + + 334 + 320 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 431 + 208 + + + 646 + 486 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 164 + 408 + + + 334 + 184 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 96 + 104 + + + 334 + 182 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 753 + 223 + + + 334 + 211 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + \ No newline at end of file diff --git a/docs/workflows/server-message-multicast.svg b/docs/workflows/server-message-multicast.svg new file mode 100644 index 0000000..8ef96ba --- /dev/null +++ b/docs/workflows/server-message-multicast.svg @@ -0,0 +1,3 @@ + +]>ClientAddressesBounceBackDealerDealerDealerDistinctByStringStringStringZipKeyDownKeyDownKeyDownIndexFirst.BufferRouterRouterMessages \ No newline at end of file From be73aeed5578305d239cd9966297e882ec3d2f91 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 23 Mar 2023 16:07:11 +0000 Subject: [PATCH 08/37] docs up to Dealer bug --- docs/articles/client-server.md | 23 +- docs/workflows/broadcast-all.bonsai | 300 ++++++++++++++ docs/workflows/broadcast-all.bonsai.layout | 385 ++++++++++++++++++ .../format-select-all-clients.bonsai | 200 +++++++++ .../format-select-all-clients.bonsai.layout | 281 +++++++++++++ docs/workflows/format-select-all-clients.svg | 3 + docs/workflows/select-all-clients.bonsai | 195 +++++++++ .../select-all-clients.bonsai.layout | 277 +++++++++++++ docs/workflows/server-next-message.bonsai | 200 +++++++++ .../server-next-message.bonsai.layout | 273 +++++++++++++ docs/workflows/server-next-message.svg | 3 + 11 files changed, 2139 insertions(+), 1 deletion(-) create mode 100644 docs/workflows/broadcast-all.bonsai create mode 100644 docs/workflows/broadcast-all.bonsai.layout create mode 100644 docs/workflows/format-select-all-clients.bonsai create mode 100644 docs/workflows/format-select-all-clients.bonsai.layout create mode 100644 docs/workflows/format-select-all-clients.svg create mode 100644 docs/workflows/select-all-clients.bonsai create mode 100644 docs/workflows/select-all-clients.bonsai.layout create mode 100644 docs/workflows/server-next-message.bonsai create mode 100644 docs/workflows/server-next-message.bonsai.layout create mode 100644 docs/workflows/server-next-message.svg diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 9877865..097eb6a 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -127,4 +127,25 @@ Create a **`KeyDown`** node followed by a **`SelectMany`**. Set the `Filter` for :::workflow ![SelectMany detour](~/workflows/select-many-detour.bonsai) -::: \ No newline at end of file +::: + +Run the project and inspect the output of the **`SelectMany`** node. If no client messages are triggered and we press ‘A’ to trigger the **`SelectMany`** nothing will be returned. If we trigger a single client and press ‘A’ again **`SelectMany`** gives us the address of that client. If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger **`SelectMany`** with a **`KeyDown`** we generate a new sequence that immediately subscribes to **`ClientAddresses`**, a **`ReplaySubject`** which replays all our unique client addresses into the sequence. We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the **`SusbcribeSubject`** directly to the workflow output and deleting **`KeyUp`** and **`TakeUntil`**). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger **`TakeUntil`** and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. + +## All client broadcast +To apply the logic of the **`SelectMany`** example to server broadcast, we need something to trigger the **`SelectMany`** sequence creation, and something to trigger termination. We already have a trigger for sequence creation in the output of the **`Router`** since we want to run our **`SelectMany`** sequence every time a client message is received. For our sequence temination trigger, we want something that is guaranteed to fire after the server receives a client message, but before the next message is received so that our **`SelectMany`** sequence for each message responds only to that particular message. A simple solution is therefore to use the arrival of the next message as our sequence termination trigger. + +To implement this, add a **`Skip`** node after the **`Router`** in a separate branch and connect this to a **`PublishSubject`**. Ensure that the **`Skip`** node's `Count` property is set to 1, and name the **`PublishSubject`** 'NextMessage'. + +:::workflow +![Server next message](~/workflows/server-next-message.bonsai) +::: + +The logic here is that we use **`Skip`** to create a sequence that lags exactly 1 message behind the **`Router`** sequence of received messages, i.e. when the first message is received, **`NextMessage`** will not produce a result until the second message is received. We can then use this inside our **`SelectMany`** logic for generating server messages. Add a **`SelectMany`** node after the **`Router`** in a separate branch and name it ‘SelectAllClients’. + +Inside the **`SelectMany`** node, create 2 **`SubscribeSubject`** nodes and link them to the **`ClientAddresses`** and **`NextMessage`** subjects. Connect the **`ClientAddresses`** subscription to the workflow output via a **`TakeUntil`** node and use **`NextMessage`** as the second input. Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a **`WithLatestFrom`** with the **`Router`** as its second input. In this context **`WithLatestFrom`** combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. + +:::workflow +![Select all clients and package message](~/workflows/select-all-clients-format.bonsai) +::: + +To send these messages back to our clients, we will modify the logic in our previous **`BounceBack`** node. This time, we'll create a **`SelectMany`** called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. This is multicast back into the router to send the original address back to all clients. diff --git a/docs/workflows/broadcast-all.bonsai b/docs/workflows/broadcast-all.bonsai new file mode 100644 index 0000000..3ca77bd --- /dev/null +++ b/docs/workflows/broadcast-all.bonsai @@ -0,0 +1,300 @@ + + + + + + RouterMessages + + + + @tcp://localhost:5557 + + + + First.Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + + 1 + + + + NextMessage + + + + BounceBack + + + + Source1 + + + First + + + + 1 + + + + + ServerResponse + + + + + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + SelectAllClients + + + + Source1 + + + ClientAddresses + + + + NextMessage + + + + + PT0.1S + PT0S + + + + + + + + + + + + + + + + + + + + BroadcastToAll + + + + Source1 + + + Item1.Item2 + + + + + + Item2 + + + + 1 + + + + + ServerResponse + + + + + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + + + Item2 + + + + + + PT0.1S + + + + + + RouterMessages + + + + Last + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/broadcast-all.bonsai.layout b/docs/workflows/broadcast-all.bonsai.layout new file mode 100644 index 0000000..8df6bb7 --- /dev/null +++ b/docs/workflows/broadcast-all.bonsai.layout @@ -0,0 +1,385 @@ + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 144 + 156 + + + 334 + 296 + + Normal + + + false + + 159 + 427 + + + 334 + 375 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 145 + 41 + + + 611 + 439 + + Normal + + + + true + + 120 + 130 + + + 334 + 396 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + false + + 338 + 241 + + + 399 + 327 + + Normal + + + + false + + 48 + 52 + + + 334 + 305 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 144 + 156 + + + 334 + 64 + + Normal + + false + + 229 + 154 + + + 738 + 436 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 24 + 26 + + + 334 + 290 + + Normal + + + false + + 859 + 203 + + + 334 + 254 + + Normal + + + false + + 96 + 104 + + + 334 + 256 + + Normal + + + false + + 169 + 186 + + + 334 + 308 + + Normal + + + true + + 173 + 409 + + + 334 + 327 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 168 + 182 + + + 334 + 64 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 528 + 309 + + + 334 + 301 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 144 + 156 + + + 334 + 200 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 856 + 293 + + + 334 + 319 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + \ No newline at end of file diff --git a/docs/workflows/format-select-all-clients.bonsai b/docs/workflows/format-select-all-clients.bonsai new file mode 100644 index 0000000..9417bdf --- /dev/null +++ b/docs/workflows/format-select-all-clients.bonsai @@ -0,0 +1,200 @@ + + + + + + RouterMessages + + + + @tcp://localhost:5557 + + + + First.Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + + BounceBack + + + + Source1 + + + First + + + + 1 + + + + + ServerResponse + + + + + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + + 1 + + + + NextMessage + + + SelectAllClients + + + + Source1 + + + ClientAddresses + + + NextMessage + + + + + + + + + + + + + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/format-select-all-clients.bonsai.layout b/docs/workflows/format-select-all-clients.bonsai.layout new file mode 100644 index 0000000..a88bcda --- /dev/null +++ b/docs/workflows/format-select-all-clients.bonsai.layout @@ -0,0 +1,281 @@ + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 703 + 201 + + + 334 + 320 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 38 + 266 + + + 458 + 375 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 890 + 207 + + + 334 + 317 + + Normal + + + false + + 722 + 280 + + + 334 + 434 + + Normal + + false + + 706 + 226 + + + 651 + 435 + + Normal + + + + false + + 24 + 26 + + + 334 + 369 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 96 + 104 + + + 334 + 141 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/format-select-all-clients.svg b/docs/workflows/format-select-all-clients.svg new file mode 100644 index 0000000..0640ef8 --- /dev/null +++ b/docs/workflows/format-select-all-clients.svg @@ -0,0 +1,3 @@ + +]>ClientAddressesBounceBackNextMessageWithLatestFromDealerDealerDealerDistinctBySkipSelectAllClientsStringStringStringZipKeyDownKeyDownKeyDownIndexFirst.BufferRouterRouterMessages \ No newline at end of file diff --git a/docs/workflows/select-all-clients.bonsai b/docs/workflows/select-all-clients.bonsai new file mode 100644 index 0000000..2fab811 --- /dev/null +++ b/docs/workflows/select-all-clients.bonsai @@ -0,0 +1,195 @@ + + + + + + RouterMessages + + + + @tcp://localhost:5557 + + + + First.Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + + BounceBack + + + + Source1 + + + First + + + + 1 + + + + + ServerResponse + + + + + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + + 1 + + + + NextMessage + + + SelectAllClients + + + + Source1 + + + ClientAddresses + + + NextMessage + + + + + + + + + + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/select-all-clients.bonsai.layout b/docs/workflows/select-all-clients.bonsai.layout new file mode 100644 index 0000000..f028879 --- /dev/null +++ b/docs/workflows/select-all-clients.bonsai.layout @@ -0,0 +1,277 @@ + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 703 + 201 + + + 334 + 320 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 168 + 197 + + + 458 + 375 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 890 + 207 + + + 334 + 317 + + Normal + + + false + + 722 + 280 + + + 334 + 434 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + false + + 706 + 226 + + + 651 + 435 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 96 + 104 + + + 334 + 141 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + \ No newline at end of file diff --git a/docs/workflows/server-next-message.bonsai b/docs/workflows/server-next-message.bonsai new file mode 100644 index 0000000..c9a0eec --- /dev/null +++ b/docs/workflows/server-next-message.bonsai @@ -0,0 +1,200 @@ + + + + + + RouterMessages + + + + @tcp://localhost:5557 + + + + First.Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + BounceBack + + + + Source1 + + + First + + + + 1 + + + + + ServerResponse + + + + + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + 1 + + + + NextMessage + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + A + false + + + + + + + Source1 + + + ClientAddresses + + + + A + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/server-next-message.bonsai.layout b/docs/workflows/server-next-message.bonsai.layout new file mode 100644 index 0000000..97b791e --- /dev/null +++ b/docs/workflows/server-next-message.bonsai.layout @@ -0,0 +1,273 @@ + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 703 + 201 + + + 334 + 320 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 431 + 208 + + + 646 + 486 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 760 + 256 + + + 334 + 195 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + \ No newline at end of file diff --git a/docs/workflows/server-next-message.svg b/docs/workflows/server-next-message.svg new file mode 100644 index 0000000..9be5129 --- /dev/null +++ b/docs/workflows/server-next-message.svg @@ -0,0 +1,3 @@ + +]>ClientAddressesBounceBackNextMessageDealerDealerDealerSelectManyDistinctBySkipStringStringStringKeyDownZipKeyDownKeyDownKeyDownIndexFirst.BufferRouterRouterMessages \ No newline at end of file From e3afe601f38e77b8a771e35ef209c304fa68caaf Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 23 Mar 2023 16:24:52 +0000 Subject: [PATCH 09/37] Change version to 0.1.1 --- src/Bonsai.ZeroMQ/Bonsai.ZeroMQ.csproj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/Bonsai.ZeroMQ/Bonsai.ZeroMQ.csproj b/src/Bonsai.ZeroMQ/Bonsai.ZeroMQ.csproj index 2a9d4bf..5f02841 100644 --- a/src/Bonsai.ZeroMQ/Bonsai.ZeroMQ.csproj +++ b/src/Bonsai.ZeroMQ/Bonsai.ZeroMQ.csproj @@ -1,4 +1,4 @@ - + Bonsai - ZeroMQ Library From e911ea3ffebe1c0d9b1b997ed267a60921c869b3 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 18 May 2023 14:09:49 -0400 Subject: [PATCH 10/37] Leave one out broadcast --- docs/articles/client-server.md | 19 +- docs/workflows/broadcast-all-clients.bonsai | 200 +++++++++++ .../broadcast-all-clients.bonsai.layout | 281 ++++++++++++++++ .../broadcast-non-sender-clients.bonsai | 272 +++++++++++++++ ...broadcast-non-sender-clients.bonsai.layout | 317 ++++++++++++++++++ .../format-select-all-clients.bonsai | 4 +- .../format-select-all-clients.bonsai.layout | 34 +- 7 files changed, 1109 insertions(+), 18 deletions(-) create mode 100644 docs/workflows/broadcast-all-clients.bonsai create mode 100644 docs/workflows/broadcast-all-clients.bonsai.layout create mode 100644 docs/workflows/broadcast-non-sender-clients.bonsai create mode 100644 docs/workflows/broadcast-non-sender-clients.bonsai.layout diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 097eb6a..7b93968 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -145,7 +145,24 @@ The logic here is that we use **`Skip`** to create a sequence that lags exactly Inside the **`SelectMany`** node, create 2 **`SubscribeSubject`** nodes and link them to the **`ClientAddresses`** and **`NextMessage`** subjects. Connect the **`ClientAddresses`** subscription to the workflow output via a **`TakeUntil`** node and use **`NextMessage`** as the second input. Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a **`WithLatestFrom`** with the **`Router`** as its second input. In this context **`WithLatestFrom`** combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. :::workflow -![Select all clients and package message](~/workflows/select-all-clients-format.bonsai) +![Select all clients and package message](~/workflows/format-select-all-clients.bonsai) ::: To send these messages back to our clients, we will modify the logic in our previous **`BounceBack`** node. This time, we'll create a **`SelectMany`** called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. This is multicast back into the router to send the original address back to all clients. + +Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of client ID / address and the received message. We take the `byte[]` address and use **`ConvertToFrame`** to convert it to a `NetMQFrame` and then merge it with the empty delimiter and the message payload. As before, we **`Take`** 3 elements to close the message construction stream, convert to a `NetMQMessage` with **`ToMessage`** and then **`Multicast`** into `RouterMessages`. If you run the workflow now you should see that each time a **`Dealer`** produces a message, all clients receive a copy of that message. + +:::workflow +![Broadcast to all clients](~/workflows/broadcast-all-clients.bonsai) +::: + +## Leave-one-out broadcast +This is getting pretty close to our original network architecture goal but there is still some redundancy present. When client 1 sends a message to the server, clients 1, 2 and 3 all receive a copy of that message back from the server. this is fine for clients 2 and 3 as they are not aware of client 1's messages without server communication; but client 1 does not need this message copy since it already originated the message. Our goal then is that the server should send message copies back to all clients except the client that originated message. + +To do this, we'll create a **`Condition`** before `BroadcastAll` to filter only non sender clients called `NonSenderClients`. Inside `NonSenderClients` expose the `byte` corresponding to the client ID and index 1 of the first frame of the `NetMQMessage` from `Source1`. **`Zip`** these together and use **`NotEqual`** to compare the client ID of the message with the existing clients and discard where the IDs are the same. + +:::workflow +![Broadcast to non-senders](~/workflows/broadcast-non-sender-clients.bonsai) +::: + +Running the workflow you should see that we have now achieved the desired architecture. When a client **`Dealer`** sends a message, it is broadcast to all other joined clients except for itself. diff --git a/docs/workflows/broadcast-all-clients.bonsai b/docs/workflows/broadcast-all-clients.bonsai new file mode 100644 index 0000000..c9fb12d --- /dev/null +++ b/docs/workflows/broadcast-all-clients.bonsai @@ -0,0 +1,200 @@ + + + + + + RouterMessages + + + + @tcp://localhost:5557 + + + + First.Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + + 1 + + + + NextMessage + + + SelectAllClients + + + + Source1 + + + ClientAddresses + + + NextMessage + + + + + + + + + + + + + + + + + + BroadcastAll + + + + Source1 + + + Item1.Item2 + + + + + + Item2 + + + + 1 + + + + Item2.Last + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/broadcast-all-clients.bonsai.layout b/docs/workflows/broadcast-all-clients.bonsai.layout new file mode 100644 index 0000000..e877db5 --- /dev/null +++ b/docs/workflows/broadcast-all-clients.bonsai.layout @@ -0,0 +1,281 @@ + + + + false + + 72 + 78 + + + 334 + 214 + + Normal + + + false + + 703 + 201 + + + 334 + 320 + + Normal + + + false + + 48 + 52 + + + 413 + 278 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 890 + 207 + + + 334 + 317 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 700 + 168 + + + 674 + 349 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 188 + 461 + + + 334 + 141 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 513 + 457 + + + 334 + 148 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 836 + 461 + + + 334 + 144 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + \ No newline at end of file diff --git a/docs/workflows/broadcast-non-sender-clients.bonsai b/docs/workflows/broadcast-non-sender-clients.bonsai new file mode 100644 index 0000000..356539a --- /dev/null +++ b/docs/workflows/broadcast-non-sender-clients.bonsai @@ -0,0 +1,272 @@ + + + + + + RouterMessages + + + + @tcp://localhost:5557 + + + + First.Buffer + + + + 1 + + + + + + + Item1 + + + ClientAddresses + + + + + 1 + + + + NextMessage + + + SelectAllClients + + + + Source1 + + + ClientAddresses + + + NextMessage + + + + + + + + + + + + + + + + + + NonSenderClients + + + + Source1 + + + Item1.Item1 + + + Item2.First.Buffer + + + + 1 + + + + + + + + + + + + + + + + + + + + + BroadcastAll + + + + Source1 + + + FilterNonSenders + + + + Source1 + + + Item1.Item1 + + + Item2.First.Buffer + + + + 1 + + + + + + + + + + + + + + + + + + + + + Item1.Item2 + + + + + + Item2 + + + + 1 + + + + Item2.Last + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + + + + + + D1 + false + + + + + Client1 + + + + + tcp://localhost:5557 + + + + + D2 + false + + + + + Client2 + + + + + tcp://localhost:5557 + + + + + D3 + false + + + + + Client3 + + + + + tcp://localhost:5557 + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/workflows/broadcast-non-sender-clients.bonsai.layout b/docs/workflows/broadcast-non-sender-clients.bonsai.layout new file mode 100644 index 0000000..3cf2ff7 --- /dev/null +++ b/docs/workflows/broadcast-non-sender-clients.bonsai.layout @@ -0,0 +1,317 @@ + + + + false + + 72 + 78 + + + 334 + 214 + + Normal + + + false + + 703 + 201 + + + 334 + 320 + + Normal + + + false + + 48 + 52 + + + 413 + 278 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 890 + 207 + + + 334 + 317 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 714 + 201 + + + 415 + 342 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 758 + 201 + + + 530 + 238 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + false + + 552 + 259 + + + 674 + 349 + + Normal + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 188 + 461 + + + 334 + 141 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 513 + 457 + + + 334 + 148 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + false + + 0 + 0 + + + 0 + 0 + + Normal + + + true + + 836 + 461 + + + 334 + 144 + + Normal + Bonsai.Design.ObjectTextVisualizer + + + + + \ No newline at end of file diff --git a/docs/workflows/format-select-all-clients.bonsai b/docs/workflows/format-select-all-clients.bonsai index 9417bdf..2f7c9ef 100644 --- a/docs/workflows/format-select-all-clients.bonsai +++ b/docs/workflows/format-select-all-clients.bonsai @@ -1,10 +1,10 @@  diff --git a/docs/workflows/format-select-all-clients.bonsai.layout b/docs/workflows/format-select-all-clients.bonsai.layout index a88bcda..f4ed4b4 100644 --- a/docs/workflows/format-select-all-clients.bonsai.layout +++ b/docs/workflows/format-select-all-clients.bonsai.layout @@ -29,12 +29,12 @@ false - 0 - 0 + 48 + 52 - 0 - 0 + 413 + 278 Normal @@ -166,9 +166,13 @@ 334 - 369 + 326 Normal + Bonsai.Design.ObjectTextVisualizer + + + false @@ -197,8 +201,8 @@ false - 96 - 104 + 195 + 461 334 @@ -233,12 +237,12 @@ false - 0 - 0 + 724 + 450 - 0 - 0 + 334 + 148 Normal @@ -269,12 +273,12 @@ false - 0 - 0 + 741 + 97 - 0 - 0 + 334 + 144 Normal From 6f6e07b73b11fc664f4d90e11628e80a7b6f48a4 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 18 May 2023 16:44:39 -0400 Subject: [PATCH 11/37] Update xrefs for all referenced nodes --- docs/articles/client-server.md | 58 +++++++++++++++++----------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 7b93968..6c15a6b 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -37,17 +37,17 @@ ZeroMQ provides a number of socket types that could be used to achieve something ## Basic client To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project, add a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node. In the `ConnectionString` property, set `Address`: localhost:5557, `Action`: Connect, `Protocol`: TCP. -In Bonsai.ZeroMQ, the **`Dealer`** can have two functions based on its inputs. On its own, as above, the **`Dealer`** node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our **`Dealer`** to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. +In Bonsai.ZeroMQ, the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) can have two functions based on its inputs. On its own, as above, the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. -If we add inputs to the **`Dealer`**, it will act as both a sender and receiver of messages on the network. Before the **`Dealer`** node add a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) and [**`String`**](xref:Bonsai.Expressions.String) node in sequence as input to the **`Dealer`**. +If we add inputs to the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer), it will act as both a sender and receiver of messages on the network. Before the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node add a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) and [**`String`**](xref:Bonsai.Expressions.StringProperty) node in sequence as input to the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer). :::workflow ![Basic Dealer input](~/workflows/dealer-basic-input.bonsai) ::: -In the node properties, set the **`KeyDown`** `Filter` to the ‘1’ key and set the **`String`** `Value` to ‘Client1’. If we run the Bonsai project now, the **`Dealer`** will continue listening for incoming messages on the network, but every time the ‘1’ key is pressed a message containing the string ‘Client1’ will be sent from the socket. +In the node properties, set the [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) `Filter` to the ‘1’ key and set the [**`String`**](xref:Bonsai.Expressions.StringProperty) `Value` to ‘Client1’. If we run the Bonsai project now, the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) will continue listening for incoming messages on the network, but every time the ‘1’ key is pressed a message containing the string ‘Client1’ will be sent from the socket. -Copy and paste this client structure a couple of times and change the **`KeyDown`** and **`String`** properties accordingly on each (2, ‘Client2’; 3, ‘Client3’) so that we have 3 total clients that send messages according to different key presses. +Copy and paste this client structure a couple of times and change the [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) and [**`String`**](xref:Bonsai.Expressions.StringProperty) properties accordingly on each (2, ‘Client2’; 3, ‘Client3’) so that we have 3 total clients that send messages according to different key presses. :::workflow ![Multiple clients](~/workflows/multiple-clients.bonsai) @@ -56,62 +56,62 @@ Copy and paste this client structure a couple of times and change the **`KeyDown > For the purposes of this article we are creating all of our clients and our server on the same Bonsai project and same machine for ease of demonstration. In a working example, each client and server could be running in separate Bonsai instances on different machines on a network. In this case, localhost would be replaced with the server machine’s IP address. ## Basic server -Now that we have our client pool set up and sending messages, let’s implement a server to listen for those messages. Add a [**`Router`**](xref:Bonsai.ZeroMQ.Router) node to the project and set its properties to match the **`Dealer`** sockets we already added so that it is running on the same network. As the **`Router`** is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. +Now that we have our client pool set up and sending messages, let’s implement a server to listen for those messages. Add a [**`Router`**](xref:Bonsai.ZeroMQ.Router) node to the project and set its properties to match the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) sockets we already added so that it is running on the same network. As the [**`Router`**](xref:Bonsai.ZeroMQ.Router) is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. -As with the **`Dealer`** node, a **`Router`** node without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the **`Router`** node, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the **`Router`**. Expanding the output the the **`Router`**, we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. To make sense of the message, let's expose the `Buffer` `byte[]` of the `First` frame. Add an **`Index (Expressions)`** node the the first frame buffer and set its `Value` to 1 to access the unique address ID. Add a [**`ConvertToString`**](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. +As with the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node, a [**`Router`**](xref:Bonsai.ZeroMQ.Router) node without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [**`Router`**](xref:Bonsai.ZeroMQ.Router) node, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [**`Router`**](xref:Bonsai.ZeroMQ.Router). Expanding the output the the [**`Router`**](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. To make sense of the message, let's expose the `Buffer` `byte[]` of the `First` frame. Add an [**`Index`**](xref:Bonsai.Expressions.IndexBuilder) node the the first frame buffer and set its `Value` to 1 to access the unique address ID. Add a [**`ConvertToString`**](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. :::workflow ![Router message parsing](~/workflows/router-message-parsing.bonsai) ::: -Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the **`Index`** node output and a corresponding `string` in the **`ConvertToString`** node output. +Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the [**`Index`**](xref:Bonsai.Expressions.IndexBuilder) node output and a corresponding `string` in the [**`ConvertToString`**](xref:Bonsai.ZeroMQ.ConvertToString) node output. ## Client address tracking -So far our network is rather one-sided. We can send client messages to the server which can in turn receive and parse them, but currently nothing is relayed back to the clients. The first goal for server feedbasck is that any time a client message is received, the server sends this message back to all connected clients. To do this, we first need a way of keeping track of all active clients. Add a [**`Zip`**](xref:Bonsai.Reactive.Zip) node to the **`Index`** node and connect the `byte[]` **`Buffer`** as the second input. +So far our network is rather one-sided. We can send client messages to the server which can in turn receive and parse them, but currently nothing is relayed back to the clients. The first goal for server feedbasck is that any time a client message is received, the server sends this message back to all connected clients. To do this, we first need a way of keeping track of all active clients. Add a [**`Zip`**](xref:Bonsai.Reactive.Zip) node to the [**`Index`**](xref:Bonsai.Expressions.IndexBuilder) node and connect the `byte[]` `Buffer` as the second input. :::workflow ![Address key-value pair](~/workflows/address-kvp.bonsai) ::: -Every time the **`Router`** receives a message, the **`Zip`** will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. Next, add a [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) node after the **`Zip`** and set the `KeySelector` property to the `byte` value (`Item1`). +Every time the [**`Router`**](xref:Bonsai.ZeroMQ.Router) receives a message, the [**`Zip`**](xref:Bonsai.Reactive.Zip) will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. Next, add a [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) node after the [**`Zip`**](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). :::workflow ![Unique key-value pair](~/workflows/unique-kvp.bonsai) ::: -The **`DistinctBy`** node filters the output of **`Zip`** according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by **`Zip`**. The output of **`DistinctBy`** will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. Add a [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) node after **`DistinctBy`** and name it ‘ClientAddresses’. +The [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) node filters the output of [**`Zip`**](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [**`Zip`**](xref:Bonsai.Reactive.Zip). The output of [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. Add a [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) node after [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) and name it ‘ClientAddresses’. :::workflow ![Address ReplaySubject](~/workflows/address-replay-subject.bonsai) ::: -A **`ReplaySubject`** has the useful feature that it stores its input sequence and replays those values to any current or future subscribers. The effect in this case is that anything that subscribes to **`ClientAddresses`** will receive all the unique client addresses encountered by the server so far. +A [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) has the useful feature that it stores its input sequence and replays those values to any current or future subscribers. The effect in this case is that anything that subscribes to `ClientAddresses` will receive all the unique client addresses encountered by the server so far. ## Server --> client communication -Eventually, we will use these unique client addresses to route server messages back to specific client. For now, we'll implement a more basic approach where the server just sends messages back to the client that originally sent them. The Bonsai ZeroMQ library provides a convenient node for this task in the form of [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse). Add this after the **`Router`** in a separate branch, and inside (double-click on **`SendResponse`**) add a **`String`** node with a generic response value like `ServerResponse` after the **`Source`** node. +Eventually, we will use these unique client addresses to route server messages back to specific client. For now, we'll implement a more basic approach where the server just sends messages back to the client that originally sent them. The Bonsai ZeroMQ library provides a convenient node for this task in the form of [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse). Add this after the [**`Router`**](xref:Bonsai.ZeroMQ.Router) in a separate branch, and inside (double-click on [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse)) add a [**`String`**](xref:Bonsai.Expressions.StringProperty) node with a generic response value like `ServerResponse` after the **`Source`** node. :::workflow ![Basic server response](~/workflows/server-basic-response.bonsai) ::: -The **`SendResponse`** node has a couple of interesting properties which may not be immediately obvious from this simple example. First, this node always transmits its response back to the ZeroMQ socket that initiated the request (in this case one of our **`Dealer`** clients) and we therefore do not need to specify an address in its processing logic. Second, the internal flow of **`SendResponse`** logic is computed asynchronously. This is very useful for responses that require more intensive computation and allows a **`Router`** to deal with frequent incoming **`Dealer`** requests efficiently. +The [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) node has a couple of interesting properties which may not be immediately obvious from this simple example. First, this node always transmits its response back to the ZeroMQ socket that initiated the request (in this case one of our [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) clients) and we therefore do not need to specify an address in its processing logic. Second, the internal flow of [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) logic is computed asynchronously. This is very useful for responses that require more intensive computation and allows a [**`Router`**](xref:Bonsai.ZeroMQ.Router) to deal with frequent incoming [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) requests efficiently. > Imagine, for example, that our Dealer sockets were sending video snippets to a Router server that is tasked with doing some processing of the video and returning the results back to the Dealers. If the responses were not computed in an asynchronous manner we would start to incur a bottleneck on the router if there were many connected Dealers or frequent Dealer requests. -Running this workflow, you should see a 'bounceback' where any **`Dealer`** client that sends a message receives a reply from the **`Router`** server. However, in order to address these messages to specific other clients we need to take a slightly different approach. Delete the **`SendResponse`** and **`ConvertToString`** branches and replace with a branch that generates a bounceback message without using the **`SendResponse`** node: +Running this workflow, you should see a 'bounceback' where any [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [**`Router`**](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. Delete the [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) and [**`ConvertToString`**](xref:Bonsai.ZeroMQ.ConvertToString) branches and replace with a branch that generates a bounceback message without using the [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) node: :::workflow ![Server message multicast](~/workflows/server-message-multicast.bonsai) ::: -We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the **`SendResponse`** node in this implementation we need to pass messages directly into the **`Router`**. To do this we generate a **`BehaviorSubject`** source with a `NetMQMessage` output type and connect it to the **`Router`** (can implement this by creating a **`ToMessage`** node, right-clicking it and creating a **`BehaviorSubject`** source). This will change the output type of the **`Router`** node from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. +We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) node in this implementation we need to pass messages directly into the [**`Router`**](xref:Bonsai.ZeroMQ.Router). To do this we generate a [**`BehaviorSubject`**](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [**`Router`**](xref:Bonsai.ZeroMQ.Response) (can implement this by creating a [**`ToMessage`**](xref:Bonsai.ZeroMQ.ToMessage) node, right-clicking it and creating a [**`BehaviorSubject`**](xref:Bonsai.Reactive.BehaviorSubject) source). This will change the output type of the [**`Router`**](xref:Bonsai.ZeroMQ.Router) node from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. - We want the **`Router`** to generate a reply message every time it receives a request from a **`Dealer`**. Since we are now building this message ourselves instead of using **`SendResponse`**, we branch off the **`Router`** with a **`SelectMany`** node. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using **`Index`** to grab the middle empty delimiter frame and creating a new **`String`** which we convert to a `NetMQFrame` for the message content. We **`Merge`** these component frames back together and use a **`Take`** node (with count = 3) followed by **`ToMessage`**. The **`Take`** node is particularly important here as 1) **`ToMessage`** will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the **`SelectMany`**. Finally, we use a **`MulticastSubject`** to send our completed message to the **`Router`**. + We want the [**`Router`**](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [**`Router`**](xref:Bonsai.ZeroMQ.Router) with a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [**`Index`**](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new **`String`** which we convert to a `NetMQFrame` for the message content. We [**`Merge`**](xref:Bonsai.Reactive.Merge) these component frames back together and use a [**`Take`**](xref:Bonsai.Reactive.Take) node (with count = 3) followed by [**`ToMessage`**](xref:Bonsai.ZeroMQ.ToMessage). The [**`Take`**](xref:Bonsai.Reactive.Take) node is particularly important here as 1) [**`ToMessage`**](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany). Finally, we use a [**`MulticastSubject`**](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [**`Router`**](xref:Bonsai.ZeroMQ.Router). If we run the workflow now, we should see the same behavior as before (server bounces message back to initiating client). ## SelectMany detour - Now our network has a complete loop of client --> server --> client communication, but only the client that sends a message receives anything back from the server. Instead we’d like all clients to know when any of the clients sends a message. We already have access to the connected clients from **`ClientAddresses`**, and we know how to package data and send it back to clients via the **`Router`**. In an imperative language we would do something like: + Now our network has a complete loop of client --> server --> client communication, but only the client that sends a message receives anything back from the server. Instead we’d like all clients to know when any of the clients sends a message. We already have access to the connected clients from `ClientAddresses`, and we know how to package data and send it back to clients via the [**`Router`**](xref:Bonsai.ZeroMQ.Router). In an imperative language we would do something like: ``` foreach (var client in Clients) { @@ -119,38 +119,38 @@ foreach (var client in Clients) { } ``` -using a loop to send the message back to each client in turn. In a reactive / observable sequence based framework we have to think about this a bit differently. The solution is to use a **`SelectMany`** operator and it is worth taking a detour here to understand its use in some detail since we have already used it to generate our bounceback message and will apply it again for addressing multiple clients. +using a loop to send the message back to each client in turn. In a reactive / observable sequence based framework we have to think about this a bit differently. The solution is to use a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) operator and it is worth taking a detour here to understand its use in some detail since we have already used it to generate our bounceback message and will apply it again for addressing multiple clients. -The **`SelectMany`** operator can be a tricky one to understand. Lee Campbell’s excellent [Introduction to Rx](http://introtorx.com/Content/v1.0.10621.0/08_Transformation.html#SelectMany) book does a good job of summarising its utility, suggesting we think of it as “from one, select many” or “from one, select zero or more”. In our case, we can think of **`SelectMany`** as a way to repeat some processing logic several times and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. +The [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) operator can be a tricky one to understand. Lee Campbell’s excellent [Introduction to Rx](http://introtorx.com/Content/v1.0.10621.0/08_Transformation.html#SelectMany) book does a good job of summarising its utility, suggesting we think of it as “from one, select many” or “from one, select zero or more”. In our case, we can think of [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) as a way to repeat some processing logic several times and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. -Create a **`KeyDown`** node followed by a **`SelectMany`**. Set the `Filter` for the **`KeyDown`** to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the **`SelectMany`** node add a **`SubscribeSubject`** and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a **`TakeUntil`** node after the **`SubscribeSubject`** and connect the output of **`TakeUntil`** to the **`WorkflowOutput`** (disconnecting the `Source` node). Finally, create a **`KeyUp`** node and connect it to **`TakeUntil`**. Set the key `Filter` for **`KeyUp`** to the same as previously created **`KeyDown`** node outside the **`SelectMany`**. +Create a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) node followed by a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node add a [**`SubscribeSubject`**](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil) node after the [**`SubscribeSubject`**](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil) to the [**`WorkflowOutput`**](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` node). Finally, create a [**`KeyUp`**](xref:Bonsai.Windows.Input.KeyUp) node and connect it to [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [**`KeyUp`**](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) node outside the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany). :::workflow ![SelectMany detour](~/workflows/select-many-detour.bonsai) ::: -Run the project and inspect the output of the **`SelectMany`** node. If no client messages are triggered and we press ‘A’ to trigger the **`SelectMany`** nothing will be returned. If we trigger a single client and press ‘A’ again **`SelectMany`** gives us the address of that client. If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger **`SelectMany`** with a **`KeyDown`** we generate a new sequence that immediately subscribes to **`ClientAddresses`**, a **`ReplaySubject`** which replays all our unique client addresses into the sequence. We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the **`SusbcribeSubject`** directly to the workflow output and deleting **`KeyUp`** and **`TakeUntil`**). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger **`TakeUntil`** and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. +Run the project and inspect the output of the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node. If no client messages are triggered and we press ‘A’ to trigger the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) nothing will be returned. If we trigger a single client and press ‘A’ again [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) gives us the address of that client. If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) with a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) we generate a new sequence that immediately subscribes to `ClientAddresses`, a [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) which replays all our unique client addresses into the sequence. We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the [**`SusbcribeSubject`**](xref:Bonsai.Expressions.SubscribeSubject) directly to the workflow output and deleting [**`KeyUp`**](xref:Bonsai.Windows.Input.KeyUp) and [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil)). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil) and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. ## All client broadcast -To apply the logic of the **`SelectMany`** example to server broadcast, we need something to trigger the **`SelectMany`** sequence creation, and something to trigger termination. We already have a trigger for sequence creation in the output of the **`Router`** since we want to run our **`SelectMany`** sequence every time a client message is received. For our sequence temination trigger, we want something that is guaranteed to fire after the server receives a client message, but before the next message is received so that our **`SelectMany`** sequence for each message responds only to that particular message. A simple solution is therefore to use the arrival of the next message as our sequence termination trigger. +To apply the logic of the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) example to server broadcast, we need something to trigger the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) sequence creation, and something to trigger termination. We already have a trigger for sequence creation in the output of the [**`Router`**](xref:Bonsai.ZeroMQ.Router) since we want to run our [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) sequence every time a client message is received. For our sequence temination trigger, we want something that is guaranteed to fire after the server receives a client message, but before the next message is received so that our [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) sequence for each message responds only to that particular message. A simple solution is therefore to use the arrival of the next message as our sequence termination trigger. -To implement this, add a **`Skip`** node after the **`Router`** in a separate branch and connect this to a **`PublishSubject`**. Ensure that the **`Skip`** node's `Count` property is set to 1, and name the **`PublishSubject`** 'NextMessage'. +To implement this, add a [**`Skip`**](xref:Bonsai.Reactive.Skip) node after the [**`Router`**](xref:Bonsai.ZeroMQ.Router) in a separate branch and connect this to a [**`PublishSubject`**](xref:Bonsai.Reactive.PublishSubject). Ensure that the [**`Skip`**](xref:Bonsai.Reactive.Skip) node's `Count` property is set to 1, and name the [**`PublishSubject`**](xref:Bonsai.Reactive.PublishSubject) 'NextMessage'. :::workflow ![Server next message](~/workflows/server-next-message.bonsai) ::: -The logic here is that we use **`Skip`** to create a sequence that lags exactly 1 message behind the **`Router`** sequence of received messages, i.e. when the first message is received, **`NextMessage`** will not produce a result until the second message is received. We can then use this inside our **`SelectMany`** logic for generating server messages. Add a **`SelectMany`** node after the **`Router`** in a separate branch and name it ‘SelectAllClients’. +The logic here is that we use [**`Skip`**](xref:Bonsai.Reactive.Skip) to create a sequence that lags exactly 1 message behind the [**`Router`**](xref:Bonsai.ZeroMQ.Router) sequence of received messages, i.e. when the first message is received, `NextMessage` will not produce a result until the second message is received. We can then use this inside our [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) logic for generating server messages. Add a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node after the [**`Router`**](xref:Bonsai.ZeroMQ.Router) in a separate branch and name it ‘SelectAllClients’. -Inside the **`SelectMany`** node, create 2 **`SubscribeSubject`** nodes and link them to the **`ClientAddresses`** and **`NextMessage`** subjects. Connect the **`ClientAddresses`** subscription to the workflow output via a **`TakeUntil`** node and use **`NextMessage`** as the second input. Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a **`WithLatestFrom`** with the **`Router`** as its second input. In this context **`WithLatestFrom`** combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. +Inside the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node, create 2 [**`SubscribeSubject`**](xref:Bonsai.Expressions.SubscribeSubject) nodes and link them to the `ClientAddresses` and `NextMessage` subjects. Connect the `ClientAddresses` subscription to the workflow output via a [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil) node and use `NextMessage` as the second input. Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a [**`WithLatestFrom`**](xref:Bonsai.Reactive.WithLatestFrom) with the [**`Router`**](xref:Bonsai.ZeroMQ.Router) as its second input. In this context [**`WithLatestFrom`**](xref:Bonsai.Reactive.WithLatestFrom) combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. :::workflow ![Select all clients and package message](~/workflows/format-select-all-clients.bonsai) ::: -To send these messages back to our clients, we will modify the logic in our previous **`BounceBack`** node. This time, we'll create a **`SelectMany`** called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. This is multicast back into the router to send the original address back to all clients. +To send these messages back to our clients, we will modify the logic in our previous `BounceBack` node. This time, we'll create a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. This is multicast back into the router to send the original address back to all clients. -Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of client ID / address and the received message. We take the `byte[]` address and use **`ConvertToFrame`** to convert it to a `NetMQFrame` and then merge it with the empty delimiter and the message payload. As before, we **`Take`** 3 elements to close the message construction stream, convert to a `NetMQMessage` with **`ToMessage`** and then **`Multicast`** into `RouterMessages`. If you run the workflow now you should see that each time a **`Dealer`** produces a message, all clients receive a copy of that message. +Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of client ID / address and the received message. We take the `byte[]` address and use [**`ConvertToFrame`**](xref:Bonsai.ZeroMQ.ConvertToFrame) to convert it to a `NetMQFrame` and then merge it with the empty delimiter and the message payload. As before, we [**`Take`**](xref:Bonsai.Reactive.Take) 3 elements to close the message construction stream, convert to a `NetMQMessage` with [**`ToMessage`**](xref:Bonsai.ZeroMQ.ToMessage) and then [**`Multicast`**](xref:Bonsai.Expressions.MulticastSubject) into `RouterMessages`. If you run the workflow now you should see that each time a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) produces a message, all clients receive a copy of that message. :::workflow ![Broadcast to all clients](~/workflows/broadcast-all-clients.bonsai) @@ -159,10 +159,10 @@ Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of ## Leave-one-out broadcast This is getting pretty close to our original network architecture goal but there is still some redundancy present. When client 1 sends a message to the server, clients 1, 2 and 3 all receive a copy of that message back from the server. this is fine for clients 2 and 3 as they are not aware of client 1's messages without server communication; but client 1 does not need this message copy since it already originated the message. Our goal then is that the server should send message copies back to all clients except the client that originated message. -To do this, we'll create a **`Condition`** before `BroadcastAll` to filter only non sender clients called `NonSenderClients`. Inside `NonSenderClients` expose the `byte` corresponding to the client ID and index 1 of the first frame of the `NetMQMessage` from `Source1`. **`Zip`** these together and use **`NotEqual`** to compare the client ID of the message with the existing clients and discard where the IDs are the same. +To do this, we'll create a [**`Condition`**](xref:Bonsai.Reactive.Condition) before `BroadcastAll` to filter only non sender clients called `NonSenderClients`. Inside `NonSenderClients` expose the `byte` corresponding to the client ID and index 1 of the first frame of the `NetMQMessage` from `Source1`. [**`Zip`**](xref:Bonsai.Reactive.Zip) these together and use [**`NotEqual`**](xref:Bonsai.Expressions.NotEqualBuilder) to compare the client ID of the message with the existing clients and discard where the IDs are the same. :::workflow ![Broadcast to non-senders](~/workflows/broadcast-non-sender-clients.bonsai) ::: -Running the workflow you should see that we have now achieved the desired architecture. When a client **`Dealer`** sends a message, it is broadcast to all other joined clients except for itself. +Running the workflow you should see that we have now achieved the desired architecture. When a client [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) sends a message, it is broadcast to all other joined clients except for itself. From b95a8d9e7fc1df9178646f64e43413f14e71cb49 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Mon, 13 Jan 2025 11:33:31 +0000 Subject: [PATCH 12/37] Add initial dealer workflow --- docs/workflows/dealer-basic-input.bonsai | 2 +- docs/workflows/dealer-basic-input.bonsai.layout | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/workflows/dealer-basic-input.bonsai b/docs/workflows/dealer-basic-input.bonsai index 80ca89e..123d71a 100644 --- a/docs/workflows/dealer-basic-input.bonsai +++ b/docs/workflows/dealer-basic-input.bonsai @@ -1,5 +1,5 @@  -280 - 334 + 336 356 Normal @@ -37,7 +37,7 @@ 119 - 334 + 336 156 Normal From b9e7e77846e342f6312b93e80aa099ece280ee9a Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Mon, 13 Jan 2025 11:43:59 +0000 Subject: [PATCH 13/37] Fix typos in xrefs --- docs/articles/client-server.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 6c15a6b..40e46d5 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -73,13 +73,13 @@ So far our network is rather one-sided. We can send client messages to the serve ![Address key-value pair](~/workflows/address-kvp.bonsai) ::: -Every time the [**`Router`**](xref:Bonsai.ZeroMQ.Router) receives a message, the [**`Zip`**](xref:Bonsai.Reactive.Zip) will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. Next, add a [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) node after the [**`Zip`**](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). +Every time the [**`Router`**](xref:Bonsai.ZeroMQ.Router) receives a message, the [**`Zip`**](xref:Bonsai.Reactive.Zip) will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. Next, add a [**`DistinctBy`**](xref:Bonsai.Reactive.DistinctBy) node after the [**`Zip`**](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). :::workflow ![Unique key-value pair](~/workflows/unique-kvp.bonsai) ::: -The [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) node filters the output of [**`Zip`**](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [**`Zip`**](xref:Bonsai.Reactive.Zip). The output of [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. Add a [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) node after [**`DistinctBy`**](xref:Bonsai.Reactive.DisctinctBy) and name it ‘ClientAddresses’. +The [**`DistinctBy`**](xref:Bonsai.Reactive.DistinctBy) node filters the output of [**`Zip`**](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [**`Zip`**](xref:Bonsai.Reactive.Zip). The output of [**`DistinctBy`**](xref:Bonsai.Reactive.DistinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. Add a [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) node after [**`DistinctBy`**](xref:Bonsai.Reactive.DistinctBy) and name it ‘ClientAddresses’. :::workflow ![Address ReplaySubject](~/workflows/address-replay-subject.bonsai) From fe65c461674956183daf7cbf086cba0ced8e9464 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Tue, 14 Jan 2025 09:46:20 +0000 Subject: [PATCH 14/37] Remove layouts and svgs --- docs/workflows/address-kvp.bonsai.layout | 222 ---------- docs/workflows/address-kvp.svg | 3 - .../address-replay-subject.bonsai.layout | 250 ------------ docs/workflows/address-replay-subject.svg | 3 - .../broadcast-all-clients.bonsai.layout | 281 ------------- docs/workflows/broadcast-all.bonsai.layout | 385 ------------------ ...broadcast-non-sender-clients.bonsai.layout | 317 -------------- .../dealer-basic-input.bonsai.layout | 49 --- docs/workflows/dealer-basic-input.svg | 3 - .../format-select-all-clients.bonsai.layout | 285 ------------- docs/workflows/format-select-all-clients.svg | 3 - docs/workflows/multiple-clients.bonsai.layout | 113 ----- docs/workflows/multiple-clients.svg | 3 - .../router-message-parsing.bonsai.layout | 206 ---------- docs/workflows/router-message-parsing.svg | 3 - .../select-all-clients.bonsai.layout | 277 ------------- .../select-many-detour.bonsai.layout | 249 ----------- docs/workflows/select-many-detour.svg | 3 - .../server-basic-response.bonsai.layout | 262 ------------ docs/workflows/server-basic-response.svg | 3 - .../server-message-format.bonsai.layout | 262 ------------ .../server-message-multicast.bonsai.layout | 233 ----------- docs/workflows/server-message-multicast.svg | 3 - .../server-next-message.bonsai.layout | 273 ------------- docs/workflows/server-next-message.svg | 3 - docs/workflows/unique-kvp.bonsai.layout | 238 ----------- docs/workflows/unique-kvp.svg | 3 - 27 files changed, 3935 deletions(-) delete mode 100644 docs/workflows/address-kvp.bonsai.layout delete mode 100644 docs/workflows/address-kvp.svg delete mode 100644 docs/workflows/address-replay-subject.bonsai.layout delete mode 100644 docs/workflows/address-replay-subject.svg delete mode 100644 docs/workflows/broadcast-all-clients.bonsai.layout delete mode 100644 docs/workflows/broadcast-all.bonsai.layout delete mode 100644 docs/workflows/broadcast-non-sender-clients.bonsai.layout delete mode 100644 docs/workflows/dealer-basic-input.bonsai.layout delete mode 100644 docs/workflows/dealer-basic-input.svg delete mode 100644 docs/workflows/format-select-all-clients.bonsai.layout delete mode 100644 docs/workflows/format-select-all-clients.svg delete mode 100644 docs/workflows/multiple-clients.bonsai.layout delete mode 100644 docs/workflows/multiple-clients.svg delete mode 100644 docs/workflows/router-message-parsing.bonsai.layout delete mode 100644 docs/workflows/router-message-parsing.svg delete mode 100644 docs/workflows/select-all-clients.bonsai.layout delete mode 100644 docs/workflows/select-many-detour.bonsai.layout delete mode 100644 docs/workflows/select-many-detour.svg delete mode 100644 docs/workflows/server-basic-response.bonsai.layout delete mode 100644 docs/workflows/server-basic-response.svg delete mode 100644 docs/workflows/server-message-format.bonsai.layout delete mode 100644 docs/workflows/server-message-multicast.bonsai.layout delete mode 100644 docs/workflows/server-message-multicast.svg delete mode 100644 docs/workflows/server-next-message.bonsai.layout delete mode 100644 docs/workflows/server-next-message.svg delete mode 100644 docs/workflows/unique-kvp.bonsai.layout delete mode 100644 docs/workflows/unique-kvp.svg diff --git a/docs/workflows/address-kvp.bonsai.layout b/docs/workflows/address-kvp.bonsai.layout deleted file mode 100644 index 9dda498..0000000 --- a/docs/workflows/address-kvp.bonsai.layout +++ /dev/null @@ -1,222 +0,0 @@ - - - - false - - 144 - 156 - - - 334 - 320 - - Normal - - - true - - 120 - 130 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 458 - 308 - - - 413 - 278 - - Normal - Bonsai.Design.Visualizers.TimeSeriesVisualizer - - - 640 - 181 - 183.5 - true - - - - - true - - 149 - 468 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - true - - 457 - 130 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - true - - 472 - 189 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/address-kvp.svg b/docs/workflows/address-kvp.svg deleted file mode 100644 index b4439a1..0000000 --- a/docs/workflows/address-kvp.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>ZipConvertToStringDealerDealerDealerIndexRequest.LastStringStringStringBufferKeyDownKeyDownKeyDownRequest.FirstRouter \ No newline at end of file diff --git a/docs/workflows/address-replay-subject.bonsai.layout b/docs/workflows/address-replay-subject.bonsai.layout deleted file mode 100644 index f49d2ac..0000000 --- a/docs/workflows/address-replay-subject.bonsai.layout +++ /dev/null @@ -1,250 +0,0 @@ - - - - false - - 144 - 156 - - - 334 - 320 - - Normal - - - false - - 120 - 130 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 458 - 308 - - - 413 - 278 - - Normal - Bonsai.Design.Visualizers.TimeSeriesVisualizer - - - 640 - 56 - 58.5 - true - - - - - true - - 149 - 468 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 24 - 26 - - - 334 - 500 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 457 - 130 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 472 - 189 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/address-replay-subject.svg b/docs/workflows/address-replay-subject.svg deleted file mode 100644 index fe1a5d0..0000000 --- a/docs/workflows/address-replay-subject.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>ClientAddressesConvertToStringDealerDealerDealerDistinctByRequest.LastStringStringStringZipKeyDownKeyDownKeyDownIndexBufferRequest.FirstRouter \ No newline at end of file diff --git a/docs/workflows/broadcast-all-clients.bonsai.layout b/docs/workflows/broadcast-all-clients.bonsai.layout deleted file mode 100644 index e877db5..0000000 --- a/docs/workflows/broadcast-all-clients.bonsai.layout +++ /dev/null @@ -1,281 +0,0 @@ - - - - false - - 72 - 78 - - - 334 - 214 - - Normal - - - false - - 703 - 201 - - - 334 - 320 - - Normal - - - false - - 48 - 52 - - - 413 - 278 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 890 - 207 - - - 334 - 317 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 700 - 168 - - - 674 - 349 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 188 - 461 - - - 334 - 141 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 513 - 457 - - - 334 - 148 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 836 - 461 - - - 334 - 144 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - \ No newline at end of file diff --git a/docs/workflows/broadcast-all.bonsai.layout b/docs/workflows/broadcast-all.bonsai.layout deleted file mode 100644 index 8df6bb7..0000000 --- a/docs/workflows/broadcast-all.bonsai.layout +++ /dev/null @@ -1,385 +0,0 @@ - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 144 - 156 - - - 334 - 296 - - Normal - - - false - - 159 - 427 - - - 334 - 375 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 145 - 41 - - - 611 - 439 - - Normal - - - - true - - 120 - 130 - - - 334 - 396 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - false - - 338 - 241 - - - 399 - 327 - - Normal - - - - false - - 48 - 52 - - - 334 - 305 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 144 - 156 - - - 334 - 64 - - Normal - - false - - 229 - 154 - - - 738 - 436 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 24 - 26 - - - 334 - 290 - - Normal - - - false - - 859 - 203 - - - 334 - 254 - - Normal - - - false - - 96 - 104 - - - 334 - 256 - - Normal - - - false - - 169 - 186 - - - 334 - 308 - - Normal - - - true - - 173 - 409 - - - 334 - 327 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 168 - 182 - - - 334 - 64 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 528 - 309 - - - 334 - 301 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 144 - 156 - - - 334 - 200 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 856 - 293 - - - 334 - 319 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - \ No newline at end of file diff --git a/docs/workflows/broadcast-non-sender-clients.bonsai.layout b/docs/workflows/broadcast-non-sender-clients.bonsai.layout deleted file mode 100644 index 3cf2ff7..0000000 --- a/docs/workflows/broadcast-non-sender-clients.bonsai.layout +++ /dev/null @@ -1,317 +0,0 @@ - - - - false - - 72 - 78 - - - 334 - 214 - - Normal - - - false - - 703 - 201 - - - 334 - 320 - - Normal - - - false - - 48 - 52 - - - 413 - 278 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 890 - 207 - - - 334 - 317 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 714 - 201 - - - 415 - 342 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 758 - 201 - - - 530 - 238 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 552 - 259 - - - 674 - 349 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 188 - 461 - - - 334 - 141 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 513 - 457 - - - 334 - 148 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 836 - 461 - - - 334 - 144 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - \ No newline at end of file diff --git a/docs/workflows/dealer-basic-input.bonsai.layout b/docs/workflows/dealer-basic-input.bonsai.layout deleted file mode 100644 index ef0cb82..0000000 --- a/docs/workflows/dealer-basic-input.bonsai.layout +++ /dev/null @@ -1,49 +0,0 @@ - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 541 - 280 - - - 336 - 356 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - true - - 543 - 119 - - - 336 - 156 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - \ No newline at end of file diff --git a/docs/workflows/dealer-basic-input.svg b/docs/workflows/dealer-basic-input.svg deleted file mode 100644 index 31094d4..0000000 --- a/docs/workflows/dealer-basic-input.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>DealerStringKeyDown \ No newline at end of file diff --git a/docs/workflows/format-select-all-clients.bonsai.layout b/docs/workflows/format-select-all-clients.bonsai.layout deleted file mode 100644 index f4ed4b4..0000000 --- a/docs/workflows/format-select-all-clients.bonsai.layout +++ /dev/null @@ -1,285 +0,0 @@ - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 703 - 201 - - - 334 - 320 - - Normal - - - false - - 48 - 52 - - - 413 - 278 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 38 - 266 - - - 458 - 375 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 890 - 207 - - - 334 - 317 - - Normal - - - false - - 722 - 280 - - - 334 - 434 - - Normal - - false - - 706 - 226 - - - 651 - 435 - - Normal - - - - false - - 24 - 26 - - - 334 - 326 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 195 - 461 - - - 334 - 141 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 724 - 450 - - - 334 - 148 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 741 - 97 - - - 334 - 144 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/format-select-all-clients.svg b/docs/workflows/format-select-all-clients.svg deleted file mode 100644 index 0640ef8..0000000 --- a/docs/workflows/format-select-all-clients.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>ClientAddressesBounceBackNextMessageWithLatestFromDealerDealerDealerDistinctBySkipSelectAllClientsStringStringStringZipKeyDownKeyDownKeyDownIndexFirst.BufferRouterRouterMessages \ No newline at end of file diff --git a/docs/workflows/multiple-clients.bonsai.layout b/docs/workflows/multiple-clients.bonsai.layout deleted file mode 100644 index 1cd0cc8..0000000 --- a/docs/workflows/multiple-clients.bonsai.layout +++ /dev/null @@ -1,113 +0,0 @@ - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/multiple-clients.svg b/docs/workflows/multiple-clients.svg deleted file mode 100644 index 2471822..0000000 --- a/docs/workflows/multiple-clients.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>DealerDealerDealerStringStringStringKeyDownKeyDownKeyDown \ No newline at end of file diff --git a/docs/workflows/router-message-parsing.bonsai.layout b/docs/workflows/router-message-parsing.bonsai.layout deleted file mode 100644 index eb127c3..0000000 --- a/docs/workflows/router-message-parsing.bonsai.layout +++ /dev/null @@ -1,206 +0,0 @@ - - - - false - - 144 - 156 - - - 334 - 320 - - Normal - - - true - - 120 - 130 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 458 - 308 - - - 413 - 278 - - Normal - Bonsai.Design.Visualizers.TimeSeriesVisualizer - - - 640 - 200 - 202.5 - true - - - - - true - - 457 - 130 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - true - - 472 - 189 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/router-message-parsing.svg b/docs/workflows/router-message-parsing.svg deleted file mode 100644 index a64db56..0000000 --- a/docs/workflows/router-message-parsing.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>IndexConvertToStringDealerDealerDealerBufferRequest.LastStringStringStringRequest.FirstKeyDownKeyDownKeyDownRouter \ No newline at end of file diff --git a/docs/workflows/select-all-clients.bonsai.layout b/docs/workflows/select-all-clients.bonsai.layout deleted file mode 100644 index f028879..0000000 --- a/docs/workflows/select-all-clients.bonsai.layout +++ /dev/null @@ -1,277 +0,0 @@ - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 703 - 201 - - - 334 - 320 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 168 - 197 - - - 458 - 375 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 890 - 207 - - - 334 - 317 - - Normal - - - false - - 722 - 280 - - - 334 - 434 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - false - - 706 - 226 - - - 651 - 435 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 96 - 104 - - - 334 - 141 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/select-many-detour.bonsai.layout b/docs/workflows/select-many-detour.bonsai.layout deleted file mode 100644 index 6d742ea..0000000 --- a/docs/workflows/select-many-detour.bonsai.layout +++ /dev/null @@ -1,249 +0,0 @@ - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 703 - 201 - - - 334 - 320 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 431 - 208 - - - 646 - 486 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 760 - 256 - - - 334 - 195 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - \ No newline at end of file diff --git a/docs/workflows/select-many-detour.svg b/docs/workflows/select-many-detour.svg deleted file mode 100644 index 932a5c7..0000000 --- a/docs/workflows/select-many-detour.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>ClientAddressesBounceBackDealerDealerDealerSelectManyDistinctByStringStringStringKeyDownZipKeyDownKeyDownKeyDownIndexFirst.BufferRouterRouterMessages \ No newline at end of file diff --git a/docs/workflows/server-basic-response.bonsai.layout b/docs/workflows/server-basic-response.bonsai.layout deleted file mode 100644 index d45fa24..0000000 --- a/docs/workflows/server-basic-response.bonsai.layout +++ /dev/null @@ -1,262 +0,0 @@ - - - - false - - 144 - 156 - - - 334 - 320 - - Normal - - - false - - 120 - 130 - - - 334 - 64 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 753 - 366 - - - 413 - 278 - - Normal - Bonsai.Design.Visualizers.TimeSeriesVisualizer - - - 640 - 38 - 40.5 - true - - - - - true - - 149 - 468 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 24 - 26 - - - 334 - 500 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 457 - 130 - - - 334 - 64 - - Normal - - - false - - 472 - 189 - - - 334 - 64 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 808 - 210 - - - 314 - 238 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 96 - 104 - - - 334 - 184 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/server-basic-response.svg b/docs/workflows/server-basic-response.svg deleted file mode 100644 index aa56963..0000000 --- a/docs/workflows/server-basic-response.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>ClientAddressesConvertToStringSendResponseDealerDealerDealerDistinctByRequest.LastStringStringStringZipKeyDownKeyDownKeyDownIndexBufferRequest.FirstRouter \ No newline at end of file diff --git a/docs/workflows/server-message-format.bonsai.layout b/docs/workflows/server-message-format.bonsai.layout deleted file mode 100644 index d45fa24..0000000 --- a/docs/workflows/server-message-format.bonsai.layout +++ /dev/null @@ -1,262 +0,0 @@ - - - - false - - 144 - 156 - - - 334 - 320 - - Normal - - - false - - 120 - 130 - - - 334 - 64 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 753 - 366 - - - 413 - 278 - - Normal - Bonsai.Design.Visualizers.TimeSeriesVisualizer - - - 640 - 38 - 40.5 - true - - - - - true - - 149 - 468 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 24 - 26 - - - 334 - 500 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 457 - 130 - - - 334 - 64 - - Normal - - - false - - 472 - 189 - - - 334 - 64 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 808 - 210 - - - 314 - 238 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 96 - 104 - - - 334 - 184 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/server-message-multicast.bonsai.layout b/docs/workflows/server-message-multicast.bonsai.layout deleted file mode 100644 index 5092152..0000000 --- a/docs/workflows/server-message-multicast.bonsai.layout +++ /dev/null @@ -1,233 +0,0 @@ - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 703 - 201 - - - 334 - 320 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 431 - 208 - - - 646 - 486 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 164 - 408 - - - 334 - 184 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 96 - 104 - - - 334 - 182 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 753 - 223 - - - 334 - 211 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - \ No newline at end of file diff --git a/docs/workflows/server-message-multicast.svg b/docs/workflows/server-message-multicast.svg deleted file mode 100644 index 8ef96ba..0000000 --- a/docs/workflows/server-message-multicast.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>ClientAddressesBounceBackDealerDealerDealerDistinctByStringStringStringZipKeyDownKeyDownKeyDownIndexFirst.BufferRouterRouterMessages \ No newline at end of file diff --git a/docs/workflows/server-next-message.bonsai.layout b/docs/workflows/server-next-message.bonsai.layout deleted file mode 100644 index 97b791e..0000000 --- a/docs/workflows/server-next-message.bonsai.layout +++ /dev/null @@ -1,273 +0,0 @@ - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 703 - 201 - - - 334 - 320 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - false - - 431 - 208 - - - 646 - 486 - - Normal - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 760 - 256 - - - 334 - 195 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - \ No newline at end of file diff --git a/docs/workflows/server-next-message.svg b/docs/workflows/server-next-message.svg deleted file mode 100644 index 9be5129..0000000 --- a/docs/workflows/server-next-message.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>ClientAddressesBounceBackNextMessageDealerDealerDealerSelectManyDistinctBySkipStringStringStringKeyDownZipKeyDownKeyDownKeyDownIndexFirst.BufferRouterRouterMessages \ No newline at end of file diff --git a/docs/workflows/unique-kvp.bonsai.layout b/docs/workflows/unique-kvp.bonsai.layout deleted file mode 100644 index bc43f5c..0000000 --- a/docs/workflows/unique-kvp.bonsai.layout +++ /dev/null @@ -1,238 +0,0 @@ - - - - false - - 144 - 156 - - - 334 - 320 - - Normal - - - false - - 120 - 130 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - true - - 458 - 308 - - - 413 - 278 - - Normal - Bonsai.Design.Visualizers.TimeSeriesVisualizer - - - 640 - 56 - 58.5 - true - - - - - true - - 149 - 468 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 24 - 26 - - - 334 - 500 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 457 - 130 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 472 - 189 - - - 334 - 64 - - Normal - Bonsai.Design.ObjectTextVisualizer - - - - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - - false - - 0 - 0 - - - 0 - 0 - - Normal - - \ No newline at end of file diff --git a/docs/workflows/unique-kvp.svg b/docs/workflows/unique-kvp.svg deleted file mode 100644 index 3fdb6d4..0000000 --- a/docs/workflows/unique-kvp.svg +++ /dev/null @@ -1,3 +0,0 @@ - -]>DistinctByConvertToStringDealerDealerDealerZipRequest.LastStringStringStringIndexKeyDownKeyDownKeyDownBufferRequest.FirstRouter \ No newline at end of file From fd6fd9d6f3071043871b1e7893abc756d2c7457b Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Jan 2025 10:29:31 +0000 Subject: [PATCH 15/37] Add to tutorials subsection --- docs/articles/client-server.md | 2 +- docs/articles/toc.yml | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 40e46d5..bbc7359 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -3,7 +3,7 @@ uid: client-server title: "Client-Server" --- -Client-Server Tutorial +Client-server tutorial ====================== The Bonsai.ZeroMQ package allows us to harness the powerful [ZeroMQ](https://zeromq.org/) library to build networked applications in Bonsai. Applications could include: diff --git a/docs/articles/toc.yml b/docs/articles/toc.yml index fddbdba..54bbdec 100644 --- a/docs/articles/toc.yml +++ b/docs/articles/toc.yml @@ -7,6 +7,6 @@ - href: push-pull.md - href: proxy.md - name: Recipes - href: recipes.md -- name: Client-Server Tutorial - href: client-server.md \ No newline at end of file +- href: recipes.md +- name: Tutorials +- href: client-server.md \ No newline at end of file From 48fcd3b79b66de5333eceb05bd70f920ae96f7c3 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Jan 2025 10:33:56 +0000 Subject: [PATCH 16/37] Add windows input package to local bonsai environment --- .bonsai/Bonsai.config | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.bonsai/Bonsai.config b/.bonsai/Bonsai.config index a85bb18..2a57896 100644 --- a/.bonsai/Bonsai.config +++ b/.bonsai/Bonsai.config @@ -9,6 +9,7 @@ + @@ -29,6 +30,7 @@ + @@ -39,6 +41,7 @@ + From 0c7c820dc0faa77b88272fe017fb54e019503ae9 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Jan 2025 10:43:33 +0000 Subject: [PATCH 17/37] Reformat operator references --- docs/articles/client-server.md | 65 ++++++++++++++++++---------------- 1 file changed, 35 insertions(+), 30 deletions(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index bbc7359..3c51e99 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -35,19 +35,24 @@ ZeroMQ provides a number of socket types that could be used to achieve something - Messages can be passed between Router / Dealer sockets without the requirement that a reply is received before the next message is sent, as is the case with the Request / Response socket pair. ## Basic client -To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project, add a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node. In the `ConnectionString` property, set `Address`: localhost:5557, `Action`: Connect, `Protocol`: TCP. +To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project: +- Add a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node. +- In the `ConnectionString` property, set `Address`: localhost:5557, `Action`: Connect, `Protocol`: TCP. -In Bonsai.ZeroMQ, the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) can have two functions based on its inputs. On its own, as above, the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. +> [!Note] +> In Bonsai.ZeroMQ, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) can have two functions based on its inputs. On its own, as above, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. -If we add inputs to the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer), it will act as both a sender and receiver of messages on the network. Before the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node add a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) and [**`String`**](xref:Bonsai.Expressions.StringProperty) node in sequence as input to the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer). +If we add inputs to the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer), it will act as both a sender and receiver of messages on the network. +- Before the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator add a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) +- Add a [`String`](xref:Bonsai.Expressions.StringProperty) operator in sequence as input to the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). :::workflow ![Basic Dealer input](~/workflows/dealer-basic-input.bonsai) ::: -In the node properties, set the [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) `Filter` to the ‘1’ key and set the [**`String`**](xref:Bonsai.Expressions.StringProperty) `Value` to ‘Client1’. If we run the Bonsai project now, the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) will continue listening for incoming messages on the network, but every time the ‘1’ key is pressed a message containing the string ‘Client1’ will be sent from the socket. +In the node properties, set the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) `Filter` to the ‘1’ key and set the [`String`](xref:Bonsai.Expressions.StringProperty) `Value` to ‘Client1’. If we run the Bonsai project now, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) will continue listening for incoming messages on the network, but every time the ‘1’ key is pressed a message containing the string ‘Client1’ will be sent from the socket. -Copy and paste this client structure a couple of times and change the [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) and [**`String`**](xref:Bonsai.Expressions.StringProperty) properties accordingly on each (2, ‘Client2’; 3, ‘Client3’) so that we have 3 total clients that send messages according to different key presses. +Copy and paste this client structure a couple of times and change the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) and [`String`](xref:Bonsai.Expressions.StringProperty) properties accordingly on each (2, ‘Client2’; 3, ‘Client3’) so that we have 3 total clients that send messages according to different key presses. :::workflow ![Multiple clients](~/workflows/multiple-clients.bonsai) @@ -56,62 +61,62 @@ Copy and paste this client structure a couple of times and change the [**`KeyDow > For the purposes of this article we are creating all of our clients and our server on the same Bonsai project and same machine for ease of demonstration. In a working example, each client and server could be running in separate Bonsai instances on different machines on a network. In this case, localhost would be replaced with the server machine’s IP address. ## Basic server -Now that we have our client pool set up and sending messages, let’s implement a server to listen for those messages. Add a [**`Router`**](xref:Bonsai.ZeroMQ.Router) node to the project and set its properties to match the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) sockets we already added so that it is running on the same network. As the [**`Router`**](xref:Bonsai.ZeroMQ.Router) is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. +Now that we have our client pool set up and sending messages, let’s implement a server to listen for those messages. Add a [`Router`](xref:Bonsai.ZeroMQ.Router) node to the project and set its properties to match the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) sockets we already added so that it is running on the same network. As the [`Router`](xref:Bonsai.ZeroMQ.Router) is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. -As with the [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) node, a [**`Router`**](xref:Bonsai.ZeroMQ.Router) node without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [**`Router`**](xref:Bonsai.ZeroMQ.Router) node, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [**`Router`**](xref:Bonsai.ZeroMQ.Router). Expanding the output the the [**`Router`**](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. To make sense of the message, let's expose the `Buffer` `byte[]` of the `First` frame. Add an [**`Index`**](xref:Bonsai.Expressions.IndexBuilder) node the the first frame buffer and set its `Value` to 1 to access the unique address ID. Add a [**`ConvertToString`**](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. +As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node, a [`Router`](xref:Bonsai.ZeroMQ.Router) node without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) node, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [`Router`](xref:Bonsai.ZeroMQ.Router). Expanding the output the the [`Router`](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. To make sense of the message, let's expose the `Buffer` `byte[]` of the `First` frame. Add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) node the the first frame buffer and set its `Value` to 1 to access the unique address ID. Add a [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. :::workflow ![Router message parsing](~/workflows/router-message-parsing.bonsai) ::: -Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the [**`Index`**](xref:Bonsai.Expressions.IndexBuilder) node output and a corresponding `string` in the [**`ConvertToString`**](xref:Bonsai.ZeroMQ.ConvertToString) node output. +Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the [`Index`](xref:Bonsai.Expressions.IndexBuilder) node output and a corresponding `string` in the [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) node output. ## Client address tracking -So far our network is rather one-sided. We can send client messages to the server which can in turn receive and parse them, but currently nothing is relayed back to the clients. The first goal for server feedbasck is that any time a client message is received, the server sends this message back to all connected clients. To do this, we first need a way of keeping track of all active clients. Add a [**`Zip`**](xref:Bonsai.Reactive.Zip) node to the [**`Index`**](xref:Bonsai.Expressions.IndexBuilder) node and connect the `byte[]` `Buffer` as the second input. +So far our network is rather one-sided. We can send client messages to the server which can in turn receive and parse them, but currently nothing is relayed back to the clients. The first goal for server feedbasck is that any time a client message is received, the server sends this message back to all connected clients. To do this, we first need a way of keeping track of all active clients. Add a [`Zip`](xref:Bonsai.Reactive.Zip) node to the [`Index`](xref:Bonsai.Expressions.IndexBuilder) node and connect the `byte[]` `Buffer` as the second input. :::workflow ![Address key-value pair](~/workflows/address-kvp.bonsai) ::: -Every time the [**`Router`**](xref:Bonsai.ZeroMQ.Router) receives a message, the [**`Zip`**](xref:Bonsai.Reactive.Zip) will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. Next, add a [**`DistinctBy`**](xref:Bonsai.Reactive.DistinctBy) node after the [**`Zip`**](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). +Every time the [`Router`](xref:Bonsai.ZeroMQ.Router) receives a message, the [`Zip`](xref:Bonsai.Reactive.Zip) will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. Next, add a [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node after the [`Zip`](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). :::workflow ![Unique key-value pair](~/workflows/unique-kvp.bonsai) ::: -The [**`DistinctBy`**](xref:Bonsai.Reactive.DistinctBy) node filters the output of [**`Zip`**](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [**`Zip`**](xref:Bonsai.Reactive.Zip). The output of [**`DistinctBy`**](xref:Bonsai.Reactive.DistinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. Add a [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) node after [**`DistinctBy`**](xref:Bonsai.Reactive.DistinctBy) and name it ‘ClientAddresses’. +The [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node filters the output of [`Zip`](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [`Zip`](xref:Bonsai.Reactive.Zip). The output of [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. Add a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) node after [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) and name it ‘ClientAddresses’. :::workflow ![Address ReplaySubject](~/workflows/address-replay-subject.bonsai) ::: -A [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) has the useful feature that it stores its input sequence and replays those values to any current or future subscribers. The effect in this case is that anything that subscribes to `ClientAddresses` will receive all the unique client addresses encountered by the server so far. +A [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) has the useful feature that it stores its input sequence and replays those values to any current or future subscribers. The effect in this case is that anything that subscribes to `ClientAddresses` will receive all the unique client addresses encountered by the server so far. ## Server --> client communication -Eventually, we will use these unique client addresses to route server messages back to specific client. For now, we'll implement a more basic approach where the server just sends messages back to the client that originally sent them. The Bonsai ZeroMQ library provides a convenient node for this task in the form of [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse). Add this after the [**`Router`**](xref:Bonsai.ZeroMQ.Router) in a separate branch, and inside (double-click on [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse)) add a [**`String`**](xref:Bonsai.Expressions.StringProperty) node with a generic response value like `ServerResponse` after the **`Source`** node. +Eventually, we will use these unique client addresses to route server messages back to specific client. For now, we'll implement a more basic approach where the server just sends messages back to the client that originally sent them. The Bonsai ZeroMQ library provides a convenient node for this task in the form of [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse). Add this after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch, and inside (double-click on [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse)) add a [`String`](xref:Bonsai.Expressions.StringProperty) node with a generic response value like `ServerResponse` after the `Source` node. :::workflow ![Basic server response](~/workflows/server-basic-response.bonsai) ::: -The [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) node has a couple of interesting properties which may not be immediately obvious from this simple example. First, this node always transmits its response back to the ZeroMQ socket that initiated the request (in this case one of our [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) clients) and we therefore do not need to specify an address in its processing logic. Second, the internal flow of [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) logic is computed asynchronously. This is very useful for responses that require more intensive computation and allows a [**`Router`**](xref:Bonsai.ZeroMQ.Router) to deal with frequent incoming [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) requests efficiently. +The [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node has a couple of interesting properties which may not be immediately obvious from this simple example. First, this node always transmits its response back to the ZeroMQ socket that initiated the request (in this case one of our [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) clients) and we therefore do not need to specify an address in its processing logic. Second, the internal flow of [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) logic is computed asynchronously. This is very useful for responses that require more intensive computation and allows a [`Router`](xref:Bonsai.ZeroMQ.Router) to deal with frequent incoming [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) requests efficiently. > Imagine, for example, that our Dealer sockets were sending video snippets to a Router server that is tasked with doing some processing of the video and returning the results back to the Dealers. If the responses were not computed in an asynchronous manner we would start to incur a bottleneck on the router if there were many connected Dealers or frequent Dealer requests. -Running this workflow, you should see a 'bounceback' where any [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [**`Router`**](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. Delete the [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) and [**`ConvertToString`**](xref:Bonsai.ZeroMQ.ConvertToString) branches and replace with a branch that generates a bounceback message without using the [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) node: +Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [`Router`](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. Delete the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) and [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) branches and replace with a branch that generates a bounceback message without using the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node: :::workflow ![Server message multicast](~/workflows/server-message-multicast.bonsai) ::: -We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse) node in this implementation we need to pass messages directly into the [**`Router`**](xref:Bonsai.ZeroMQ.Router). To do this we generate a [**`BehaviorSubject`**](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [**`Router`**](xref:Bonsai.ZeroMQ.Response) (can implement this by creating a [**`ToMessage`**](xref:Bonsai.ZeroMQ.ToMessage) node, right-clicking it and creating a [**`BehaviorSubject`**](xref:Bonsai.Reactive.BehaviorSubject) source). This will change the output type of the [**`Router`**](xref:Bonsai.ZeroMQ.Router) node from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. +We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node in this implementation we need to pass messages directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). To do this we generate a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [`Router`](xref:Bonsai.ZeroMQ.Response) (can implement this by creating a [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) node, right-clicking it and creating a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source). This will change the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) node from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. - We want the [**`Router`**](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [**`SendResponse`**](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [**`Router`**](xref:Bonsai.ZeroMQ.Router) with a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [**`Index`**](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new **`String`** which we convert to a `NetMQFrame` for the message content. We [**`Merge`**](xref:Bonsai.Reactive.Merge) these component frames back together and use a [**`Take`**](xref:Bonsai.Reactive.Take) node (with count = 3) followed by [**`ToMessage`**](xref:Bonsai.ZeroMQ.ToMessage). The [**`Take`**](xref:Bonsai.Reactive.Take) node is particularly important here as 1) [**`ToMessage`**](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany). Finally, we use a [**`MulticastSubject`**](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [**`Router`**](xref:Bonsai.ZeroMQ.Router). + We want the [`Router`](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [`Router`](xref:Bonsai.ZeroMQ.Router) with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [`Index`](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new `String` which we convert to a `NetMQFrame` for the message content. We [`Merge`](xref:Bonsai.Reactive.Merge) these component frames back together and use a [`Take`](xref:Bonsai.Reactive.Take) node (with count = 3) followed by [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage). The [`Take`](xref:Bonsai.Reactive.Take) node is particularly important here as 1) [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Finally, we use a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [`Router`](xref:Bonsai.ZeroMQ.Router). If we run the workflow now, we should see the same behavior as before (server bounces message back to initiating client). ## SelectMany detour - Now our network has a complete loop of client --> server --> client communication, but only the client that sends a message receives anything back from the server. Instead we’d like all clients to know when any of the clients sends a message. We already have access to the connected clients from `ClientAddresses`, and we know how to package data and send it back to clients via the [**`Router`**](xref:Bonsai.ZeroMQ.Router). In an imperative language we would do something like: + Now our network has a complete loop of client --> server --> client communication, but only the client that sends a message receives anything back from the server. Instead we’d like all clients to know when any of the clients sends a message. We already have access to the connected clients from `ClientAddresses`, and we know how to package data and send it back to clients via the [`Router`](xref:Bonsai.ZeroMQ.Router). In an imperative language we would do something like: ``` foreach (var client in Clients) { @@ -119,38 +124,38 @@ foreach (var client in Clients) { } ``` -using a loop to send the message back to each client in turn. In a reactive / observable sequence based framework we have to think about this a bit differently. The solution is to use a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) operator and it is worth taking a detour here to understand its use in some detail since we have already used it to generate our bounceback message and will apply it again for addressing multiple clients. +using a loop to send the message back to each client in turn. In a reactive / observable sequence based framework we have to think about this a bit differently. The solution is to use a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator and it is worth taking a detour here to understand its use in some detail since we have already used it to generate our bounceback message and will apply it again for addressing multiple clients. -The [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) operator can be a tricky one to understand. Lee Campbell’s excellent [Introduction to Rx](http://introtorx.com/Content/v1.0.10621.0/08_Transformation.html#SelectMany) book does a good job of summarising its utility, suggesting we think of it as “from one, select many” or “from one, select zero or more”. In our case, we can think of [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) as a way to repeat some processing logic several times and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. +The [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator can be a tricky one to understand. Lee Campbell’s excellent [Introduction to Rx](http://introtorx.com/Content/v1.0.10621.0/08_Transformation.html#SelectMany) book does a good job of summarising its utility, suggesting we think of it as “from one, select many” or “from one, select zero or more”. In our case, we can think of [`SelectMany`](xref:Bonsai.Reactive.SelectMany) as a way to repeat some processing logic several times and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. -Create a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) node followed by a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node add a [**`SubscribeSubject`**](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil) node after the [**`SubscribeSubject`**](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil) to the [**`WorkflowOutput`**](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` node). Finally, create a [**`KeyUp`**](xref:Bonsai.Windows.Input.KeyUp) node and connect it to [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [**`KeyUp`**](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) node outside the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany). +Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) node followed by a [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node add a [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) node after the [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) to the [`WorkflowOutput`](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` node). Finally, create a [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) node and connect it to [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) node outside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). :::workflow ![SelectMany detour](~/workflows/select-many-detour.bonsai) ::: -Run the project and inspect the output of the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node. If no client messages are triggered and we press ‘A’ to trigger the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) nothing will be returned. If we trigger a single client and press ‘A’ again [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) gives us the address of that client. If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) with a [**`KeyDown`**](xref:Bonsai.Windows.Input.KeyDown) we generate a new sequence that immediately subscribes to `ClientAddresses`, a [**`ReplaySubject`**](xref:Bonsai.Reactive.ReplaySubject) which replays all our unique client addresses into the sequence. We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the [**`SusbcribeSubject`**](xref:Bonsai.Expressions.SubscribeSubject) directly to the workflow output and deleting [**`KeyUp`**](xref:Bonsai.Windows.Input.KeyUp) and [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil)). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil) and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. +Run the project and inspect the output of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node. If no client messages are triggered and we press ‘A’ to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) nothing will be returned. If we trigger a single client and press ‘A’ again [`SelectMany`](xref:Bonsai.Reactive.SelectMany) gives us the address of that client. If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger [`SelectMany`](xref:Bonsai.Reactive.SelectMany) with a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) we generate a new sequence that immediately subscribes to `ClientAddresses`, a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) which replays all our unique client addresses into the sequence. We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the [`SusbcribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) directly to the workflow output and deleting [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) and [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil)). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. ## All client broadcast -To apply the logic of the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) example to server broadcast, we need something to trigger the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) sequence creation, and something to trigger termination. We already have a trigger for sequence creation in the output of the [**`Router`**](xref:Bonsai.ZeroMQ.Router) since we want to run our [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) sequence every time a client message is received. For our sequence temination trigger, we want something that is guaranteed to fire after the server receives a client message, but before the next message is received so that our [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) sequence for each message responds only to that particular message. A simple solution is therefore to use the arrival of the next message as our sequence termination trigger. +To apply the logic of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) example to server broadcast, we need something to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence creation, and something to trigger termination. We already have a trigger for sequence creation in the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) since we want to run our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence every time a client message is received. For our sequence temination trigger, we want something that is guaranteed to fire after the server receives a client message, but before the next message is received so that our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence for each message responds only to that particular message. A simple solution is therefore to use the arrival of the next message as our sequence termination trigger. -To implement this, add a [**`Skip`**](xref:Bonsai.Reactive.Skip) node after the [**`Router`**](xref:Bonsai.ZeroMQ.Router) in a separate branch and connect this to a [**`PublishSubject`**](xref:Bonsai.Reactive.PublishSubject). Ensure that the [**`Skip`**](xref:Bonsai.Reactive.Skip) node's `Count` property is set to 1, and name the [**`PublishSubject`**](xref:Bonsai.Reactive.PublishSubject) 'NextMessage'. +To implement this, add a [`Skip`](xref:Bonsai.Reactive.Skip) node after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch and connect this to a [`PublishSubject`](xref:Bonsai.Reactive.PublishSubject). Ensure that the [`Skip`](xref:Bonsai.Reactive.Skip) node's `Count` property is set to 1, and name the [`PublishSubject`](xref:Bonsai.Reactive.PublishSubject) 'NextMessage'. :::workflow ![Server next message](~/workflows/server-next-message.bonsai) ::: -The logic here is that we use [**`Skip`**](xref:Bonsai.Reactive.Skip) to create a sequence that lags exactly 1 message behind the [**`Router`**](xref:Bonsai.ZeroMQ.Router) sequence of received messages, i.e. when the first message is received, `NextMessage` will not produce a result until the second message is received. We can then use this inside our [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) logic for generating server messages. Add a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node after the [**`Router`**](xref:Bonsai.ZeroMQ.Router) in a separate branch and name it ‘SelectAllClients’. +The logic here is that we use [`Skip`](xref:Bonsai.Reactive.Skip) to create a sequence that lags exactly 1 message behind the [`Router`](xref:Bonsai.ZeroMQ.Router) sequence of received messages, i.e. when the first message is received, `NextMessage` will not produce a result until the second message is received. We can then use this inside our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) logic for generating server messages. Add a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch and name it ‘SelectAllClients’. -Inside the [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) node, create 2 [**`SubscribeSubject`**](xref:Bonsai.Expressions.SubscribeSubject) nodes and link them to the `ClientAddresses` and `NextMessage` subjects. Connect the `ClientAddresses` subscription to the workflow output via a [**`TakeUntil`**](xref:Bonsai.Reactive.TakeUntil) node and use `NextMessage` as the second input. Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a [**`WithLatestFrom`**](xref:Bonsai.Reactive.WithLatestFrom) with the [**`Router`**](xref:Bonsai.ZeroMQ.Router) as its second input. In this context [**`WithLatestFrom`**](xref:Bonsai.Reactive.WithLatestFrom) combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. +Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node, create 2 [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) nodes and link them to the `ClientAddresses` and `NextMessage` subjects. Connect the `ClientAddresses` subscription to the workflow output via a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) node and use `NextMessage` as the second input. Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a [`WithLatestFrom`](xref:Bonsai.Reactive.WithLatestFrom) with the [`Router`](xref:Bonsai.ZeroMQ.Router) as its second input. In this context [`WithLatestFrom`](xref:Bonsai.Reactive.WithLatestFrom) combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. :::workflow ![Select all clients and package message](~/workflows/format-select-all-clients.bonsai) ::: -To send these messages back to our clients, we will modify the logic in our previous `BounceBack` node. This time, we'll create a [**`SelectMany`**](xref:Bonsai.Reactive.SelectMany) called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. This is multicast back into the router to send the original address back to all clients. +To send these messages back to our clients, we will modify the logic in our previous `BounceBack` node. This time, we'll create a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. This is multicast back into the router to send the original address back to all clients. -Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of client ID / address and the received message. We take the `byte[]` address and use [**`ConvertToFrame`**](xref:Bonsai.ZeroMQ.ConvertToFrame) to convert it to a `NetMQFrame` and then merge it with the empty delimiter and the message payload. As before, we [**`Take`**](xref:Bonsai.Reactive.Take) 3 elements to close the message construction stream, convert to a `NetMQMessage` with [**`ToMessage`**](xref:Bonsai.ZeroMQ.ToMessage) and then [**`Multicast`**](xref:Bonsai.Expressions.MulticastSubject) into `RouterMessages`. If you run the workflow now you should see that each time a [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) produces a message, all clients receive a copy of that message. +Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of client ID / address and the received message. We take the `byte[]` address and use [`ConvertToFrame`](xref:Bonsai.ZeroMQ.ConvertToFrame) to convert it to a `NetMQFrame` and then merge it with the empty delimiter and the message payload. As before, we [`Take`](xref:Bonsai.Reactive.Take) 3 elements to close the message construction stream, convert to a `NetMQMessage` with [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) and then [`Multicast`](xref:Bonsai.Expressions.MulticastSubject) into `RouterMessages`. If you run the workflow now you should see that each time a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) produces a message, all clients receive a copy of that message. :::workflow ![Broadcast to all clients](~/workflows/broadcast-all-clients.bonsai) @@ -159,10 +164,10 @@ Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of ## Leave-one-out broadcast This is getting pretty close to our original network architecture goal but there is still some redundancy present. When client 1 sends a message to the server, clients 1, 2 and 3 all receive a copy of that message back from the server. this is fine for clients 2 and 3 as they are not aware of client 1's messages without server communication; but client 1 does not need this message copy since it already originated the message. Our goal then is that the server should send message copies back to all clients except the client that originated message. -To do this, we'll create a [**`Condition`**](xref:Bonsai.Reactive.Condition) before `BroadcastAll` to filter only non sender clients called `NonSenderClients`. Inside `NonSenderClients` expose the `byte` corresponding to the client ID and index 1 of the first frame of the `NetMQMessage` from `Source1`. [**`Zip`**](xref:Bonsai.Reactive.Zip) these together and use [**`NotEqual`**](xref:Bonsai.Expressions.NotEqualBuilder) to compare the client ID of the message with the existing clients and discard where the IDs are the same. +To do this, we'll create a [`Condition`](xref:Bonsai.Reactive.Condition) before `BroadcastAll` to filter only non sender clients called `NonSenderClients`. Inside `NonSenderClients` expose the `byte` corresponding to the client ID and index 1 of the first frame of the `NetMQMessage` from `Source1`. [`Zip`](xref:Bonsai.Reactive.Zip) these together and use [`NotEqual`](xref:Bonsai.Expressions.NotEqualBuilder) to compare the client ID of the message with the existing clients and discard where the IDs are the same. :::workflow ![Broadcast to non-senders](~/workflows/broadcast-non-sender-clients.bonsai) ::: -Running the workflow you should see that we have now achieved the desired architecture. When a client [**`Dealer`**](xref:Bonsai.ZeroMQ.Dealer) sends a message, it is broadcast to all other joined clients except for itself. +Running the workflow you should see that we have now achieved the desired architecture. When a client [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) sends a message, it is broadcast to all other joined clients except for itself. From 097ce49c481a790f255a5591f7ab898257bb3708 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Jan 2025 11:29:08 +0000 Subject: [PATCH 18/37] Reformat workflow action points as bullet points --- docs/articles/client-server.md | 73 ++++++++++++++++++++++++---------- 1 file changed, 53 insertions(+), 20 deletions(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 3c51e99..42bc486 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -43,27 +43,37 @@ To begin with, we’ll create a simple client that sends basic messages on a net > In Bonsai.ZeroMQ, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) can have two functions based on its inputs. On its own, as above, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. If we add inputs to the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer), it will act as both a sender and receiver of messages on the network. -- Before the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator add a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) +- Before the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator add a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown). - Add a [`String`](xref:Bonsai.Expressions.StringProperty) operator in sequence as input to the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). :::workflow ![Basic Dealer input](~/workflows/dealer-basic-input.bonsai) ::: -In the node properties, set the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) `Filter` to the ‘1’ key and set the [`String`](xref:Bonsai.Expressions.StringProperty) `Value` to ‘Client1’. If we run the Bonsai project now, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) will continue listening for incoming messages on the network, but every time the ‘1’ key is pressed a message containing the string ‘Client1’ will be sent from the socket. +- In the node properties, set the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) `Filter` to the ‘1’ key. +- Set the [`String`](xref:Bonsai.Expressions.StringProperty) `Value` to ‘Client1’. -Copy and paste this client structure a couple of times and change the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) and [`String`](xref:Bonsai.Expressions.StringProperty) properties accordingly on each (2, ‘Client2’; 3, ‘Client3’) so that we have 3 total clients that send messages according to different key presses. +If we run the Bonsai project now, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) will continue listening for incoming messages on the network, but every time the ‘1’ key is pressed a message containing the string ‘Client1’ will be sent from the socket. + +- Copy and paste this client structure a couple of times and change the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) and [`String`](xref:Bonsai.Expressions.StringProperty) properties accordingly on each (2, ‘Client2’; 3, ‘Client3’) so that we have 3 total clients that send messages according to different key presses. :::workflow ![Multiple clients](~/workflows/multiple-clients.bonsai) ::: +> [!Note] > For the purposes of this article we are creating all of our clients and our server on the same Bonsai project and same machine for ease of demonstration. In a working example, each client and server could be running in separate Bonsai instances on different machines on a network. In this case, localhost would be replaced with the server machine’s IP address. ## Basic server -Now that we have our client pool set up and sending messages, let’s implement a server to listen for those messages. Add a [`Router`](xref:Bonsai.ZeroMQ.Router) node to the project and set its properties to match the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) sockets we already added so that it is running on the same network. As the [`Router`](xref:Bonsai.ZeroMQ.Router) is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. +Now that we have our client pool set up and sending messages, let’s implement a server to listen for those messages. +- Add a [`Router`](xref:Bonsai.ZeroMQ.Router) node to the project and set its properties to match the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) sockets we already added so that it is running on the same network. +- As the [`Router`](xref:Bonsai.ZeroMQ.Router) is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. + +As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node, a [`Router`](xref:Bonsai.ZeroMQ.Router) node without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) node, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [`Router`](xref:Bonsai.ZeroMQ.Router). Expanding the output the the [`Router`](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. -As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node, a [`Router`](xref:Bonsai.ZeroMQ.Router) node without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) node, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [`Router`](xref:Bonsai.ZeroMQ.Router). Expanding the output the the [`Router`](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. To make sense of the message, let's expose the `Buffer` `byte[]` of the `First` frame. Add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) node the the first frame buffer and set its `Value` to 1 to access the unique address ID. Add a [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. +- Expose the `Buffer` `byte[]` of the `First` frame. +- Add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) node the the first frame buffer and set its `Value` to 1 to access the unique address ID. +- Add a [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. :::workflow ![Router message parsing](~/workflows/router-message-parsing.bonsai) @@ -72,19 +82,25 @@ As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node, a [`Router`](xref:Bonsai Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the [`Index`](xref:Bonsai.Expressions.IndexBuilder) node output and a corresponding `string` in the [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) node output. ## Client address tracking -So far our network is rather one-sided. We can send client messages to the server which can in turn receive and parse them, but currently nothing is relayed back to the clients. The first goal for server feedbasck is that any time a client message is received, the server sends this message back to all connected clients. To do this, we first need a way of keeping track of all active clients. Add a [`Zip`](xref:Bonsai.Reactive.Zip) node to the [`Index`](xref:Bonsai.Expressions.IndexBuilder) node and connect the `byte[]` `Buffer` as the second input. +So far our network is rather one-sided. We can send client messages to the server which can in turn receive and parse them, but currently nothing is relayed back to the clients. The first goal for server feedback is that any time a client message is received, the server sends this message back to all connected clients. To do this, we first need a way of keeping track of all active clients. + +- Add a [`Zip`](xref:Bonsai.Reactive.Zip) node to the [`Index`](xref:Bonsai.Expressions.IndexBuilder) node and connect the `byte[]` `Buffer` as the second input. :::workflow ![Address key-value pair](~/workflows/address-kvp.bonsai) ::: -Every time the [`Router`](xref:Bonsai.ZeroMQ.Router) receives a message, the [`Zip`](xref:Bonsai.Reactive.Zip) will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. Next, add a [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node after the [`Zip`](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). +Every time the [`Router`](xref:Bonsai.ZeroMQ.Router) receives a message, the [`Zip`](xref:Bonsai.Reactive.Zip) will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. + +- Add a [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node after the [`Zip`](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). :::workflow ![Unique key-value pair](~/workflows/unique-kvp.bonsai) ::: -The [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node filters the output of [`Zip`](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [`Zip`](xref:Bonsai.Reactive.Zip). The output of [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. Add a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) node after [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) and name it ‘ClientAddresses’. +The [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node filters the output of [`Zip`](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [`Zip`](xref:Bonsai.Reactive.Zip). The output of [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. + +- Add a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) node after [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) and name it ‘ClientAddresses’. :::workflow ![Address ReplaySubject](~/workflows/address-replay-subject.bonsai) @@ -93,7 +109,10 @@ The [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node filters the output of [ A [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) has the useful feature that it stores its input sequence and replays those values to any current or future subscribers. The effect in this case is that anything that subscribes to `ClientAddresses` will receive all the unique client addresses encountered by the server so far. ## Server --> client communication -Eventually, we will use these unique client addresses to route server messages back to specific client. For now, we'll implement a more basic approach where the server just sends messages back to the client that originally sent them. The Bonsai ZeroMQ library provides a convenient node for this task in the form of [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse). Add this after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch, and inside (double-click on [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse)) add a [`String`](xref:Bonsai.Expressions.StringProperty) node with a generic response value like `ServerResponse` after the `Source` node. +Eventually, we will use these unique client addresses to route server messages back to specific client. For now, we'll implement a more basic approach where the server just sends messages back to the client that originally sent them. The Bonsai ZeroMQ library provides a convenient operator for this task in the form of [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse). + +- Add a [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch. +- Inside (double-click on [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse)) add a [`String`](xref:Bonsai.Expressions.StringProperty) node with a generic response value like `ServerResponse` after the `Source` operator. :::workflow ![Basic server response](~/workflows/server-basic-response.bonsai) @@ -101,9 +120,13 @@ Eventually, we will use these unique client addresses to route server messages b The [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node has a couple of interesting properties which may not be immediately obvious from this simple example. First, this node always transmits its response back to the ZeroMQ socket that initiated the request (in this case one of our [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) clients) and we therefore do not need to specify an address in its processing logic. Second, the internal flow of [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) logic is computed asynchronously. This is very useful for responses that require more intensive computation and allows a [`Router`](xref:Bonsai.ZeroMQ.Router) to deal with frequent incoming [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) requests efficiently. +> [!Note] > Imagine, for example, that our Dealer sockets were sending video snippets to a Router server that is tasked with doing some processing of the video and returning the results back to the Dealers. If the responses were not computed in an asynchronous manner we would start to incur a bottleneck on the router if there were many connected Dealers or frequent Dealer requests. -Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [`Router`](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. Delete the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) and [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) branches and replace with a branch that generates a bounceback message without using the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node: +Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [`Router`](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. + +- Delete the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) and [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) branches. +- Replace with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BouceBack` that generates a bounceback message without using the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node: :::workflow ![Server message multicast](~/workflows/server-message-multicast.bonsai) @@ -111,14 +134,14 @@ Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:B We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node in this implementation we need to pass messages directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). To do this we generate a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [`Router`](xref:Bonsai.ZeroMQ.Response) (can implement this by creating a [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) node, right-clicking it and creating a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source). This will change the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) node from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. - We want the [`Router`](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [`Router`](xref:Bonsai.ZeroMQ.Router) with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [`Index`](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new `String` which we convert to a `NetMQFrame` for the message content. We [`Merge`](xref:Bonsai.Reactive.Merge) these component frames back together and use a [`Take`](xref:Bonsai.Reactive.Take) node (with count = 3) followed by [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage). The [`Take`](xref:Bonsai.Reactive.Take) node is particularly important here as 1) [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Finally, we use a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [`Router`](xref:Bonsai.ZeroMQ.Router). +We want the [`Router`](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [`Router`](xref:Bonsai.ZeroMQ.Router) with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [`Index`](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new `String` which we convert to a `NetMQFrame` for the message content. We [`Merge`](xref:Bonsai.Reactive.Merge) these component frames back together and use a [`Take`](xref:Bonsai.Reactive.Take) node (with count = 3) followed by [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage). The [`Take`](xref:Bonsai.Reactive.Take) node is particularly important here as 1) [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Finally, we use a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [`Router`](xref:Bonsai.ZeroMQ.Router). - If we run the workflow now, we should see the same behavior as before (server bounces message back to initiating client). +If we run the workflow now, we should see the same behavior as before (server bounces message back to initiating client). - ## SelectMany detour - Now our network has a complete loop of client --> server --> client communication, but only the client that sends a message receives anything back from the server. Instead we’d like all clients to know when any of the clients sends a message. We already have access to the connected clients from `ClientAddresses`, and we know how to package data and send it back to clients via the [`Router`](xref:Bonsai.ZeroMQ.Router). In an imperative language we would do something like: +## SelectMany detour +Now our network has a complete loop of client --> server --> client communication, but only the client that sends a message receives anything back from the server. Instead we’d like all clients to know when any of the clients sends a message. We already have access to the connected clients from `ClientAddresses`, and we know how to package data and send it back to clients via the [`Router`](xref:Bonsai.ZeroMQ.Router). In an imperative language we would do something like: - ``` +``` foreach (var client in Clients) { Router.SendMessage(client.Address, Message); } @@ -139,21 +162,29 @@ Run the project and inspect the output of the [`SelectMany`](xref:Bonsai.Reactiv ## All client broadcast To apply the logic of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) example to server broadcast, we need something to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence creation, and something to trigger termination. We already have a trigger for sequence creation in the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) since we want to run our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence every time a client message is received. For our sequence temination trigger, we want something that is guaranteed to fire after the server receives a client message, but before the next message is received so that our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence for each message responds only to that particular message. A simple solution is therefore to use the arrival of the next message as our sequence termination trigger. -To implement this, add a [`Skip`](xref:Bonsai.Reactive.Skip) node after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch and connect this to a [`PublishSubject`](xref:Bonsai.Reactive.PublishSubject). Ensure that the [`Skip`](xref:Bonsai.Reactive.Skip) node's `Count` property is set to 1, and name the [`PublishSubject`](xref:Bonsai.Reactive.PublishSubject) 'NextMessage'. +- Add a [`Skip`](xref:Bonsai.Reactive.Skip) operator after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch and connect this to a [`PublishSubject`](xref:Bonsai.Reactive.PublishSubject). +- Set the [`Skip`](xref:Bonsai.Reactive.Skip) operator's `Count` property is set to 1, and name the [`PublishSubject`](xref:Bonsai.Reactive.PublishSubject) 'NextMessage'. :::workflow ![Server next message](~/workflows/server-next-message.bonsai) ::: -The logic here is that we use [`Skip`](xref:Bonsai.Reactive.Skip) to create a sequence that lags exactly 1 message behind the [`Router`](xref:Bonsai.ZeroMQ.Router) sequence of received messages, i.e. when the first message is received, `NextMessage` will not produce a result until the second message is received. We can then use this inside our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) logic for generating server messages. Add a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch and name it ‘SelectAllClients’. +The logic here is that we use [`Skip`](xref:Bonsai.Reactive.Skip) to create a sequence that lags exactly 1 message behind the [`Router`](xref:Bonsai.ZeroMQ.Router) sequence of received messages, i.e. when the first message is received, `NextMessage` will not produce a result until the second message is received. We can then use this inside our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) logic for generating server messages. + +- Add a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch and name it ‘SelectAllClients’. +- Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node, create 2 [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) nodes and link them to the `ClientAddresses` and `NextMessage` subjects. +- Connect the `ClientAddresses` subscription to the workflow output via a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) node and use `NextMessage` as the second input. -Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node, create 2 [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) nodes and link them to the `ClientAddresses` and `NextMessage` subjects. Connect the `ClientAddresses` subscription to the workflow output via a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) node and use `NextMessage` as the second input. Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a [`WithLatestFrom`](xref:Bonsai.Reactive.WithLatestFrom) with the [`Router`](xref:Bonsai.ZeroMQ.Router) as its second input. In this context [`WithLatestFrom`](xref:Bonsai.Reactive.WithLatestFrom) combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. +Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a [`WithLatestFrom`](xref:Bonsai.Reactive.WithLatestFrom) with the [`Router`](xref:Bonsai.ZeroMQ.Router) as its second input. In this context [`WithLatestFrom`](xref:Bonsai.Reactive.WithLatestFrom) combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. :::workflow ![Select all clients and package message](~/workflows/format-select-all-clients.bonsai) ::: -To send these messages back to our clients, we will modify the logic in our previous `BounceBack` node. This time, we'll create a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. This is multicast back into the router to send the original address back to all clients. +To send these messages back to our clients, we will modify the logic in our previous `BounceBack` node. + +- Create a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. +- [`Multicast`](xref:Bonsai.Expressions.MulticastSubject) back into the router to send the original address back to all clients. Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of client ID / address and the received message. We take the `byte[]` address and use [`ConvertToFrame`](xref:Bonsai.ZeroMQ.ConvertToFrame) to convert it to a `NetMQFrame` and then merge it with the empty delimiter and the message payload. As before, we [`Take`](xref:Bonsai.Reactive.Take) 3 elements to close the message construction stream, convert to a `NetMQMessage` with [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) and then [`Multicast`](xref:Bonsai.Expressions.MulticastSubject) into `RouterMessages`. If you run the workflow now you should see that each time a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) produces a message, all clients receive a copy of that message. @@ -164,7 +195,9 @@ Inside `BroadcastAll` the source consists of a `Tuple` of our key-value-pair of ## Leave-one-out broadcast This is getting pretty close to our original network architecture goal but there is still some redundancy present. When client 1 sends a message to the server, clients 1, 2 and 3 all receive a copy of that message back from the server. this is fine for clients 2 and 3 as they are not aware of client 1's messages without server communication; but client 1 does not need this message copy since it already originated the message. Our goal then is that the server should send message copies back to all clients except the client that originated message. -To do this, we'll create a [`Condition`](xref:Bonsai.Reactive.Condition) before `BroadcastAll` to filter only non sender clients called `NonSenderClients`. Inside `NonSenderClients` expose the `byte` corresponding to the client ID and index 1 of the first frame of the `NetMQMessage` from `Source1`. [`Zip`](xref:Bonsai.Reactive.Zip) these together and use [`NotEqual`](xref:Bonsai.Expressions.NotEqualBuilder) to compare the client ID of the message with the existing clients and discard where the IDs are the same. +- Create a [`Condition`](xref:Bonsai.Reactive.Condition) before `BroadcastAll` to filter only non sender clients called `NonSenderClients`. +- Inside `NonSenderClients` expose the `byte` corresponding to the client ID and index 1 of the first frame of the `NetMQMessage` from `Source1`. +- [`Zip`](xref:Bonsai.Reactive.Zip) these together and use [`NotEqual`](xref:Bonsai.Expressions.NotEqualBuilder) to compare the client ID of the message with the existing clients and discard where the IDs are the same. :::workflow ![Broadcast to non-senders](~/workflows/broadcast-non-sender-clients.bonsai) From 07f2ad2d03af1e687eea99bdb1c6e035506c5315 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Jan 2025 11:32:41 +0000 Subject: [PATCH 19/37] Replace references to 'node' with 'operator' --- docs/articles/client-server.md | 44 +++++++++++++++++----------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index 42bc486..b82e659 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -36,11 +36,11 @@ ZeroMQ provides a number of socket types that could be used to achieve something ## Basic client To begin with, we’ll create a simple client that sends basic messages on a network. In a new Bonsai project: -- Add a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node. +- Add a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator. - In the `ConnectionString` property, set `Address`: localhost:5557, `Action`: Connect, `Protocol`: TCP. > [!Note] -> In Bonsai.ZeroMQ, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) can have two functions based on its inputs. On its own, as above, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. +> In Bonsai.ZeroMQ, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) can have two functions based on its inputs. On its own, as above, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator creates a Dealer socket that listens for messages on the specified network. With the properties specified, we are asking our [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) to listen for messages on the local machine on port 5557 using the TCP protocol. We use the ‘Connect’ argument for the `SocketConnection` property to tell the dealer that it will connect to a static part of the network with a known IP address, in this case the server which we will implement later. If we add inputs to the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer), it will act as both a sender and receiver of messages on the network. - Before the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator add a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown). @@ -50,7 +50,7 @@ If we add inputs to the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer), it will act as bo ![Basic Dealer input](~/workflows/dealer-basic-input.bonsai) ::: -- In the node properties, set the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) `Filter` to the ‘1’ key. +- In the operator properties, set the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) `Filter` to the ‘1’ key. - Set the [`String`](xref:Bonsai.Expressions.StringProperty) `Value` to ‘Client1’. If we run the Bonsai project now, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) will continue listening for incoming messages on the network, but every time the ‘1’ key is pressed a message containing the string ‘Client1’ will be sent from the socket. @@ -66,25 +66,25 @@ If we run the Bonsai project now, the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) will ## Basic server Now that we have our client pool set up and sending messages, let’s implement a server to listen for those messages. -- Add a [`Router`](xref:Bonsai.ZeroMQ.Router) node to the project and set its properties to match the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) sockets we already added so that it is running on the same network. +- Add a [`Router`](xref:Bonsai.ZeroMQ.Router) operator to the project and set its properties to match the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) sockets we already added so that it is running on the same network. - As the [`Router`](xref:Bonsai.ZeroMQ.Router) is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. -As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) node, a [`Router`](xref:Bonsai.ZeroMQ.Router) node without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) node, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [`Router`](xref:Bonsai.ZeroMQ.Router). Expanding the output the the [`Router`](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. +As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator, a [`Router`](xref:Bonsai.ZeroMQ.Router) operator without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [`Router`](xref:Bonsai.ZeroMQ.Router). Expanding the output the the [`Router`](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. - Expose the `Buffer` `byte[]` of the `First` frame. -- Add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) node the the first frame buffer and set its `Value` to 1 to access the unique address ID. +- Add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator the the first frame buffer and set its `Value` to 1 to access the unique address ID. - Add a [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. :::workflow ![Router message parsing](~/workflows/router-message-parsing.bonsai) ::: -Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the [`Index`](xref:Bonsai.Expressions.IndexBuilder) node output and a corresponding `string` in the [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) node output. +Running the workflow and then triggering client messages with key presses, we should see a unique `byte` value for each client in the [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator output and a corresponding `string` in the [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) operator output. ## Client address tracking So far our network is rather one-sided. We can send client messages to the server which can in turn receive and parse them, but currently nothing is relayed back to the clients. The first goal for server feedback is that any time a client message is received, the server sends this message back to all connected clients. To do this, we first need a way of keeping track of all active clients. -- Add a [`Zip`](xref:Bonsai.Reactive.Zip) node to the [`Index`](xref:Bonsai.Expressions.IndexBuilder) node and connect the `byte[]` `Buffer` as the second input. +- Add a [`Zip`](xref:Bonsai.Reactive.Zip) operator to the [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator and connect the `byte[]` `Buffer` as the second input. :::workflow ![Address key-value pair](~/workflows/address-kvp.bonsai) @@ -92,15 +92,15 @@ So far our network is rather one-sided. We can send client messages to the serve Every time the [`Router`](xref:Bonsai.ZeroMQ.Router) receives a message, the [`Zip`](xref:Bonsai.Reactive.Zip) will create a `Tuple` that can be thought of as a key-value pair, with the unique `byte` address of the client as the key, and the full `byte[]` address used by ZeroMQ for routing as the value. -- Add a [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node after the [`Zip`](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). +- Add a [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) operator after the [`Zip`](xref:Bonsai.Reactive.Zip) and set the `KeySelector` property to the `byte` value (`Item1`). :::workflow ![Unique key-value pair](~/workflows/unique-kvp.bonsai) ::: -The [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) node filters the output of [`Zip`](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [`Zip`](xref:Bonsai.Reactive.Zip). The output of [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. +The [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) operator filters the output of [`Zip`](xref:Bonsai.Reactive.Zip) according to the unique `byte` value and produces a sequence containing only the distinct – or ‘new’ – values produced by [`Zip`](xref:Bonsai.Reactive.Zip). The output of [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) will therefore effectively be a sequence of unique client addresses corresponding to each connected client. We also need to store these unique values and make them available to other parts of the Bonsai workflow. -- Add a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) node after [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) and name it ‘ClientAddresses’. +- Add a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) operator after [`DistinctBy`](xref:Bonsai.Reactive.DistinctBy) and name it ‘ClientAddresses’. :::workflow ![Address ReplaySubject](~/workflows/address-replay-subject.bonsai) @@ -112,13 +112,13 @@ A [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) has the useful feature t Eventually, we will use these unique client addresses to route server messages back to specific client. For now, we'll implement a more basic approach where the server just sends messages back to the client that originally sent them. The Bonsai ZeroMQ library provides a convenient operator for this task in the form of [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse). - Add a [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch. -- Inside (double-click on [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse)) add a [`String`](xref:Bonsai.Expressions.StringProperty) node with a generic response value like `ServerResponse` after the `Source` operator. +- Inside (double-click on [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse)) add a [`String`](xref:Bonsai.Expressions.StringProperty) operator with a generic response value like `ServerResponse` after the `Source` operator. :::workflow ![Basic server response](~/workflows/server-basic-response.bonsai) ::: -The [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node has a couple of interesting properties which may not be immediately obvious from this simple example. First, this node always transmits its response back to the ZeroMQ socket that initiated the request (in this case one of our [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) clients) and we therefore do not need to specify an address in its processing logic. Second, the internal flow of [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) logic is computed asynchronously. This is very useful for responses that require more intensive computation and allows a [`Router`](xref:Bonsai.ZeroMQ.Router) to deal with frequent incoming [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) requests efficiently. +The [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator has a couple of interesting properties which may not be immediately obvious from this simple example. First, this operator always transmits its response back to the ZeroMQ socket that initiated the request (in this case one of our [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) clients) and we therefore do not need to specify an address in its processing logic. Second, the internal flow of [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) logic is computed asynchronously. This is very useful for responses that require more intensive computation and allows a [`Router`](xref:Bonsai.ZeroMQ.Router) to deal with frequent incoming [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) requests efficiently. > [!Note] > Imagine, for example, that our Dealer sockets were sending video snippets to a Router server that is tasked with doing some processing of the video and returning the results back to the Dealers. If the responses were not computed in an asynchronous manner we would start to incur a bottleneck on the router if there were many connected Dealers or frequent Dealer requests. @@ -126,15 +126,15 @@ The [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node has a couple of inter Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [`Router`](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. - Delete the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) and [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) branches. -- Replace with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BouceBack` that generates a bounceback message without using the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node: +- Replace with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BouceBack` that generates a bounceback message without using the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator: :::workflow ![Server message multicast](~/workflows/server-message-multicast.bonsai) ::: -We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) node in this implementation we need to pass messages directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). To do this we generate a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [`Router`](xref:Bonsai.ZeroMQ.Response) (can implement this by creating a [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) node, right-clicking it and creating a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source). This will change the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) node from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. +We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator in this implementation we need to pass messages directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). To do this we generate a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [`Router`](xref:Bonsai.ZeroMQ.Response) (can implement this by creating a [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) operator, right-clicking it and creating a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source). This will change the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. -We want the [`Router`](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [`Router`](xref:Bonsai.ZeroMQ.Router) with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [`Index`](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new `String` which we convert to a `NetMQFrame` for the message content. We [`Merge`](xref:Bonsai.Reactive.Merge) these component frames back together and use a [`Take`](xref:Bonsai.Reactive.Take) node (with count = 3) followed by [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage). The [`Take`](xref:Bonsai.Reactive.Take) node is particularly important here as 1) [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Finally, we use a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [`Router`](xref:Bonsai.ZeroMQ.Router). +We want the [`Router`](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [`Router`](xref:Bonsai.ZeroMQ.Router) with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [`Index`](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new `String` which we convert to a `NetMQFrame` for the message content. We [`Merge`](xref:Bonsai.Reactive.Merge) these component frames back together and use a [`Take`](xref:Bonsai.Reactive.Take) operator (with count = 3) followed by [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage). The [`Take`](xref:Bonsai.Reactive.Take) operator is particularly important here as 1) [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Finally, we use a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [`Router`](xref:Bonsai.ZeroMQ.Router). If we run the workflow now, we should see the same behavior as before (server bounces message back to initiating client). @@ -151,13 +151,13 @@ using a loop to send the message back to each client in turn. In a reactive / ob The [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator can be a tricky one to understand. Lee Campbell’s excellent [Introduction to Rx](http://introtorx.com/Content/v1.0.10621.0/08_Transformation.html#SelectMany) book does a good job of summarising its utility, suggesting we think of it as “from one, select many” or “from one, select zero or more”. In our case, we can think of [`SelectMany`](xref:Bonsai.Reactive.SelectMany) as a way to repeat some processing logic several times and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. -Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) node followed by a [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node add a [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) node after the [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) to the [`WorkflowOutput`](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` node). Finally, create a [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) node and connect it to [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) node outside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). +Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator followed by a [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) oeprator add a [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) operator after the [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) to the [`WorkflowOutput`](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` operator). Finally, create a [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) operator and connect it to [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator outside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). :::workflow ![SelectMany detour](~/workflows/select-many-detour.bonsai) ::: -Run the project and inspect the output of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node. If no client messages are triggered and we press ‘A’ to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) nothing will be returned. If we trigger a single client and press ‘A’ again [`SelectMany`](xref:Bonsai.Reactive.SelectMany) gives us the address of that client. If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger [`SelectMany`](xref:Bonsai.Reactive.SelectMany) with a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) we generate a new sequence that immediately subscribes to `ClientAddresses`, a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) which replays all our unique client addresses into the sequence. We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the [`SusbcribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) directly to the workflow output and deleting [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) and [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil)). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. +Run the project and inspect the output of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator. If no client messages are triggered and we press ‘A’ to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) nothing will be returned. If we trigger a single client and press ‘A’ again [`SelectMany`](xref:Bonsai.Reactive.SelectMany) gives us the address of that client. If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger [`SelectMany`](xref:Bonsai.Reactive.SelectMany) with a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) we generate a new sequence that immediately subscribes to `ClientAddresses`, a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) which replays all our unique client addresses into the sequence. We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the [`SusbcribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) directly to the workflow output and deleting [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) and [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil)). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. ## All client broadcast To apply the logic of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) example to server broadcast, we need something to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence creation, and something to trigger termination. We already have a trigger for sequence creation in the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) since we want to run our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence every time a client message is received. For our sequence temination trigger, we want something that is guaranteed to fire after the server receives a client message, but before the next message is received so that our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence for each message responds only to that particular message. A simple solution is therefore to use the arrival of the next message as our sequence termination trigger. @@ -171,9 +171,9 @@ To apply the logic of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) exampl The logic here is that we use [`Skip`](xref:Bonsai.Reactive.Skip) to create a sequence that lags exactly 1 message behind the [`Router`](xref:Bonsai.ZeroMQ.Router) sequence of received messages, i.e. when the first message is received, `NextMessage` will not produce a result until the second message is received. We can then use this inside our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) logic for generating server messages. -- Add a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch and name it ‘SelectAllClients’. -- Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) node, create 2 [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) nodes and link them to the `ClientAddresses` and `NextMessage` subjects. -- Connect the `ClientAddresses` subscription to the workflow output via a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) node and use `NextMessage` as the second input. +- Add a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator after the [`Router`](xref:Bonsai.ZeroMQ.Router) in a separate branch and name it ‘SelectAllClients’. +- Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator, create 2 [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) operators and link them to the `ClientAddresses` and `NextMessage` subjects. +- Connect the `ClientAddresses` subscription to the workflow output via a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) operator and use `NextMessage` as the second input. Now, our `SelectAllClients` will produce a sequence of all unique client addresses every time the server receives a message. Connect the output of `SelectAllClients` to a [`WithLatestFrom`](xref:Bonsai.Reactive.WithLatestFrom) with the [`Router`](xref:Bonsai.ZeroMQ.Router) as its second input. In this context [`WithLatestFrom`](xref:Bonsai.Reactive.WithLatestFrom) combines each client address from `SelectAllClients` with the most recent received message. The result is that when a message is received from the client, we produce several copies of the message 'addressed' to each connected client. @@ -181,7 +181,7 @@ Now, our `SelectAllClients` will produce a sequence of all unique client address ![Select all clients and package message](~/workflows/format-select-all-clients.bonsai) ::: -To send these messages back to our clients, we will modify the logic in our previous `BounceBack` node. +To send these messages back to our clients, we will modify the logic in our previous `BounceBack` operator. - Create a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BroadcastAll` that takes the `byte[]` addresses from `SelectAllClients` and reformats the original message with this address as the first frame. - [`Multicast`](xref:Bonsai.Expressions.MulticastSubject) back into the router to send the original address back to all clients. From 75f5a15fae2543591eeb5f0da4a40ce12c4b5616 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 16 Jan 2025 11:37:59 +0000 Subject: [PATCH 20/37] Refer to router-dealer pattern with explicit link --- docs/articles/client-server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index b82e659..cde8218 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -30,7 +30,7 @@ sequenceDiagram An important requirement to point out here is that our server should be choosy about which clients it broadcasts information to. If client 1 updates the server with its current state, that information needs to be sent to all other connected clients, but there is no need to send it back to client 1 as it already knows its own state and this feedback message would be redundant. -ZeroMQ provides a number of socket types that could be used to achieve something approaching this architecture. The Router / Dealer socket pair acting as Server / Client has a couple of advantages for this design: +ZeroMQ provides a number of socket types that could be used to achieve something approaching this architecture. The @router-dealer socket pair acting as Server / Client has a couple of advantages for this design: - Routers assign a unique address for each connected client allowing clients in turn to be addressed individually - Messages can be passed between Router / Dealer sockets without the requirement that a reply is received before the next message is sent, as is the case with the Request / Response socket pair. From 485e8c7c377d8ad43801cce6d49586488b72b719 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Fri, 17 Jan 2025 10:09:49 +0000 Subject: [PATCH 21/37] Break up final section of SelectMany detour --- docs/articles/client-server.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/articles/client-server.md b/docs/articles/client-server.md index cde8218..551b6f4 100644 --- a/docs/articles/client-server.md +++ b/docs/articles/client-server.md @@ -157,7 +157,14 @@ Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator followed by a [ ![SelectMany detour](~/workflows/select-many-detour.bonsai) ::: -Run the project and inspect the output of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator. If no client messages are triggered and we press ‘A’ to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) nothing will be returned. If we trigger a single client and press ‘A’ again [`SelectMany`](xref:Bonsai.Reactive.SelectMany) gives us the address of that client. If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger [`SelectMany`](xref:Bonsai.Reactive.SelectMany) with a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) we generate a new sequence that immediately subscribes to `ClientAddresses`, a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) which replays all our unique client addresses into the sequence. We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the [`SusbcribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) directly to the workflow output and deleting [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) and [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil)). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. +- Run the project and inspect the output of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator. +- If no client messages are triggered and we press ‘A’ to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) nothing will be returned. +- If we trigger a single client and press ‘A’ again [`SelectMany`](xref:Bonsai.Reactive.SelectMany) gives us the address of that client. +- If we trigger a second client and press ‘A’ we get the addresses of these first two clients in sequence, and so on if we add the third client. + +Whenever we press ‘A’ we get a sequence of all the connected client addresses. Every time we trigger [`SelectMany`](xref:Bonsai.Reactive.SelectMany) with a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) we generate a new sequence that immediately subscribes to `ClientAddresses`, a [`ReplaySubject`](xref:Bonsai.Reactive.ReplaySubject) which replays all our unique client addresses into the sequence. + +We could keep initiating these new sequences by continually pressing ‘A’ and if a new client address were to be added then all these sequences would report the new address (you can test this by connecting the [`SusbcribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) directly to the workflow output and deleting [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) and [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil)). Instead, we want to complete each new sequence once it has given us all the client addresses so we use an arbitrary event (releasing the key that initiated the sequence) to trigger [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) and close the sequence. The overall effect is something similar to a loop that iterates over all client addresses every time we request it with a key press. This is the general structure of what we want to achieve next in our server logic to broadcast messages back to all connected clients. ## All client broadcast To apply the logic of the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) example to server broadcast, we need something to trigger the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence creation, and something to trigger termination. We already have a trigger for sequence creation in the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) since we want to run our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence every time a client message is received. For our sequence temination trigger, we want something that is guaranteed to fire after the server receives a client message, but before the next message is received so that our [`SelectMany`](xref:Bonsai.Reactive.SelectMany) sequence for each message responds only to that particular message. A simple solution is therefore to use the arrival of the next message as our sequence termination trigger. From ebc25282517a2c871ddabf9f355261a6a8993c1e Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Fri, 17 Jan 2025 10:20:29 +0000 Subject: [PATCH 22/37] Move client-server article to new tutorial section --- docs/articles/toc.yml | 4 +--- docs/docfx.json | 8 ++++++++ docs/toc.yml | 2 ++ docs/{articles => tutorials}/client-server.md | 0 docs/tutorials/toc.yml | 2 ++ 5 files changed, 13 insertions(+), 3 deletions(-) rename docs/{articles => tutorials}/client-server.md (100%) create mode 100644 docs/tutorials/toc.yml diff --git a/docs/articles/toc.yml b/docs/articles/toc.yml index 54bbdec..a6625ae 100644 --- a/docs/articles/toc.yml +++ b/docs/articles/toc.yml @@ -7,6 +7,4 @@ - href: push-pull.md - href: proxy.md - name: Recipes -- href: recipes.md -- name: Tutorials -- href: client-server.md \ No newline at end of file +- href: recipes.md \ No newline at end of file diff --git a/docs/docfx.json b/docs/docfx.json index 9681da5..adcc558 100644 --- a/docs/docfx.json +++ b/docs/docfx.json @@ -31,6 +31,14 @@ "*.md" ] }, + { + "files": [ + "tutorials/**.md", + "tutorials/**/toc.yml", + "toc.yml", + "*.md" + ] + }, { "exclude": [ "_site/**", diff --git a/docs/toc.yml b/docs/toc.yml index e61b382..8de88a1 100644 --- a/docs/toc.yml +++ b/docs/toc.yml @@ -2,3 +2,5 @@ href: articles/ - name: Reference href: api/ +- name: Tutorials + href: tutorials/ diff --git a/docs/articles/client-server.md b/docs/tutorials/client-server.md similarity index 100% rename from docs/articles/client-server.md rename to docs/tutorials/client-server.md diff --git a/docs/tutorials/toc.yml b/docs/tutorials/toc.yml new file mode 100644 index 0000000..366a154 --- /dev/null +++ b/docs/tutorials/toc.yml @@ -0,0 +1,2 @@ +- name: Tutorials +- href: client-server.md \ No newline at end of file From 00cf0ffc9a9089fe3234f8a396f91bd3eff01d3c Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Fri, 17 Jan 2025 10:26:06 +0000 Subject: [PATCH 23/37] Clean up content definition for tutorials in docfx.json --- docs/docfx.json | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/docfx.json b/docs/docfx.json index adcc558..8f3a2d9 100644 --- a/docs/docfx.json +++ b/docs/docfx.json @@ -34,9 +34,7 @@ { "files": [ "tutorials/**.md", - "tutorials/**/toc.yml", - "toc.yml", - "*.md" + "tutorials/toc.yml" ] }, { From 0af7e4a824cc152192ba63ffcd1a9603b04bd706 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 23 Jan 2025 11:36:09 +0000 Subject: [PATCH 24/37] Fix some typos --- docs/tutorials/client-server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index 551b6f4..eb72183 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -69,7 +69,7 @@ Now that we have our client pool set up and sending messages, let’s implement - Add a [`Router`](xref:Bonsai.ZeroMQ.Router) operator to the project and set its properties to match the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) sockets we already added so that it is running on the same network. - As the [`Router`](xref:Bonsai.ZeroMQ.Router) is acting as server and will be the ‘static’ part of the network, set its `Action` to ‘Bind’. -As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator, a [`Router`](xref:Bonsai.ZeroMQ.Router) operator without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [`Router`](xref:Bonsai.ZeroMQ.Router). Expanding the output the the [`Router`](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. +As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator, a [`Router`](xref:Bonsai.ZeroMQ.Router) operator without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [`Router`](xref:Bonsai.ZeroMQ.Router). Expanding the output the [`Router`](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. - Expose the `Buffer` `byte[]` of the `First` frame. - Add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator the the first frame buffer and set its `Value` to 1 to access the unique address ID. From 4ecd0f2e1417979b836414edf5532094b38c6ba0 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Fri, 24 Jan 2025 09:45:16 +0000 Subject: [PATCH 25/37] Step section for BounceBack SelectMany --- docs/tutorials/client-server.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index eb72183..6fe71de 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -72,7 +72,7 @@ Now that we have our client pool set up and sending messages, let’s implement As with the [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) operator, a [`Router`](xref:Bonsai.ZeroMQ.Router) operator without any input will simply listen for messages on the network and not send anything in return. If we run the project now and monitor the output of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator, we'll see that each time the client sends a message triggered by its associated key press we get a `ResponseContext` produced at the [`Router`](xref:Bonsai.ZeroMQ.Router). Expanding the output the [`Router`](xref:Bonsai.ZeroMQ.Router), we can see it contains a `NetMQMessage`. We [expect](https://netmq.readthedocs.io/en/latest/router-dealer/) this message to be composed of 3 frames: an address (in this case the address of the client that sent the message), an empty delimiter frame and the message content. - Expose the `Buffer` `byte[]` of the `First` frame. -- Add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator the the first frame buffer and set its `Value` to 1 to access the unique address ID. +- Add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator the first frame buffer and set its `Value` to 1 to access the unique address ID. - Add a [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) to the `Last` frame. :::workflow @@ -126,7 +126,15 @@ The [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator has a couple of i Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [`Router`](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. - Delete the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) and [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) branches. -- Replace with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BouceBack` that generates a bounceback message without using the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator: +- Create a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and name it 'RouterMessages' (right-click on an operator with a `NetMQMessage` output type >> CreateSource >> BehaviorSubject). Connect it as an input to the [`Router`](xref:Bonsai.ZeroMQ.Router). +- Delete the `Request.First` and `Buffer` outputs from [`Router`](xref:Bonsai.ZeroMQ.Router). Right-click on the [`Router`](xref:Bonsai.ZeroMQ.Router) and expose the `NetMQMessage` >> `First` >> `Buffer` output. +- Add a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BouceBack`. Connect [`Router`](xref:Bonsai.ZeroMQ.Router) to `BounceBack`. +- Inside `BounceBack`, expose the `First` property of the `Source1` output. +- On a separate branch, add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator with an index 'Value' of 1. Connect `Source1` as its input. +- On a further separate branch, add a [`String`](xref:Bonsai.Expressions.StringProperty) operator with a 'Value' of 'ServerResponse'. Connect this to a [`ConvertToFrame`](xref:Bonsai.ZeroMQ.ConvertToFrame) operator. Connect `Source1` as an input to the [`String`](xref:Bonsai.Expressions.StringProperty). +- Use [`Merge`](xref:Bonsai.Reactive.Merge) to combine the outputs of these 3 branches (`Source1.First`, `Index`, `ConvertToFrame`). +- Convert the output to a `NetMQMessage` by connecting a [`Take`](xref:Bonsai.Reactive.Take) operator with a 'Count' property of 3 followed by a [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) operator. +- Finally, connect the output of [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) to a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) targeting `RouterMessages`. Connect the [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) output to the `WorkflowOutput`. :::workflow ![Server message multicast](~/workflows/server-message-multicast.bonsai) From 1070ee04bb6bcd11a39d3247bef41aee4e28b142 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Fri, 24 Jan 2025 09:48:46 +0000 Subject: [PATCH 26/37] Remove redundant information about source subject creation --- docs/tutorials/client-server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index 6fe71de..4f00d2c 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -140,7 +140,7 @@ Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:B ![Server message multicast](~/workflows/server-message-multicast.bonsai) ::: -We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator in this implementation we need to pass messages directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). To do this we generate a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [`Router`](xref:Bonsai.ZeroMQ.Response) (can implement this by creating a [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) operator, right-clicking it and creating a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source). This will change the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. +We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator in this implementation we need to pass messages directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). To do this we generate a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [`Router`](xref:Bonsai.ZeroMQ.Response). This will change the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. We want the [`Router`](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [`Router`](xref:Bonsai.ZeroMQ.Router) with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [`Index`](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new `String` which we convert to a `NetMQFrame` for the message content. We [`Merge`](xref:Bonsai.Reactive.Merge) these component frames back together and use a [`Take`](xref:Bonsai.Reactive.Take) operator (with count = 3) followed by [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage). The [`Take`](xref:Bonsai.Reactive.Take) operator is particularly important here as 1) [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Finally, we use a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [`Router`](xref:Bonsai.ZeroMQ.Router). From f957e365031ab0a147c8778bc9eb25f062d58488 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Fri, 24 Jan 2025 10:07:12 +0000 Subject: [PATCH 27/37] Reformat SelectMany explanation --- docs/tutorials/client-server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index 4f00d2c..066c282 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -157,7 +157,7 @@ foreach (var client in Clients) { using a loop to send the message back to each client in turn. In a reactive / observable sequence based framework we have to think about this a bit differently. The solution is to use a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator and it is worth taking a detour here to understand its use in some detail since we have already used it to generate our bounceback message and will apply it again for addressing multiple clients. -The [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator can be a tricky one to understand. Lee Campbell’s excellent [Introduction to Rx](http://introtorx.com/Content/v1.0.10621.0/08_Transformation.html#SelectMany) book does a good job of summarising its utility, suggesting we think of it as “from one, select many” or “from one, select zero or more”. In our case, we can think of [`SelectMany`](xref:Bonsai.Reactive.SelectMany) as a way to repeat some processing logic several times and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. +The [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator can be a tricky one to understand. One way to describe the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operation is as *from one, create zero, one or many*. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator in Bonsai we define the creation of a new, parallel observable stream that governs this transformation. In our case, we can think of [`SelectMany`](xref:Bonsai.Reactive.SelectMany) as a way to repeat some processing logic several times for each input and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator followed by a [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) oeprator add a [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) operator after the [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) to the [`WorkflowOutput`](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` operator). Finally, create a [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) operator and connect it to [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator outside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). From f93dc4170d61e56cc277a91ad041a3854e8a1a29 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Fri, 24 Jan 2025 10:08:47 +0000 Subject: [PATCH 28/37] Fix typo --- docs/tutorials/client-server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index 066c282..ea26625 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -159,7 +159,7 @@ using a loop to send the message back to each client in turn. In a reactive / ob The [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator can be a tricky one to understand. One way to describe the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operation is as *from one, create zero, one or many*. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator in Bonsai we define the creation of a new, parallel observable stream that governs this transformation. In our case, we can think of [`SelectMany`](xref:Bonsai.Reactive.SelectMany) as a way to repeat some processing logic several times for each input and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. -Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator followed by a [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) oeprator add a [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) operator after the [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) to the [`WorkflowOutput`](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` operator). Finally, create a [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) operator and connect it to [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator outside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). +Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator followed by a [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator add a [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) operator after the [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) to the [`WorkflowOutput`](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` operator). Finally, create a [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) operator and connect it to [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator outside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). :::workflow ![SelectMany detour](~/workflows/select-many-detour.bonsai) From fa0a8ff11e33a347063b41f881baf5b57d5b4ac5 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Fri, 24 Jan 2025 10:14:09 +0000 Subject: [PATCH 29/37] Create extra step breakout for SelectMany detour --- docs/tutorials/client-server.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index ea26625..4871ee8 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -159,7 +159,11 @@ using a loop to send the message back to each client in turn. In a reactive / ob The [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator can be a tricky one to understand. One way to describe the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operation is as *from one, create zero, one or many*. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator in Bonsai we define the creation of a new, parallel observable stream that governs this transformation. In our case, we can think of [`SelectMany`](xref:Bonsai.Reactive.SelectMany) as a way to repeat some processing logic several times for each input and feed the output of each repeat into a single sequence. More concretely, taking a single message and repeating the act of sending it several times for each client address. It is easier to show by example, so let’s set up a toy example in our project. -Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator followed by a [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator add a [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. Add a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) operator after the [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) to the [`WorkflowOutput`](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` operator). Finally, create a [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) operator and connect it to [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil). Set the key `Filter` for [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator outside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). +- Create a [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator followed by a [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Set the `Filter` for the [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) to a key that hasn’t been assigned to a client yet – here I will use ‘A’. +- Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator add a [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and set its subscription to the `ClientAddresses` subject we created earlier to replay unique client addresses. +- Add a [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) operator after the [`SubscribeSubject`](xref:Bonsai.Expressions.SubscribeSubject) and connect the output of [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil) to the [`WorkflowOutput`](xref:Bonsai.Expressions.WorkflowOutputBuilder) (disconnecting the `Source` operator). +- Create a [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) operator and connect it to [`TakeUntil`](xref:Bonsai.Reactive.TakeUntil). +- Set the key `Filter` for [`KeyUp`](xref:Bonsai.Windows.Input.KeyUp) to the same as previously created [`KeyDown`](xref:Bonsai.Windows.Input.KeyDown) operator outside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). :::workflow ![SelectMany detour](~/workflows/select-many-detour.bonsai) From 3576b55059b8a5dd015813d7efe963b1c7d7816b Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Wed, 29 Jan 2025 09:31:05 +0000 Subject: [PATCH 30/37] Clarify purpose of replacing SendResponse operator Co-authored-by: Shawn Tan --- docs/tutorials/client-server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index 4871ee8..d14eea8 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -123,7 +123,7 @@ The [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator has a couple of i > [!Note] > Imagine, for example, that our Dealer sockets were sending video snippets to a Router server that is tasked with doing some processing of the video and returning the results back to the Dealers. If the responses were not computed in an asynchronous manner we would start to incur a bottleneck on the router if there were many connected Dealers or frequent Dealer requests. -Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [`Router`](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. +Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:Bonsai.ZeroMQ.Dealer) client that sends a message receives a reply from the [`Router`](xref:Bonsai.ZeroMQ.Router) server. However, in order to address these messages to specific other clients we need to take a slightly different approach. Instead of using the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator we are going to build messages ourselves and pass them directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). - Delete the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) and [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) branches. - Create a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and name it 'RouterMessages' (right-click on an operator with a `NetMQMessage` output type >> CreateSource >> BehaviorSubject). Connect it as an input to the [`Router`](xref:Bonsai.ZeroMQ.Router). From 6e59042fd40adf0fbad4e3a4ad9c214b0df6bbe9 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Wed, 29 Jan 2025 09:36:27 +0000 Subject: [PATCH 31/37] Further explanation for BounceBack operator --- docs/tutorials/client-server.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index d14eea8..df5e449 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -128,7 +128,14 @@ Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:B - Delete the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) and [`ConvertToString`](xref:Bonsai.ZeroMQ.ConvertToString) branches. - Create a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and name it 'RouterMessages' (right-click on an operator with a `NetMQMessage` output type >> CreateSource >> BehaviorSubject). Connect it as an input to the [`Router`](xref:Bonsai.ZeroMQ.Router). - Delete the `Request.First` and `Buffer` outputs from [`Router`](xref:Bonsai.ZeroMQ.Router). Right-click on the [`Router`](xref:Bonsai.ZeroMQ.Router) and expose the `NetMQMessage` >> `First` >> `Buffer` output. -- Add a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BouceBack`. Connect [`Router`](xref:Bonsai.ZeroMQ.Router) to `BounceBack`. +- Add a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) called `BounceBack`. Connect [`Router`](xref:Bonsai.ZeroMQ.Router) to `BounceBack`. + +:::workflow +![Server message multicast](~/workflows/server-message-multicast.bonsai) +::: + +Since the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator has changed from a `ResponseContext` to a `NetMQMessage` (due to the change in its input) we made some modifications to how we process the stream. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator we will construct messages by splitting the `NetMQMessage` into its component `NetMQFrame` parts, extracting the relevant frames and merging them together. + - Inside `BounceBack`, expose the `First` property of the `Source1` output. - On a separate branch, add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator with an index 'Value' of 1. Connect `Source1` as its input. - On a further separate branch, add a [`String`](xref:Bonsai.Expressions.StringProperty) operator with a 'Value' of 'ServerResponse'. Connect this to a [`ConvertToFrame`](xref:Bonsai.ZeroMQ.ConvertToFrame) operator. Connect `Source1` as an input to the [`String`](xref:Bonsai.Expressions.StringProperty). From 8cb89d021a5fb488d365e2b60e761767ed04ca85 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Wed, 29 Jan 2025 09:38:56 +0000 Subject: [PATCH 32/37] Add inside peek to BounceBack operator Co-authored-by: Shawn Tan --- docs/tutorials/client-server.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index df5e449..306ac90 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -144,7 +144,7 @@ Since the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator has - Finally, connect the output of [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) to a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) targeting `RouterMessages`. Connect the [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) output to the `WorkflowOutput`. :::workflow -![Server message multicast](~/workflows/server-message-multicast.bonsai) +![BounceBack](~/workflows/server-bounceback.bonsai) ::: We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator in this implementation we need to pass messages directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). To do this we generate a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [`Router`](xref:Bonsai.ZeroMQ.Response). This will change the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. From ecb08e8479917db9cc1f41872c40f7093eb7a9fb Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Wed, 29 Jan 2025 10:05:35 +0000 Subject: [PATCH 33/37] Add zeromq dependency to local environment and add bounceback workflow --- .bonsai/Bonsai.config | 21 +++++++++ .bonsai/Bonsai.exe.settings | 18 ++++++++ docs/workflows/server-bounceback.bonsai | 58 +++++++++++++++++++++++++ 3 files changed, 97 insertions(+) create mode 100644 .bonsai/Bonsai.exe.settings create mode 100644 docs/workflows/server-bounceback.bonsai diff --git a/.bonsai/Bonsai.config b/.bonsai/Bonsai.config index 2a57896..b1e9778 100644 --- a/.bonsai/Bonsai.config +++ b/.bonsai/Bonsai.config @@ -1,6 +1,7 @@  + @@ -10,15 +11,24 @@ + + + + + + + + + @@ -31,8 +41,10 @@ + + @@ -42,17 +54,26 @@ + + + + + + + + + diff --git a/.bonsai/Bonsai.exe.settings b/.bonsai/Bonsai.exe.settings new file mode 100644 index 0000000..e459a08 --- /dev/null +++ b/.bonsai/Bonsai.exe.settings @@ -0,0 +1,18 @@ + + + Maximized + Light + 397 + + 930 + 475 + 700 + 450 + + + + 2025-01-29T10:05:14.8047649+00:00 + C:\Users\erski\source\repos\RoboDoig\zeromq\docs\workflows\server-bounceback.bonsai + + + \ No newline at end of file diff --git a/docs/workflows/server-bounceback.bonsai b/docs/workflows/server-bounceback.bonsai new file mode 100644 index 0000000..220bf68 --- /dev/null +++ b/docs/workflows/server-bounceback.bonsai @@ -0,0 +1,58 @@ + + + + + + Source1 + + + First + + + + 1 + + + + + ServerResponse + + + + + + + + + + + 3 + + + + + + + RouterMessages + + + + + + + + + + + + + + + + + + \ No newline at end of file From e8895e7b97e7523af4670ac37b9343dca7a8b504 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Wed, 29 Jan 2025 10:24:58 +0000 Subject: [PATCH 34/37] Clearer bounceback explanation Co-authored-by: Shawn Tan --- docs/tutorials/client-server.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index 306ac90..456ea19 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -136,9 +136,9 @@ Running this workflow, you should see a 'bounceback' where any [`Dealer`](xref:B Since the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator has changed from a `ResponseContext` to a `NetMQMessage` (due to the change in its input) we made some modifications to how we process the stream. Inside the [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator we will construct messages by splitting the `NetMQMessage` into its component `NetMQFrame` parts, extracting the relevant frames and merging them together. -- Inside `BounceBack`, expose the `First` property of the `Source1` output. -- On a separate branch, add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator with an index 'Value' of 1. Connect `Source1` as its input. -- On a further separate branch, add a [`String`](xref:Bonsai.Expressions.StringProperty) operator with a 'Value' of 'ServerResponse'. Connect this to a [`ConvertToFrame`](xref:Bonsai.ZeroMQ.ConvertToFrame) operator. Connect `Source1` as an input to the [`String`](xref:Bonsai.Expressions.StringProperty). +- Inside `BounceBack`, expose the `First` property of the `Source1` output, which will give us the address of the `NetMQMessage`. +- On a separate branch, add an [`Index`](xref:Bonsai.Expressions.IndexBuilder) operator with an index 'Value' of 1. Connect `Source1` as its input. This allows us to grab the middle empty delimiter frame. +- On a further separate branch, add a [`String`](xref:Bonsai.Expressions.StringProperty) operator with a 'Value' of 'ServerResponse'. Connect this to a [`ConvertToFrame`](xref:Bonsai.ZeroMQ.ConvertToFrame) operator. Connect `Source1` as an input to the [`String`](xref:Bonsai.Expressions.StringProperty). This will serve as the message content. - Use [`Merge`](xref:Bonsai.Reactive.Merge) to combine the outputs of these 3 branches (`Source1.First`, `Index`, `ConvertToFrame`). - Convert the output to a `NetMQMessage` by connecting a [`Take`](xref:Bonsai.Reactive.Take) operator with a 'Count' property of 3 followed by a [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) operator. - Finally, connect the output of [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) to a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) targeting `RouterMessages`. Connect the [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) output to the `WorkflowOutput`. From d0f8e9599a217fdc4b1492727cf5e1424870f56c Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Wed, 29 Jan 2025 10:25:35 +0000 Subject: [PATCH 35/37] Avoid redundant explanation from step breakout Co-authored-by: Shawn Tan --- docs/tutorials/client-server.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/tutorials/client-server.md b/docs/tutorials/client-server.md index 456ea19..5d19586 100644 --- a/docs/tutorials/client-server.md +++ b/docs/tutorials/client-server.md @@ -147,9 +147,7 @@ Since the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator has ![BounceBack](~/workflows/server-bounceback.bonsai) ::: -We had to change quite a few things to modify this workflow so let's step through the general logic. The first thing to note is that since we are avoiding the [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse) operator in this implementation we need to pass messages directly into the [`Router`](xref:Bonsai.ZeroMQ.Router). To do this we generate a [`BehaviorSubject`](xref:Bonsai.Reactive.BehaviorSubject) source with a `NetMQMessage` output type and connect it to the [`Router`](xref:Bonsai.ZeroMQ.Response). This will change the output type of the [`Router`](xref:Bonsai.ZeroMQ.Router) operator from a `ResponseContext` to a `NetMQMessage` so we need to make some modifications to how we process the stream. - -We want the [`Router`](xref:Bonsai.ZeroMQ.Router) to generate a reply message every time it receives a request from a [`Dealer`](xref:Bonsai.ZeroMQ.Dealer). Since we are now building this message ourselves instead of using [`SendResponse`](xref:Bonsai.ZeroMQ.SendResponse), we branch off the [`Router`](xref:Bonsai.ZeroMQ.Router) with a [`SelectMany`](xref:Bonsai.Reactive.SelectMany) operator. Inside, we split the `NetMQMessage` into its component `NetMQFrame` parts, taking the `First` frame for the address, using [`Index`](xref:Bonsai.Expressions.IndexBuilder) to grab the middle empty delimiter frame and creating a new `String` which we convert to a `NetMQFrame` for the message content. We [`Merge`](xref:Bonsai.Reactive.Merge) these component frames back together and use a [`Take`](xref:Bonsai.Reactive.Take) operator (with count = 3) followed by [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage). The [`Take`](xref:Bonsai.Reactive.Take) operator is particularly important here as 1) [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). Finally, we use a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [`Router`](xref:Bonsai.ZeroMQ.Router). +The [`Take`](xref:Bonsai.Reactive.Take) operator is particularly important here as 1) [`ToMessage`](xref:Bonsai.ZeroMQ.ToMessage) will only complete the message once the observable stream is completed and 2) We need to close the observable anyway to complete the [`SelectMany`](xref:Bonsai.Reactive.SelectMany). We use a [`MulticastSubject`](xref:Bonsai.Expressions.MulticastSubject) to send our completed message to the [`Router`](xref:Bonsai.ZeroMQ.Router). If we run the workflow now, we should see the same behavior as before (server bounces message back to initiating client). From 463c767d6f65fce77ad2101bdab9537d29161bfd Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 30 Jan 2025 11:51:48 +0000 Subject: [PATCH 36/37] Ignore and stop tracking bonsai .exe.settings --- .bonsai/.gitignore | 1 + .bonsai/Bonsai.exe.settings | 18 ------------------ 2 files changed, 1 insertion(+), 18 deletions(-) delete mode 100644 .bonsai/Bonsai.exe.settings diff --git a/.bonsai/.gitignore b/.bonsai/.gitignore index 0cdead6..93f09a9 100644 --- a/.bonsai/.gitignore +++ b/.bonsai/.gitignore @@ -1,2 +1,3 @@ *.exe +*.exe.settings Packages/ \ No newline at end of file diff --git a/.bonsai/Bonsai.exe.settings b/.bonsai/Bonsai.exe.settings deleted file mode 100644 index e459a08..0000000 --- a/.bonsai/Bonsai.exe.settings +++ /dev/null @@ -1,18 +0,0 @@ - - - Maximized - Light - 397 - - 930 - 475 - 700 - 450 - - - - 2025-01-29T10:05:14.8047649+00:00 - C:\Users\erski\source\repos\RoboDoig\zeromq\docs\workflows\server-bounceback.bonsai - - - \ No newline at end of file From c326648cbaf2c05bc83b1f9ece23a7544e337978 Mon Sep 17 00:00:00 2001 From: RoboDoig Date: Thu, 30 Jan 2025 11:58:39 +0000 Subject: [PATCH 37/37] Revert bonsai.config to exclude ZeroMQ and dependencies --- .bonsai/Bonsai.config | 21 --------------------- 1 file changed, 21 deletions(-) diff --git a/.bonsai/Bonsai.config b/.bonsai/Bonsai.config index b1e9778..2a57896 100644 --- a/.bonsai/Bonsai.config +++ b/.bonsai/Bonsai.config @@ -1,7 +1,6 @@  - @@ -11,24 +10,15 @@ - - - - - - - - - @@ -41,10 +31,8 @@ - - @@ -54,26 +42,17 @@ - - - - - - - - -