Kafka ML configuration

[Hannick5]

Hello,

I've been using the opentwins solution and I find some difficulties in configuring kafka-ml is there a way to interact with it inside grafana ? Or do I need to enable something during the installation process with Helm ?

[juliarobles]

Hello,

To connect Kafka-ML and OpenTwins you have to create an Eclipse Ditto connection in OpenTwins that receives the data coming from Kafka-ML and updates the corresponding twin. For that you have to create an Eclipse Ditto JavaScript mapping that does that conversion.

For example, suppose you have an ML model in Kafka-ML that returns the predicted humidity and temperature of a device in the format [humidity, temperature]. You also have in OpenTwins a twin with id example:sensor with two features, one for humidity and one for temperature, with the following structure in Eclipse Ditto:

Note: Each feature must have properties and value.

JavaScript mapping would be something like this, which transforms each message to twin features and updates it:

function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {
    const jsonData = JSON.parse(textPayload);
    const humidity = jsonData[0];
    const temperature = jsonData[1];

    headers = Object.assign(headers, { 'Content-Type': 'application/merge-patch+json' });

    var features = {
          humidity: {
               properties: {
                    value: humidity 
               }
         },
         temperature: {
              properties: {
                   value: temperature
              }
         }
    };

    return Ditto.buildDittoProtocolMsg(
        'example',
        'sensor', 
        'things',
        'twin',
        'commands',
        'merge',
        '/features',
        headers,
        features
    );
}

Now create a connection through the OpenTwins interface to Apache Kafka where the prediction is received, including that JS. You can see the fields in the Eclipse Ditto documentation, just emphasize that in “Authorization context” you have to put "pre-authenticated:kafkaml-connection" and don't forget to select in the source "Payload mapping" the id that you have put to the JS mapping. In this way, both technologies would already be connected.

For example, if [23.45, 23.12] is received, the twin would look as follows:

I hope this helps you and if you have any problems, let us know.