Shelly: Failed to create MQTT channel #11

FloppyKiller · 2020-05-07T06:52:12Z

Fehler beim hinzufügen von Shellys auf neu aufgesetzten nymea server.
Installation des Servers nach dieser Anleitung: [https://nymea.io/wiki/nymea/master/install/ubuntu
]

OS
Ubuntu Server 18.04LTS

Output der Debug:

I | Shelly: Found shelly thing! ZeroConfServiceEntry("shelly1-76FB03")
location: "10.66.66.12":80
hostname: "shelly1-76FB03.local"
domain: "local"
service type: "_http._tcp"
protocol: QAbstractSocket::NetworkLayerProtocol(IPv4Protocol)
txt: ("id=shelly1-76FB03", "fw_id=20200320-123430/v1.6.2@514044b4", "arch=esp8266")

W | Mqtt: Unable to find a matching MQTT server port for client address "10.66.66.12"
W | Shelly: Failed to create MQTT channel.
W | ThingManager: Thing setup failed. Not adding thing to system.

Ports am Server sind offen:

root@nymea:~# lsof -i -P -n
COMMAND PID USER FD TYPE systemd-n 74 systemd-network 16u IPv4 96983506 systemd-r 97 systemd-resolve 12u IPv4 96180568 systemd-r 97 systemd-resolve 13u IPv4 96180569 avahi-dae 154 avahi 12u IPv4 96180397 avahi-dae 154 avahi 13u IPv6 96180398 avahi-dae 154 avahi 14u IPv4 96180399 avahi-dae 154 avahi 15u IPv6 96180400 nymead 201 root 9u IPv6 96182559 nymead 201 root 12u IPv4 96181575 nymead 201 root 13u IPv4 96181576 nymead 201 root 14u IPv4 96182481 nymead 201 root 15u IPv4 96182482 nymead 201 root 16u IPv4 96182483 nymead 201 root 18u IPv4 96182572 nymead 201 root 20u IPv4 96202037 nymead 201 root 21u IPv4 96182592 nymead 201 root 24u IPv6 96219377 nymead 201 root 25u IPv4 96999542 nymead 201 root 26u IPv6 96999940 sshd 218 root 3u IPv4 96181299 sshd 218 root 4u IPv6 96181313 master 392 root 13u IPv4 96181803 master 392 root 14u IPv6 96181804 DEVICE SIZE/OFF NODE NAME
0t0 UDP 10.0.0.135:68
0t0 UDP 127.0.0.53:53
0t0 TCP 127.0.0.53:53 (LISTEN)
0t0 UDP *:5353
0t0 UDP *:5353
0t0 UDP *:47749
0t0 UDP *:41444
0t0 UDP *:49880
0t0 TCP *:2222 (LISTEN)
0t0 TCP *:4444 (LISTEN)
0t0 TCP *:443 (LISTEN)
0t0 TCP *:80 (LISTEN)
0t0 TCP *:1883 (LISTEN)
0t0 UDP *:1900
0t0 TCP *:1884 (LISTEN)
0t0 TCP 10.0.0.135:2222->10.0.11.15:55156 (ESTABLISHED)
0t0 TCP *:2626 (LISTEN)
0t0 TCP 10.0.0.135:80->10.0.1.29:41238 (ESTABLISHED)
0t0 TCP 10.0.0.135:2626->10.0.11.15:59286 (ESTABLISHED)
0t0 TCP *:22 (LISTEN)
0t0 TCP *:22 (LISTEN)
0t0 TCP 127.0.0.1:25 (LISTEN)
0t0 TCP [::1]:25 (LISTEN)

Nymea Logs nach Neustart von nymead über systemctl und erneuten Versuch einen Shelly hinzuzufügen:

root@nymea:# journalctl -f -u nymead
-- Logs begin at Tue 2020-05-05 20:13:07 UTC. --
Mai 06 07:45:07 nymea systemd[1]: nymead.service: Failed with result 'signal'.
Mai 06 07:45:07 nymea systemd[1]: Stopped nymead - IoT server.
Mai 06 07:45:17 nymea systemd[1]: nymead.service: Failed to reset devices.list: Operation not permitted
Mai 06 07:45:17 nymea systemd[1]: Started nymead - IoT server.
Mai 06 07:45:17 nymea nymead[1172]: I | Application: =====================================
Mai 06 07:45:17 nymea nymead[1172]: I | Application: nymead 0.20.0+202004051443bionic1 started as root.
Mai 06 07:45:17 nymea nymead[1172]: I | Application: =====================================
Mai 06 07:45:18 nymea nymead[1172]: W | default: QFSFileEngine::open: No file name specified
Mai 06 07:45:18 nymea nymead[1172]: W | ServerManager: Could not open "" : "No file name specified"
Mai 06 07:45:18 nymea nymead[1172]: W | ServerManager: Using fallback self-signed SSL certificate: "/var/lib/nymea/certs/nymead-certificate.crt"
Mai 06 07:45:43 nymea nymead[1172]: W | default: Cannot find a running Bluez. Please check the Bluez installation.
Mai 06 07:45:43 nymea nymead[1172]: I | Mqtt: MQTT server running on QHostAddress("0.0.0.0") : 1883
Mai 06 07:45:43 nymea nymead[1172]: I | Mqtt: MQTT server running on QHostAddress("0.0.0.0") : 1884
Mai 06 07:46:08 nymea nymead[1172]: W | Bluetooth: No bluetooth adapter found. Resource not available.
Mai 06 07:46:33 nymea nymead[1172]: W | Bluez: Invalid DBus ObjectManager interface.
Mai 06 07:46:33 nymea nymead[1172]: W | Nuki: Bluetooth not available
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Wemo" ( "2e3b5ce0-ecf1-43de-98f0-07df4068a583" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "OpenUv" ( "9b7d9cc8-77df-4197-a6fc-8a365747a3b1" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "TexasInstruments" ( "ae550a91-e734-4331-9d71-9f37df0b0fa6" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Netatmo" ( "69d14951-0c02-4877-bcef-dffdf48b7ccb" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Bose" ( "472a3f24-b05c-49b3-ad9a-dfda608b6760" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "NetworkDetector" ( "8e0f791e-b273-4267-8605-b7c2f55a68ab" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "UsbRelay" ( "ed0035d3-561c-498e-bdb2-2b574cbd0a2f" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "EQ3" ( "f324c43c-9680-48d8-852a-93b2227139b9" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "DateTime" ( "c26014c6-87fb-4233-85ed-01d18625018d" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "OpenWeatherMap" ( "bc6af567-2338-41d5-aac1-462dec6e4783" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "WakeOnLan" ( "b5a87848-de56-451e-84a6-edd26ad4958f" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Denon" ( "cd758269-dbbb-4ef0-80ab-48bd9a8a2765" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Nanoleaf" ( "360867ec-1594-498d-8182-fbab1fe17489" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "AirQualityIndex" ( "57d69b76-4d2d-41ec-bef6-949a79ffbe6b" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "MailNotification" ( "1ae35df1-1b51-4c93-94fa-3febc77e0318" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Tado" ( "b4f2d2ee-50bb-4786-b7f5-261fed204fa5" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "unifi" ( "88bc00c7-9ea8-4aa6-8aec-831639e8fccc" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "shelly" ( "6162773b-0435-408c-a4f8-7860d38031a9" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "tasmota" ( "d136e0c0-0cbf-4731-aabb-b2201088d6cb" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Senic" ( "413e9d77-335f-4ecf-abbc-8f2a8a399c39" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Elgato" ( "c5c03ad4-bfdb-444a-8eca-2c234c46cc27" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Kodi" ( "e7186890-99fa-4c5b-8247-09c6d450d490" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "PhilipsHue" ( "5f2e634b-b7f3-48ee-976a-b5ae22aa5c55" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "LgSmartTv" ( "4ef7a68b-9da0-4c62-b9ac-f478dc6f9f52" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "anelElektronik" ( "7a3e5b64-20e4-42bd-b86b-989b84afc22a" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "KebaKeContact" ( "9142b09f-30a9-43d0-9ede-2f8debe075ac" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "awattar" ( "9c261c33-d44e-461e-8ec1-68803cb73f12" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Pushbullet" ( "46986575-0e62-483d-b5a8-76ac356fcce7" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "Sonos" ( "cdb07719-c445-4fa5-9c7a-564ee02a4412" )
Mai 06 07:46:33 nymea nymead[1172]: W | Translations: * Could not load translation "de_AT" for plugin "daylightSensor" ( "4b7d63a7-fc39-4a50-a459-457fa7653089" )
Mai 06 07:46:39 nymea nymead[1172]: I | Shelly: Found shelly thing! ZeroConfServiceEntry("shelly1-76FB03")
Mai 06 07:46:39 nymea nymead[1172]: location: "10.66.66.12":80
Mai 06 07:46:39 nymea nymead[1172]: hostname: "shelly1-76FB03.local"
Mai 06 07:46:39 nymea nymead[1172]: domain: "local"
Mai 06 07:46:39 nymea nymead[1172]: service type: "_http._tcp"
Mai 06 07:46:39 nymea nymead[1172]: protocol: QAbstractSocket::NetworkLayerProtocol(IPv4Protocol)
Mai 06 07:46:39 nymea nymead[1172]: txt: ("id=shelly1-76FB03", "fw_id=20200320-123430/v1.6.2@514044b4", "arch=esp8266")
Mai 06 07:46:43 nymea nymead[1172]: W | Mqtt: Unable to find a matching MQTT server port for client address "10.66.66.12"
Mai 06 07:46:43 nymea nymead[1172]: W | Shelly: Failed to create MQTT channel.
Mai 06 07:46:43 nymea nymead[1172]: W | ThingManager: Thing setup failed. Not adding thing to system.

locales stimmen eigentlich auch: (habe verschiedene durchprobiert: C, de_AT.UTF-8, de_DE)

root@nymea:~# locale
LANG=de_AT
LANGUAGE=
LC_CTYPE="de_AT"
LC_NUMERIC="de_AT"
LC_TIME="de_AT"
LC_COLLATE="de_AT"
LC_MONETARY="de_AT"
LC_MESSAGES="de_AT"
LC_PAPER="de_AT"
LC_NAME="de_AT"
LC_ADDRESS="de_AT"
LC_TELEPHONE="de_AT"
LC_MEASUREMENT="de_AT"
LC_IDENTIFICATION="de_AT"
LC_ALL=

mzanetti · 2020-05-07T08:46:21Z

Thanks for the report. Would you mind attaching the output of ifconfig please?

Seems like the code fails to match the Shelly's IP address to a network interface on the server. Afaict from the logs the server address is 10.0.0.135 and the shelly's is 10.66.66.12.

I have created this sample code to reproduce it

    QHostAddress shelly("10.66.66.12");
    QHostAddress serverAddress("10.0.0.135");
    QHostAddress serverNetmask("255.0.0.0");
    QNetworkAddressEntry serverAddressEntry;
    serverAddressEntry.setIp(serverAddress);
    serverAddressEntry.setNetmask(serverNetmask);
    qDebug() << "Is in subnet:" << shelly.isInSubnet(serverAddressEntry.ip(), serverAddressEntry.prefixLength());

but it returns "true" so it should work. Which probably means that either I got one of the IP addresses wrong from the logs, or your subnet mask is not 255.0.0.0.

The output of ifconfig would allow me to verify both.

FloppyKiller · 2020-05-07T09:06:54Z

my subnet mask is /16, so 255.255.0.0 on the server network and /24 - 255.255.255.0 on the shelly network. Both networks are completely open in the firewall.

I don't have ifconfig installed, but ip a shows:

root@nymea:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
126: eth0@if127: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 56:23:bb:cc:d1:17 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.0.0.135/16 brd 10.0.255.255 scope global dynamic eth0
valid_lft 3352sec preferred_lft 3352sec
inet6 fe80::5423:bbff:fecc:d117/64 scope link
valid_lft forever preferred_lft forever

mzanetti · 2020-05-07T09:10:36Z

right... so as the subnetmask is /24 (which is the same as 255.255.255.0) the shelly is in a different network... With a mask of 255.255.255.0 your machine can only communicate to other hosts having IP addresses from 10.0.0.1 to 10.0.0.254.

As you're apparently in a class A network, you should set the mask to /8 (or 255.0.0.0). Then it should start working.

mzanetti · 2020-05-07T09:13:10Z

sorry... I've mistaken the mask a bit, but still the argument is valid...

With this ip config 10.0.0.135/16 you'd be able to communicate with hosts in 10.0.x.x, but the shelly is in 10.66.66.12/24, so it would only be able to communicate with hosts of 10.66.66.x

mzanetti · 2020-05-07T09:19:30Z

If you've received both ip addresses via DCHP it either hints to a bad router configuration or your administrator doesn't want you to communicate between workstations and mobile devices (which would be rather bad security tho, because the client can override the netmask anyways). Also a bit weird that the shelly got a different subnet mask than your workstation.... Would make me think now that shellies might only take the IP address from DHCP and hardcode a /24 netmask...

FloppyKiller · 2020-05-07T09:24:24Z

I know. But the those are two separate networks.

on net A: (server site net)
root@nymea:~# ping 10.66.66.12
PING 10.66.66.12 (10.66.66.12) 56(84) bytes of data.
64 bytes from 10.66.66.12: icmp_seq=1 ttl=127 time=5.87 ms

on net B (shelly site net):
SmacBook-Pro-HD:~ rcan$ ping 10.0.0.135
PING 10.0.0.135 (10.0.0.135): 56 data bytes
64 bytes from 10.0.0.135: icmp_seq=0 ttl=63 time=1.111 ms

There is one hop:
SmacBook-Pro-HD:~ rcan$ traceroute 10.0.0.135
traceroute to 10.0.0.135 (10.0.0.135), 64 hops max, 52 byte packets
1 10.66.66.1 (10.66.66.1) 1.149 ms 1.141 ms 0.890 ms
2 10.0.0.135 (10.0.0.135) 1.096 ms 8.074 ms 9.905 ms

mzanetti · 2020-05-07T09:47:22Z

hmmm I see... so those are indeed separate networks but routed between each other... The code doesn't cope with that (yet)...

So the caveat there is that nymea needs to configure the MQTT server on the shelly... For that it needs to find out which IP address it should tell the shelly to connect to. But given both devices are in different networks, it can't know that there is a router routing between them.

I'm thinking how I could find out in the code that this would still work by just telling the shelly to use 10.0.0.135...
Perhaps one option could be to add an additional parameter during setup where you'd need to specify that manually for such "complex" network setups.
Or maybe I could try to resolve a route to the shelly and extract information from that... hmm... need to fiddle with this.

FloppyKiller · 2020-05-07T09:53:47Z

The code doesn't cope with that (yet)

Ok. I'll try adding the server into the shelly network for now.

I think this issue could be closed for now.

Thank you.
best regards,
Ricardo

mzanetti · 2020-05-07T10:02:31Z

Hmm... no... I'll keep this open... IMHO this can and should be fixed in nymea... It actually belongs to nymea itself tho, not nymea-mqtt (as the MQTT implementation itlsef really doesn't care about this stuff)

mzanetti · 2020-05-07T10:02:40Z

Thanks again for the report

FloppyKiller closed this as completed May 7, 2020

mzanetti reopened this May 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Shelly: Failed to create MQTT channel #11

Shelly: Failed to create MQTT channel #11

FloppyKiller commented May 7, 2020

mzanetti commented May 7, 2020

FloppyKiller commented May 7, 2020

mzanetti commented May 7, 2020

mzanetti commented May 7, 2020

mzanetti commented May 7, 2020

FloppyKiller commented May 7, 2020

mzanetti commented May 7, 2020

FloppyKiller commented May 7, 2020 •

edited

Loading

mzanetti commented May 7, 2020

mzanetti commented May 7, 2020

Shelly: Failed to create MQTT channel #11

Shelly: Failed to create MQTT channel #11

Comments

FloppyKiller commented May 7, 2020

mzanetti commented May 7, 2020

FloppyKiller commented May 7, 2020

mzanetti commented May 7, 2020

mzanetti commented May 7, 2020

mzanetti commented May 7, 2020

FloppyKiller commented May 7, 2020

mzanetti commented May 7, 2020

FloppyKiller commented May 7, 2020 • edited Loading

mzanetti commented May 7, 2020

mzanetti commented May 7, 2020

FloppyKiller commented May 7, 2020 •

edited

Loading