Microinverter HEARTBEAT (COUNTER) response implementation for cloudless time synchronization

[davidrapan]

Hi,

• Implemented in https://github.com/davidrapan/ha-solarman/blob/58409d64e83ab1a053c26dc88e0fd49dae5be4b5/custom_components/solarman/api.py#L56
• Used in https://github.com/davidrapan/ha-solarman/blob/58409d64e83ab1a053c26dc88e0fd49dae5be4b5/custom_components/solarman/api.py#L107

As it's currently "glued in" in a really ugly way (w/ refactored COUNTER to V5_HEARTBEAT) but for proof of concept enough I guess. So what would be the best place of handling that? My first thoughts were about removing that check from _received_frame_is_valid and add some additional flow later in the main loop but where? Any ideas?

Log:

2024-12-20 09:34:06.435 DEBUG (MainThread) [custom_components.solarman.api] [38????????] V5_HEARTBEAT: a5 01 00 10 47 02 d7 cb ea 91 e6 00 5d 15
2024-12-20 09:34:06.436 DEBUG (MainThread) [custom_components.solarman.api] [38????????] V5_HEARTBEAT RESPONSE: a5 0a 00 10 17 03 cb ea 91 e6 00 01 fe 2b 65 67 00 00 00 00 56 15

😉

[githubDante]

If the time of the inverter is synchronized from the V5 frame probably we can add an optional argument to enable time stamping of all packets. Or maybe the synchronization only happens with the keep alive response from the server?

---

By the way this V5_HEARTBEAT RESPONSE is wrong and probably not processed at all. The correct one should be:

a5 0a 00 10 17 03 d7 cb ea 91 e6 00 01 fe 2b 65 67 00 00 00 00 2d 15

It is generated today at 2024-12-20 08:34:06+00:00

Edit: timestamp correction

[davidrapan]

By the way this V5_HEARTBEAT RESPONSE is wrong and probably not processed at all. The correct one should be:

Ah mistakes were made. request_frame[5:6] should have been request_frame[5:7], thanks!

Edit3: fix: Incomplete seq num for V5 HEARTBEAT response

Or maybe the synchronization only happens with the keep alive response from the server?

As far as I know it happens with response to HANDSHAKE packets (which are sent only in server mode, I think) and with response to that HEARTBEAT (COUNTER)... not sure if there are any other ways. I don't have currently any microinverters to test it out so I rely solely on echos from users atm, hah.

Edit: HANDSHAKE code: 0x41 w/ response code 0x11 and same response content.

Edit2: Hm, for that seq number should be proly used struct.pack("<H", self._get_next_sequence_number()) anyway.

[davidrapan]

[Async]
So I guess the best approach will be to add new method right behind (in _conn_keeper method):

if not self._received_frame_is_valid(data):
    continue

w/ similar footprint which would contain some matching for 0x41 and 0x47 packets.

[Sync]
And exactly the same but in _data_receiver method.

The only question I have is about "send flow". In my example i just copy pasted sending part of _send_receive_v5_frame but wouldn't be better to cut it out into separate method so the same code is used in mentioned method and also for those specific HANDSHAKE and HEARTBEAT replies?

[githubDante]

From what I see, the communication with the Solarman servers is a bit different in terms of packet structure. Timestamp is included in all server generated packets (ACKs to data packets sent by the logger). The format looks like:

• UNIX timestamp (4 bytes) [Total Working Time here]
• TZ offset in minutes (4 bytes) [Power On Time here]

---

• A complete server packet structure
  

---

Note: A logger attached to a Deye (SUN-12K-SG04LP3 / LSW-3) inverter completely ignores the server timestamp and uses its own. The difference is close to 2 hours between the two and does not seem to change. The inverter time is completely different.

[davidrapan]

Yes that seems about right. Hybrid inverters doesn't even sends that heartbeat packets but microinverters do and I also know that microinverters will use time send in that packet.

[githubDante]

Hybrid inverters doesn't even sends that heartbeat packets

They do, plus a lot more:

    LoggerConnect = 0x4110
    LoggerDataReport = 0x4210
    LoggerSSIDReport = 0x4310
    Logger4810Report = 0x4810
    """ Strange logger report ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff """

    ServerPingResponse = 0x1710
    ServerSSIDAck = 0x1310
    ServerDataAck = 0x1210
    ServerConnOK = 0x1110
    Server4810Ack = 0x1810

From these is obvious that the control code in the server response is always -> (logger control code) - 0x3000

[davidrapan]

My device so far never did in an inverter as server type of a communication...

Edit: But maybe it's the matter of a firmware version or something. 😉

[githubDante]

This is information uploaded to the Solarman servers. We see only the keep-alive/heartbeat/counter packet (0x4710) when communicating with the logger, the mode doesn't matter.

[davidrapan]

We see only the keep-alive/heartbeat/counter packet (0x4710) when communicating with the logger, the mode doesn't matter.

Sure, I understand, I'm just saying that I don't see those w/ my SG04LP3.

[davidrapan]

May I ask you what FW are you running w/ your stick and inverter?

I'm now on:

• LSW3_32U_5406_1.06
• 2005-1123-1807

[jmccrohan]

@githubDante

    LoggerConnect = 0x4110
    LoggerDataReport = 0x4210
    LoggerSSIDReport = 0x4310
    Logger4810Report = 0x4810
    """ Strange logger report ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff """

    ServerPingResponse = 0x1710
    ServerSSIDAck = 0x1310
    ServerDataAck = 0x1210
    ServerConnOK = 0x1110
    Server4810Ack = 0x1810

From these is obvious that the control code in the server response is always -> (logger control code) - 0x3000

Thanks for this. This is news to me as my inverter doesn't send these frames. How did you determine this? Just from packet captures? Could you document for the V5 Protocol documentation page if possible?

@davidrapan Could we tidy up _v5_time_response_frame please so that aligns more closely with _v5_frame_encoder? Or possibly merge both? The proposed implementation has a few too many magic numbers (to those that haven't read this thread) for my liking. Could you also add these new frame types to the test suite as well please? Cheers.

[davidrapan]

How did you determine this?

You can see them in pcap but also when you use proxy from examples. 😉 They are used in server mode communication, just the 0x47 is sometimes used in the client mode too.

I did it outside of _v5_frame_encoder because of sequence number cause I was not really sure if reusing would not disrupt usual flow where there the number is read (in decoder counter part) and then incremented + response code build as request code - 0x30.

But sure I do agree it could be improved in terms of code reuse. I did the PR to discuss exactly that and what I did so far was w/ intention not to touch existing code too much so is the purpose of new code clear as much as possible for initial draft. 😉

Edit: The only way forward would be to split both current encoder and decoder methods and extract encoding and decoding of the header I guess.

Edit2: And also the order would be in this case decoder -> encoder instead of usual encoder -> decoder calls and the fact that currently processed packet is reflected into current state of the class is quite confusing to me. 😆

Edit3:

Could you also add these new frame types to the test suite as well please? Cheers.

Sure!

[githubDante]

Hi,

Sorry for the delay. I needed some time offline.

@jmccrohan

How did you determine this? Just from packet captures? Could you document for the V5 Protocol documentation page if possible?

Yes, a tcpdump of the entire communication with the solarman servers.

I see that David has already added them in #82

---

@davidrapan

May I ask you what FW are you running w/ your stick and inverter?

LSW3_32U_5406_1.06 - from LoggerConnect (0x4110)
0-2005-1123 0-1001-c031 - from LoggerDataReport (0x4210)