Looking for ideas how to extract data to Home Assistant

[damclarkd]

Jason, 1st off - Great tool! I have used this for over year since getting solar and 2 PW's. Running on rpi4 with zero issues. Getting all the data I want on Grafana dashboards.
Recently moved over to Home Assistant running on a separate VM on a NUC. HA has a Tesla Powerwall integration that I am testing but having some issues maintaining a connection. (Not at all as stable as Powerwall-Dashboard). What I am wondering is if all the info is in Powerwall-Dashboard with pypowerwall, is there a way to pull that data out and get it into HA somehow?
Again, just looking for suggestions on a path to pursue.

Thanks and cheers!

[jasonacox]

Hi @damclarkd ! Thanks for the great feedback!

On the HA Tesla Powerwall integration, I think I may know why you see some issues connecting to the Powerwall. The reason I wrote the pyPowerwall API was because I couldn't find a good library that didn't have connection issues or that didn't cause the Powerwall Gateway to crash after a while. Upon examination, most of the other libraries flood the Powerwall Gateway API with a mountain of requests. The web server in the Gateway isn't that robust, especially if you are hitting it with many different monitoring tools. I decided to write a library that would help with that. The pyPowerwall library caches all API calls for 10s (default setting) and reuses the same session token until it expires. That made a huge impact on stability.

The HA plugin for Tesla Powerwall appears to be using the tesla_powerwall python library but doesn't provide for any caching. I suspect pyPowerwall could be used instead. It would offer more stability as well as offer up additional data to graph via the device vitals payload that as far as I know, tesla_powerwall doesn't yet pull (it is a protobuf encoded payload). I will take a look to see if we can fork the integration and use pyPowerwall instead. If you are already running the pyPowerwall Proxy container, it could be as simple as pointing the integration to use that. But, I would welcome any help if anyone has time. :)

[damclarkd]

So since posting here I have made several changes to my lab setup that has currently solved the HA integration problem. For the last 24 hours the Tesla PW integration has been rock solid.
Steps taken to reduce load on PW calls:
1.) I have your Powerwall Dashboard running on docker in a VM. As stated already I was able to move all my historical data off the install on a RPi into this new VM install.
2.) I removed another system on a RPi that was polling the PW and uploading to PVoutput.
3.) I removed a testing lab RPi from PW usage.

Because I find your PW Dashboard to be so stable, that is the only local process polling the PW's now.

After all those changes, the HA PW integration has stopped dropping offline.
I now use Node Red in HA to gather and push the data I need up to PVoutput. Because that data is already in HA, there is no extra load on calls to the PW.
Because not everything I want in HA is in the Tesla PW Integration, I am using REST GET commands in Node Red to pypowerwall to get the extra data I am looking for. Again, no extra load on calls to PW.
While all this is new, it appears that with your PW Dashboard/pypowerwall installed locally and the HA Integration being the only other load, all seems happy.
Time will tell. Thanks for your tools and help.
Cheers

[jasonacox]

That's great news @damclarkd - some follow up questions if you don't mind:

• What are you using to push the data into PVoutput? For those not using HA, I'm wondering if I can make this easy since the same data is in InfluxDB, perhaps with an included script/cronjob.
• In yoursetup, it seems like you have two processes polling PW - the Dashboard and the HA Tesla PW Integration - is that correct? If I were to submit an update (PR or new integration) to the HA integration to allow it to use the pypowerwall proxy and include Powerwall+ and other metrics (e.g. string data or other elements you are using Node Red to gather), would that be interesting?
• Can you share some screenshots or code details on your HA setup, specifically the Node Red added data?

Thanks for the feedback and help!

[damclarkd]

@jasonacox In answer to your bullet point:

1. I am currently using Node Red in HA to push to PVoutput. I was using https://github.com/VirtualWolf/powerwall-to-pvoutput-uploader previously. Obviously that was adding load on the calls to PW. I found and adapted someones flow for uploading to PVoutput to fit my needs. PVoutput does have pretty good documentation for their API at https://pvoutput.org/help/api_specification.html. I am a donating member so have access to 5 minute uploads and extended data. I would think that this PVOutput option as part of PW Dashboard/pypowerwall would be well received. I can send a JSON of the flow that you can load into Node Red if that would help.
2. Yes, very interesting. In fact I have started to move away from the entities in the HA PW Integration because I am getting everything I need (plus more) just using GET calls to my pypowerwall install. (Why have more calls to the PW two just get duplicate data?) With the integration we are limited to only what they expose. I am using GET calls in Node Red for those pypowerwall data points I want but you can also use the REST sensor configuration in HA to pull and populate entities.
3. Sure. Do you want those posted here?

[jasonacox]

Thanks @damclarkd - this is super helpful! Yes, please feel free to copy/paste any images (Github convert images pastes to uploads) or code that would be helpful for the community here. I'll work on integrating more of it into the Dashboard stack. My TODO thoughts:

1. Add an easy option/automation to upload 5-minute or daily stats to PVOutput. Please send the JSON of the flow you use in Node Red so others can use that too. I created a tools directory placeholder in the repo that we can use to list these tools and any helpful files or scripts.
2. Easier and more comprehensive data integration with HA - I think your approach is genius and suspect others could quickly do the same (Node Red or REST config path). I suspect the popularity of HA is due to the ease of adding an integration with a simple click. I'm going to work on a near in-place pypowerwall library version of the current integration (with added data point) as well as an option to pull the data via the pypowerwall proxy. Can you share what data set that you are pulling via NodeRed right now?

[damclarkd]

@jasonacox
Node Red GET's from PyPowerwall currently. (Very much in flux as I convert over from HA Tesla Powerwall entities.)

[damclarkd]

Hello @jasonacox
Just wanted to update you that the Node Red solution using GET's to pyPowewall mentioned above, is working so well that I have abandoned using the HA Tesla Powerwall integration all together. I get everything I want, plus more, and thus don't have the extra calls to the PW from the HA Integration. Still think your idea of using pyPowerwall in the HA Integration would be a great solution long term. But for now I am very happy without the HA Integration.
Cheers!

[chinswain]

Did you ever get anything setup attempting to connect Home Assistant via pyPowerwall proxy @jasonacox? Seems like the perfect solution with all of the current issues around the Tesla API and offical HA integration not being able to use the PW3 API.

[jasonacox]

Interesting. What options are there for HA right now? I don't mind trying to help get a plugin to work, but don't want to duplicate if there is another project we could contribute to. Any HA gurus able to help?