Mike_C

Despite continued timeouts via the ios app, this implementation seems rocksolid. I run this every 5 minutes with a crontab in a docker container built on alpine:3.12.4 I honestly found that the "Simple Implementation" section of the github page was all that i needed to get up and running. Whereas the python script needed multiple calls, this seems to pull everything back in a single call. This does make it a little trickier to find the data you're after but a bit of 'jq' processing is all it takes to find the path of data you are after. As requested, I attach the file that i now run. I've commented it where i think it needs it and added some retry logic for the login and api call. I don't suggest this is perfect coding as its still rough and ready but working. I find that calling this every 5 minutes is enough for me to graph in near realtime. The git page details the configuration changes that could be applied; if you are after more historical data, you can change the options={} with options={historyAll:true}.runSolarUpdate_mike.js

The success rate with the python script has recently (since mid Sep 2023) degraded significantly. I also notice that the iOS app and the website are both rather slow or have a high failure rate - at login and/or to display data. I have migrated to a node / javascript implementation I found. It's one login, one method call and one logout. It can be found here: https://github.com/PLCHome/growatt/tree/master I've created a simple docker container from alpine (utilising this to isolate from all other things on my server), wrapped some json parsing functionality around the calls (which i've learnt on the go) to perform the same data processing as with the python script. For me, this appears to be FAR more stable and reliable that the python version and funnily enough the iOS app. Whilst the iOS app was presening network timeouts, I found this script happily returning the data I needed. Thought I would mention this in this thread if others are still struggling with the python version (which I have stopped using myself).

Yes. It’s the same via the app though. Logins (mainly timeouts I feel) are a problem. There are definitely times in the day when it’s more common to have timeouts. perhaps there is a timeout configuration for the api client (and / or some error handling) that can be added. I’ve not checked

Attached processToFiles.py

I don’t believe running as sudo is actually needed hence worth removing that. Keen to understand steps that 0xEF did to get working. I’m unsure from here. Clearly the username / password combo via the scripts should work as well as per the API. I have found that the app has been a bit hit and miss today - with timeouts. Perhaps retrying the API at another time??

Hurrah - glad to help! Main issues were therefore around permissions / paths in Linux rather than the script itself. Do bear in mind that /tmp (as a temporary directory) gets wiped each time the server restarts. Enjoy

let's try setting the outFolder to "/tmp/"

hmmm ... the error that concerns me is : ppvToday_dayAPI = mix_detail_day['chartData']['ppv'][DayOfMonth_i] This suggests that one of the calls made is not returning any data and/or.in the right format. I vaguely understand the error but have not seen it myself and unable to diagnose (after all I'm calling someone else's code). Looking at your original post, and what you are looking to achieve, I have removed the second half of the script (that does much of the calculations) to create a light version (and attached to this post). I propose two options from here 1. growatt-mix_detail-YYYY-MM-DD.json should contain the data you are after. It is in JSON format as returned via the API call, and will need some reformatting, e.g. with the 'jq' command. 2. assuming there are now no errors in this light script version, you should find I have undertaken this reformatting and created growatt-mix_detail_chartData-YYYY-MM-DD.csv. hope this gets you closer to the goal. processToFiles_light.py

You shouldn’t need to run as sudo. Either run as normal user, or try re-running the pip install growattServer command as sudo as well. (My Ubuntu is a bit rusty but there seems to be permissions / who you are running commands as that is causing some issues here). the server.growatt url IS in the package that I’m using. I don’t call it specifically.

The username and password in the script need to be those used to login to the growatt server - https://server.growatt.com/login (and NOT those used to connect / ssh to the ubuntu server). Relative paths can also be used, so try outFolder = "./" # ensure this folder exists and this will create files wherever you run the script from. (to answer your specific question, there is nothing preceding the ~mike in my path. The ~ (tilda) means the user's home directory. You are connecting exactly as I was to the AWS VM (i.e. ubuntu with pem cert file and no password).

I suspect you don’t have permission to write to that folder. If the script fails, at line 91, all files written before that will still be in the folder.

Will be in the output folder (ie created by the script execution)

My cron expression is : */5 23 * * * /opt/homebrew/bin/python3 ~mike/bin/solar/processToFiles.py I'd expect all calls to work for all inverters (and may give inaccurate data rather than throwing an error). If you can send me a copy of your growatt-mix_detail_day-YYYY-MM-DD.json file (it doesn't contain any of your login details), I can see if I can work out the error; it's not one that I've seen in my testing. The alternative is just to comment it out - and all downstream, dependent calls or calculations.

What type of inverter do you have? I believe some of the API calls only work for certain inverter types (I’m more comfortable with coding than inverter types tbh).

Ubuntu and windows installs both work - instruction overview below. I should add that I run this script every 5 minutes via cron between 23:00 and 23:55. This ensures that I get the full day's data and provides some protection in case a single execution times out (which the script does not currently handle very well). With AWS of course, you would not likely need the VM running 24x7, but just when you needed the script to run. Ubuntu: I created an AWS VM this am (based on Ubuntu Server 22.04 LTS (HVM), 64-bit (x86). AMI ID : ami-00c90dbdc12232b58. From ssh session: sudo apt-get update sudo apt-get upgrade // python3 was already installed sudo apt-get install python3-pip pip install growattServer Windows - standard win10 under virtual box: Install python via https://www.python.org/downloads/windows/ ( I went for 3.10.4. Selected options to include pip and add python3.10 to path). From the command line: pip install growattserver Common (Ubuntu and Windows): Make config changes, create output directory, and run...

My overdue update of #3. I have found it to be stable enough / gives me what I want that I have not explored option #2 at all. I have found that growattServer 1.2.0 to be the best for my needs. It can be installed via: pip install growattServer Details can be found here: https://pypi.org/project/growattServer I have no affiliation to this project; I am just one customer / user that has found it to give me what I need. I have this running on an m1 mac and have attached my python3 script. I can't comment on how easier it is to translate this to windows / linux (if any 'translation' is needed at all). Either way, before running, complete the set up variables configuration (line 11 to 15). To run (based on today’s date) : python3 ./processToFiles.py To run (based on YYYY-MM-DD) provide three separate parameters : python3 ./processToFiles.py YYYY MM DD The files generated are a combination of direct API call output and some calculated values. A summary csv is generated detailing (and this is how I think about energy created / used): Solar (PV) to battery (i.e. stored) Solar (PV) to user (i.e. used at time of generation) Solar (PV) to grid (i.e. exported) Battery to user (i.e. used post generation) Grid to battery (should be close to zero but fluctuates) Grid to User (imported energy) I then compare this CSV in Excel to that provided by my energy provider - i.e. exported and imported values. processToFiles.py

Nice! Thank you for sharing.

This question is in reference to the ShineWIFI-S (firmware version 3.0.0.0) with the standard data update interval of 5 minutes (connected to a growatt hybrid inverter). Does anyone have any data as to how much data is stored locally, e.g. in the event of an internet disruption or the destination server (server.growatt.com) not being reachable? I'm assuming a TCP connection, rather than UDP, is made and as such data is acknowledged on the server before the data logger 'forgets' the data.

Which version of fireware were you on / are you on ? I see mine as 3.0.0.0; nothing is broken currently (so not that keen to upgrade the firmware unless necessary).. Keen to understand your rationale / requirement to update - as well as the versions, and current status to the above post.

Follow up for others who may see this. I believe the issue was that the CT clamp (placed between power company’s meter and the consumer unit) was faulty / inaccurate. Installers actually cut away some of the insulation, pls a “turn it off, turn it back on” strategy and all appears to be reporting the right data and no more charging battery from the grid.

Further reading suggests that the historical data download feature used to exist but was removed on 23/11/2020. Site and extracted text. It’s exactly this reason that I like to control my own data. https://github.com/johanmeijer/grott/issues/77 Dear users: In order to bring you a more convenient product experience, we will update the system service platform on November 23th. The specific updates are as follows: 1. New Disclaimer for device setting System Dashboard / Device List, the Disclaimer is added in the device setting, please tick to agree; 2. Stop accessing historical data The system will stop accessing the Dashboard / Device List / Historical Data function; Thank you for you understanding and support

A couple of weeks on, and I'm reporting on my findings to date. There appear to be three options 1. query locally via direct cable - reports of the software for this as sketchy at best. 2. set up a man-in-the-middle - capture the data for local processing and then forward to server.growatt.com. 3. query the growatt hosted server.growatt.com I have not tried #1. I don't plan to as I can't see a current use case (for me) I'm about to try #2. All my home automation 'operations' do not rely on the internet (it's all locally hosted on a mac with indigo software with zwave (rather than wifi) communication protocols). This has been super solid for 10+ years. The use case for #2 is if Growatt shut up shop , or start to charge for the service etc.. - again something I don't want to do. I'm going to try running this in a docker container on an always-on mac. Also, I believe it is possible to increase the frequency of data - from the standard 5 minutes to 1 minute. This will be great to see. This amount of data locally is not a big deal. I have had success with #3. There are a lot of API calls that can be invoked - including getting the 5 minute 'chart data' that you are after. It provides the data in json, so you'll need to do some parsing / processing to get it into Excel friendly format. To date, I have got the json being returned, and am about to work out exactly which is the key data I want. I'm thinking I am going to call all the key APIs every day (for the previous full day as well as intra-day). I will dump these to local files. I plan to import into my influx db and/or create csv for that Excel friendly format. Hope this is of use and/or interest. Let me know if a further update would be useful, or any questions on the above. Full disclosure, I'm based in the UK but , as with most things internet / tech, this does not seem to be a major issue. Just wish I had more Sun hitting the panels.

try: growatt yyyymmdd (no spaces and all lowercase), eg 6th March would be: growatt20220306

Growatt SPH3600 with GBLI 6532 (6KWhbattery) and 12 panels. installed this week - installer being ridiculously slow at responding to my request for support. Settings on inverter do not allow charging battery from grid , yet every night the battery is being charged. The effect being that when solar is generated it’s going back to the grid (for free currently) as battery is already charged!! is this familiar to anyone? Any ideas of help much appreciated. mike

parallel thread (just started). Looks like the answer is no!