Jump to content

Goodwe 5048ES Protocol reverse engineering


Recommended Posts

Seeing as the 'smart' home bug has bitten me in a bad way, I've been working on integrating all the various bits of electronics in the house. One of the obvious things to get integrated is the power monitoring side and in my case, a Goodwe 5048ES inverter. I'm using Home Assistant as my automation system and there is integration for this available, but only to the SEMS portal. This won't do, as the SEMS portal isn't great, it sits in the cloud and it only updates every 5 minutes or so. 

Ideally you want to integrate directly to the inverter and get much more frequent updates, but I couldn't find anything that is integrated directly. Looking at the inverter there's a few RS485 ports available, so it might be possible to get some data off this. But it's also possible that these are only for battery integration or otherwise unused. So, I figured a more elegant way would be to try and talk to the inverter through the network seeing as it's sitting there already. Again, there's pretty much no information available on achieving this but I figured it was worth a try.

Running PV master on a mobile device and a packet sniffer running in the background, I managed to log some interesting data in order to allow me to talk to the inverter through it's network interface. I've made some progress, but this is very much a work in progress; decoding the data is a bit of a challenge but I think I'm about halfway identifying all the data. I'm posting this here as hopefully someone else has a bit more insight and can help filling in the missing blanks, or at the very least, it might help someone with a bit more smarts to crack it properly.

Basically, the communication between PV Master and the inverter is in the form of UDP packets with the inverter listening on port 8899. Because it's UDP, it's connectionless. There's no need to explicitly connect to the inverter (as would be the case for a TCP connection), it simply sits and listens for the relevant requests and then broadcasts back with the results. Nice and simple, although (because of the inherent handicap of the UDP protocol) there's no guarantee that there will be a response. In testing I've had responses from the inverter in most cases, but it did happen where the response went missing. So, you'll have to make sure your implementation is robust enough for this case and for ensuring that all the data packets do arrive and that they are complete. There is a checksum in the inverter's protocol in order to validate the data, but it's something to take note of.

There's a few requests you can send the inverter, and based on that the inverter can respond with things like it's model/serial, it's status or confirmations whens you change settings. For the most part I'm only interested in getting the inverter status. Interestingly, there's no authentication mechanism. You fire off the request, and the inverter will answer - provided your request makes sense.

I've attached a document outlining the information I was able to decode from the information request response from the inverter. Still a few blanks there but the results are encouraging. In order to get the response below, you need to issue the following bytes to inverter IP on port 8899: 0xaa, 0x55, 0xc0, 0x7f, 0x01, 0x06, 0x00, 0x02, 0x45. The 6th byte is the request type and the 9th byte is the checksum. For the purposes of this exercise I won't go to deep into the various requests you can send to the inverter. 

Comms.thumb.png.99116d71500c1b201269e0919c8cd27e.png

Result.png.e6c56612d7062233fedd673802d08837.png

Link to post
Share on other sites
  • Replies 50
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Seeing as the 'smart' home bug has bitten me in a bad way, I've been working on integrating all the various bits of electronics in the house. One of the obvious things to get integrated is the power m

It seems various inverter models and/or firmware versions slightly differ in the response data (resp. their lengths). So I've replaced the primitive response length check with proper checksum and re

Aaaaand we have liftoff. Still a few bugs to sort out with the parsing of the power figures but success! There's a loooot more info available from the integration but I'll add that once everything has

Posted Images

Sooo, I've had a bit more time to fine tune and verify the data. I've also managed to expose a few more bits of info from the packets; it's about at a point where there's enough identified data to be useful for logging. The idea is for this parsing to be done by a python script that runs every few seconds to feed into Home Assistant. When I have a little more time to mess with it I'll get that going, running on a Pi on my network.

Below is where I'm at. Items in black I'm 99% confident of, items in Red still needs to be verified properly. Some of the AC wattage stuff isn't 100% accurate, I suspect those are VA values, not Watts, and I've not figured out if the protocol reports power factor in order to get the AC consumption figures correctly calculated.

Comms2.thumb.png.8ac68aa23199384dae71124df37c9788.png

Result2.png.935093cbda6ac4a5262c3820a2d17eb5.png

Wireshark.png.b9c48aa907bc77ae694ccd57e062c793.png

Edited by gbyleveldt
Link to post
Share on other sites

Ho.Lee.Shit! Thank you thank you thank you! I see they posted it a few days ago, saved me a LOT of pain!

EDIT: Ok, so that’s for the ET inverter, not the ES. That being said, all the heavy lifting has been done and the protocols are very similar (the ET protocol caters for 3 phase operation). Shouldn’t be too hard to add ES support in this seeing as I’ve documented that part. Still, saves a helluva lot of effort.

Edited by gbyleveldt
Link to post
Share on other sites

I am posting on that GitHub as "wishie" (same as here) and am having some issues finding some of the data that was available from SEMS... like, how much PV I have SOLD, and how much power I have BOUGHT from the grid... also, some of the states of the sensors have bad naming currently.. I hope we can all collaborate to tidy it up and make this custom component awesome!

Link to post
Share on other sites
6 hours ago, wishie said:

I am posting on that GitHub as "wishie" (same as here) and am having some issues finding some of the data that was available from SEMS... like, how much PV I have SOLD, and how much power I have BOUGHT from the grid... also, some of the states of the sensors have bad naming currently.. I hope we can all collaborate to tidy it up and make this custom component awesome!

Working on it bud. I’ve identified some missing packets that I’m testing on my side at the moment, I’ve made note of it on GitHub 

Link to post
Share on other sites

So I've been trying to get this to work but no success.  I'm a noob but learning a lot fast.

Info:

  • Goodwe ES5048 (about 6 years old, firmware updated about 18 months ago)
  • Deco M5 mesh network
  • Old original EzManage app and latest PV Master apps work perfectly (but not at the same time)
  • Phone IP is 192.168.68.106
  • Goodwe IP is 192.168.68.110
  • Home automation hub: Openhab on a RPI (but I don't think that is immediately relevant)

I downloaded Wireshark to my laptop to see what is going on on the network and it was surprisingly difficult to identify the Goodwe.  Eventually I found it and I can log into it from my laptop.  So I know the IP address is correct and it is connected.

I first scanned my phone's IP address and opened the EzManager app.  I can see it is sending packets via UDP to 255.255.255.255 using port 48899 but nothing is going back.  In the meantime the app is working and getting data from the inverter via the network.  Then I open the PVMaster app and the same happens.  I'm stumped.

322965221_Screenshotphone.thumb.png.7256117eb15c457facb94a2d1778c270.pnget some gibberish

 

Then I scan the Goodwe's IP and get some gibberish via TCP:

1523861567_ScreenshotGoodwe.thumb.png.86012b1d352e11c738bab7ddf89fe835.png

 

For the life of me I can't figure out how the app and the Goodwe are talking to each other.  Am I using Wireshark incorrectly?  Any ideas?

 

Link to post
Share on other sites

Yebo, you can’t use wire shark that way. While UDP packets are broadcast, your switch prevents those packets from flooding the network. The switch is routing the relevant packets directly between your phone and Goodwe dongle. It’s not available on the rest of the network. You’ll have to run a packet sniffer on the phone with the PV master app to get the relevant packets. That’s how I got the data initially.

Head over to the GitHub page listed above. The integration has come a long way over the last week. Unfortunately it’s directly integrated into HA at the moment. Ideally it should be done through MQTT, but I don’t think it’s on the roadmap for the moment.

‘You might want to consider trying HA. I prefer it over openhab anyway.

Edited by gbyleveldt
Link to post
Share on other sites
On 2020/09/21 at 4:42 PM, gbyleveldt said:

Aaaaand we have liftoff. Still a few bugs to sort out with the parsing of the power figures but success! There's a loooot more info available from the integration but I'll add that once everything has been nicely verified. The author really did us all a solid by doing all the Home Assistant backend stuff.

 

 

Wow

Link to post
Share on other sites

For sniffing the packets going between your phone and the Inverter, couldn't you use your PI or a linux PC? I'm sure with socat you can do port forwarding so you basically use the PI (or PC) as a proxy. i.e. You connect to the PI with your phone and it forwards the data to the inverter. Then you could use a packet sniffer to view the data.

Link to post
Share on other sites

OK... so I'm maintaining a watching brief on this one. I've not even updated the software on my Goodwe 5048ES yet, and draw almost zero data from it.

This is a hurdle I have to overcome sometime in the future (after connecting up my 5th array of panels, and finishing off the drip irrigation system in my vegetable field, and the plumbing from my well to the main water tank, and, and, and...).

I'm hoping that some clever guy is going to set himself up as the Goodwe expert and I'll be able to hire him to come do all of that for me 😂

Link to post
Share on other sites

There's no real need for udp traffic sniffing anymore, I believe we have managed to understand most of the protocol.
The python library is still being tweaked here and there, but it is close to being stable.

Although java (which Openhab is written in) is my "native" language, to be honest, I don't have much intention (or time to be more precise) to port it there at the moment.
I was selecting between Home Assistant and OpenHab, decided for the HA and I'm happy so far.

 

 

Link to post
Share on other sites
14 hours ago, gbyleveldt said:

Yebo, you can’t use wire shark that way. While UDP packets are broadcast, your switch prevents those packets from flooding the network. The switch is routing the relevant packets directly between your phone and Goodwe dongle. It’s not available on the rest of the network. You’ll have to run a packet sniffer on the phone with the PV master app to get the relevant packets. That’s how I got the data initially.

Head over to the GitHub page listed above. The integration has come a long way over the last week. Unfortunately it’s directly integrated into HA at the moment. Ideally it should be done through MQTT, but I don’t think it’s on the roadmap for the moment.

‘You might want to consider trying HA. I prefer it over openhab anyway.

Thanks, this helped a lot.  I installed Packet Capture on my phone and picked up the packets.

I would consider moving to HA if I haven't spent so much time customising my OH set up already.  I'm going to try to get the TCP/UDP binding working and see where that gets me.

Link to post
Share on other sites
12 hours ago, Stanley said:

For sniffing the packets going between your phone and the Inverter, couldn't you use your PI or a linux PC? I'm sure with socat you can do port forwarding so you basically use the PI (or PC) as a proxy. i.e. You connect to the PI with your phone and it forwards the data to the inverter. Then you could use a packet sniffer to view the data.

That sounds...difficult. I simply used an app called tPacketCapture on an Android tablet that had PVMaster installed. Those saved captures were opened in Wireshark and it gave an idea of what was happening. Nice and simple.

@mletenayactually used some hacking process to decompile the PVMaster app and managed to get much quicker results than I had in trying to decode the protocol. Be that as it may, the UDP protocol is 95% understood and documented currently, so I seriously doubt anyone will need to go the packet sniffing route again.

Link to post
Share on other sites
12 minutes ago, Niel said:

Thanks, this helped a lot.  I installed Packet Capture on my phone and picked up the packets.

I would consider moving to HA if I haven't spent so much time customising my OH set up already.  I'm going to try to get the TCP/UDP binding working and see where that gets me.

As I said above. Just go have a look at the Goodwe python stuff linked above. Pretty much all the hard work has been done already. No need to re-invent the wheel

Link to post
Share on other sites

So a bit of an update.  I ran the inverter_test Python code and received the following:

2137475642_ScreenshotUDPcomms.thumb.png.72e6360811bd6327e94fa59e6d337674.png

So even though the response could not be deciphered, at least there was a response and that is a start.

So next I installed the TCP/UDP binding in OH (https://www.openhab.org/addons/bindings/tcp1/), configured the udp.cfg file and added two items:

String GoodweData { udp="<[192.168.68.110:8899:]"}
String GoodweRequest { udp=">[192.168.68.110:8899:]"}

and to test this the following command is run every minute:

  sendCommand(GoodweRequest"aa55c07f0102000241")

I expect an update to item GoodweData and I can see in the log that this command runs, but GoodweData does not get any update and remains NULL.  Can anyone see where I'm going wrong?  If I could just crack this step, I should be able to pull all the info I need from GoodweData.

 

 

 

Link to post
Share on other sites

Hmmm. Running the inverter_test python script should give you not only the data stream but it should parse the stream as well. I forked from the current version some time ago (I'll get it back on track once the issues on my inverter is resolved), but here's basically what you should be seeing:

PS C:\Goodwe(latest)>  & 'C:\Users\gertb\AppData\Local\Microsoft\WindowsApps\python.exe' 'c:\Users\gertb\.vscode\extensions\ms-python.python-2020.9.112786\pythonFiles\lib\python\debugpy\launcher' '61877' '--' 'c:\Goodwe(latest)\inverter_test.py'
2020-10-06 08:53:15,369 __init__(623) - DEBUG: Using proactor: IocpProactor
2020-10-06 08:53:15,371 discover(481) - DEBUG: Probing inverter at 192.168.1.45:8899
2020-10-06 08:53:15,372 connection_made(319) - DEBUG: Send: 'aa55c07f0102000241'    
2020-10-06 08:53:16,040 datagram_received(330) - DEBUG: Received: 'aa557fc001824d3138313843475735303438442d45532331302020202020202020202020202039353034384553553230345730303836333630303431302d30343032352d31380e3431302d30323033342d31320611ec'
2020-10-06 08:53:16,042 discover(495) - DEBUG: Detected ES inverter GW5048D-ES, S/N:95048ESU204W0086
2020-10-06 08:53:16,047 __init__(623) - DEBUG: Using proactor: IocpProactor
2020-10-06 08:53:16,051 connection_made(319) - DEBUG: Send: 'aa55c07f0106000245'
2020-10-06 08:53:16,900 datagram_received(330) - DEBUG: Received: 'aa557fc001868c0c50000b020c93000a0202140001005000dc0000000a00640000640064640300000108dc001e0150138a0108dc002d0001138a0102012a0000000000002bab000005b4000e00650000447702520100300203a600010000fdb2000800200000140a0608351100000000000000000000000000000000000000000000000000000c4800000aff0200000000000012aa'
vpv1: PV1 Voltage = 315.2 V
ipv1: PV1 Current = 1.1 A
ppv1: PV1 Power = 347 W
pv1_mode: PV1 Mode = 2
pv1_mode_label: PV1 Mode = Working,PV has a power output
vpv2: PV2 Voltage = 321.9 V
ipv2: PV2 Current = 1.0 A
ppv2: PV2 Power = 322 W
pv2_mode: PV2 Mode = 2
pv2_mode_label: PV2 Mode = Working,PV has a power output
ppv: PV Power = 669 W
vbattery1: Battery Voltage = 53.2 V
battery_temperature: Battery Temperature = 22.0 C
ibattery1: Battery Current = -0.0 A
pbattery1: Battery Power = 0 W
battery_charge_limit: Battery Charge Limit = 10 A
battery_discharge_limit: Battery Discharge Limit = 100 A
battery_status: Battery Status = 0
battery_soc: Battery State of Charge = 100 %
battery_soh: Battery State of Health = 100 %
battery_mode: Battery Mode = 3
battery_mode_label: Battery Mode = Charge
battery_warning: Battery Warning = 0
meter_status: Meter status = 1
vgrid: On-grid Voltage = 226.8 V
igrid: On-grid Current = 3.0 A
pgrid: On-grid Export Power = -336 W
fgrid: On-grid Frequency = 50.02 Hz
grid_mode: Work Mode = 1 
grid_mode_label: Work Mode = The inverter is generating
vload: Back-up Voltage = 226.8 V
iload: Back-up Current = 4.5 A
pload: On-grid Power = 1 W
fload: Back-up Frequency = 50.02 Hz
load_mode: Load Mode = 1
load_mode_label: Load Mode = Inverter and the load is disconnected
work_mode: Energy Mode = 2
work_mode_label: Energy Mode = Normal(On-Grid)
temperature: Inverter Temperature = 29.8 C
error_codes: Error Codes = 0
e_total: Total PV Generation = 1117.9 kWh
h_total: Hours Total = 1460
e_day: Today's PV Generation = 1.4 kWh
e_load_day: Today's Load Consumption = 10.1 kWh
e_load_total: Total Load = 1752.7 kW
total_power: Total Power = 594 W
grid_in_out: On-grid Mode = 2
grid_in_out_label: On-grid Mode = Inverter gets power from grid
pback_up: Back-up Power = 934 W
plant_power: Plant Power = 935 W
diagnose_result: Diag Status = 524320
house_consumption: House Comsumption = 1005 W
PS C:\Goodwe(latest)>

If you not getting this, there's no point going forward to be honest...

 

EDIT: You're trying to query the inverter info with the command you sending (aa55c07f0102000241). That will only give the inverter info, not the status. I would seriously look at the Goodwe component as well. Although it's meant for Home Assistant, there should be a wealth of info there for you to convert to Openhab. Provided that you can run Python on Openhab of course. If you want to run Java, you're on your own...

EDIT2: I also see that you're not getting the complete UDP packet in your query, so the inverter_test script doesn't complete. Like I said above, you really need to get this part working correctly before you try and proceed. How are you connecting to the inverter? Maybe give us some idea of your setup?

Edited by gbyleveldt
Link to post
Share on other sites

We have a Deco M5 mesh wifi network (192.168.68.xxx) to which the Goodwe, phones, laptop etc is connected.  The Raspberry Pi is connected to the primary router (192.168.0.100) which also provides the internet to the Deco mesh.  All the SmartHome controllers (pool pump, weather station, water tank, garage lights) are on the Deco wifi network and communicate seamlessly with OH via MQTT.

The Goodwe is a 6 year old ES model (S/N 35xxx which may explain the different data structure?) and we connect using an app on my phone.  Both PVMaster and the predecessor EZManage connect with no problem.  Interestingly, when I change the (85, 86) to (73, 74) and (142, 149) to (104, 111) in the goodwe_inverter.py file, it runs perfectly.

Regarding the command being sent and info received by inverter_test, it looks the same as what the app does:

1183763875_Screenshot_20201006-200925_PacketCapture.thumb.jpg.0e22bbc339d9de4c47bfccaa1405d327.jpg

 

So now I guess my problem now is figuring out why the Openhab code does not get information from the Goodwe...

 

 

Link to post
Share on other sites

Sure, here you go:

 

2020-10-07 20:35:40,796 discover(449) - DEBUG: Probing ES inverter at 192.168.68.110:8899
2020-10-07 20:35:40,796 connection_made(323) - DEBUG: Send: 'aa55c07f0102000241'
2020-10-07 20:35:41,696 datagram_received(334) - DEBUG: Received: 'aa557fc00182403037582020475735303438442d4553ffffffffffffffffffffffffffffffff33353034384553553135333030303331333630303431302d30303436382d30370e1d15'
2020-10-07 20:35:41,745 discover(451) - DEBUG: Detected ES protocol inverter GW5048D-ES, S/N:35048ESU15300031
2020-10-07 20:35:41,745 __init__(623) - DEBUG: Using proactor: IocpProactor
2020-10-07 20:35:41,755 connection_made(323) - DEBUG: Send: 'aa55c07f0106000245'
2020-10-07 20:35:42,245 datagram_received(334) - DEBUG: Received: 'aa557fc001865f0000000100000000000001ee0000005000f6005f004a004a00003c003c620200000108eb0017000213770108eb001a009c1377010200e500000000000381000000b4fc007b00890004c88801890100300002170001000000050700004000001189'
vpv1: PV1 Voltage = 0.0 V
ipv1: PV1 Current = 0.1 A
ppv1: PV1 Power = 0 W
pv1_mode: PV1 Mode = 0 
pv1_mode_label: PV1 Mode = Disconnect the inverter and PV panels 
vpv2: PV2 Voltage = 0.0 V
ipv2: PV2 Current = 0.0 A
ppv2: PV2 Power = 0 W
pv2_mode: PV2 Mode = 0 
pv2_mode_label: PV2 Mode = Disconnect the inverter and PV panels 
ppv: PV Power = 0 W
vbattery1: Battery Voltage = 49.4 V
battery_temperature: Battery Temperature = 24.6 C
ibattery1: Battery Current = 9.5 A
pbattery1: Battery Power = 469 W
battery_charge_limit: Battery Charge Limit = 74 A
battery_discharge_limit: Battery Discharge Limit = 74 A
battery_status: Battery Status = 0 
battery_soc: Battery State of Charge = 60 %
battery_soh: Battery State of Health = 98 %
battery_mode: Battery Mode = 2 
battery_mode_label: Battery Mode = Discharge 
battery_warning: Battery Warning = 0 
meter_status: Meter status = 1 
vgrid: On-grid Voltage = 228.3 V
igrid: On-grid Current = 2.3 A
pgrid: On-grid Power (EzMeter) = 2 W
fgrid: On-grid Frequency = 49.83 Hz
grid_mode: Work Mode = 1 
grid_mode_label: Work Mode = The inverter is generating 
vload: Back-up Voltage = 228.3 V
iload: Back-up Current = 2.6 A
pload: On-grid Power = 156 W
fload: Back-up Frequency = 49.83 Hz
load_mode: Load Mode = 1 
load_mode_label: Load Mode = Inverter and the load is disconnected 
work_mode: Energy Mode = 2 
work_mode_label: Energy Mode = Normal(On-Grid) 
temperature: Inverter Temperature = 22.9 C
error_codes: Error Codes = 0 
e_total: Total PV Generation = 16409.6 kWh
h_total: Hours Total = 46332 
e_day: Today's PV Generation = 12.3 kWh
e_load_day: Today's Load Consumption = 13.7 kWh
e_load_total: Total Load = 24794.4 kW
total_power: Total Power = 393 W
grid_in_out: On-grid Mode = 0 
grid_in_out_label: On-grid Mode = Inverter neither send power to grid,nor get power from grid 
active_power: Active Power = 2 W
pback_up: Back-up Power = 535 W
plant_power: Plant Power = 691 W
diagnose_result: Diag Status = 117440576 
house_consumption: House Comsumption = 467 W
>>> 

 

It (unsuccessfully) looks for a ET series inverter before this code but I assume that is not relevant.

 

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...