Python code to log to pvoutput.org from venus/ccgx

[___]

Hi all,

This is something I've wanted to do for a long time. This is very raw still, probably needs a lot more work, and of course you need a CCGX, Venus-GX, or you need to run Venus on your Raspberry Pi. Nevertheless, here it is.

There are some assumptions in this, such as that you don't have PV-inverters anywhere, and that you have a grid meter. This should however work in off-grid settings as well.

For now, this is my little site, some mistakes were made in the beginning, but it should even out eventually :-)

 

[___]

What's actually funny about this is that power going into the batteries is being logged as "export" and taking power from the batteries are show as import. Which is probably right on some level. How is this handled by other software?

Looking at their documentation, it doesn't look as if this is really designed with battery/self-consumption rigs in mind.

[___]

So I noticed that it's using the wrong timezone. That's because the CCGX uses UTC as its system timezone, and the actual timezone is stored in local settings. So I added a few lines of code to correctly set the timezone environment variable.

The code now has a proper repo here.

[Mark]

@plonkster I don't have an energy meter and was wondering if you could advise.  I get the following error...

Thanks

Mark

INFO:__main__:Found solarchargers at com.victronenergy.solarcharger.ttyS6, com.victronenergy.solarcharger.ttyS7
INFO:__main__:Found grid meters at
INFO:__main__:Found vebus at dbus.Array([], signature=dbus.Signature('i'), variant_level=1)
Traceback (most recent call last):
  File "dbus-pvoutput.py", line 148, in <module>
    main()
  File "dbus-pvoutput.py", line 104, in main
    track(conn, consumers, vebus, "/Energy/InverterToAcOut", "c1")
  File "dbus-pvoutput.py", line 58, in track
    state[target] = value = unwrap_dbus_value(query(conn, service, path))
  File "dbus-pvoutput.py", line 54, in query
    return conn.call_blocking(service, path, None, "GetValue", '', [])
  File "/usr/lib/python2.7/site-packages/dbus/connection.py", line 638, in call_blocking
    method=method)
ValueError: Invalid bus name 'dbus.Array([], signature=dbus.Signature('i'), variant_level=1)': contains invalid character '('

[___]

1 hour ago, Mark said:

INFO:__main__:Found vebus at dbus.Array([], signature=dbus.Signature('i'), variant_level=1)

There's your problem. Nothing to do with not having a meter, the script can work with zero meters.

The above usually means the Multi is off or not found for whatever reason. On line 92 you'll see it query systemcalc to get the vebus service, and this should respond with something like com.victronenergy.vebus.ttyO5, but instead it responds with an empty array, which in Venus land is used to indicate that the value is undefined.

I just remembered again tonight why it had to be so unwieldy... its because .system does not keep track of the energy values, only power, so I had no choice. That... and it was written after lunch on boxing day

 

[Mark]

@plonkster I don't have an energy meter and was wondering if you could advise.  I get the following error...

Th

8 hours ago, plonkster said:

There's your problem. Nothing to do with not having a meter, the script can work with zero meters.

The above usually means the Multi is off or not found for whatever reason. On line 92 you'll see it query systemcalc to get the vebus service, and this should respond with something like com.victronenergy.vebus.ttyO5, but instead it responds with an empty array, which in Venus land is used to indicate that the value is undefined.

I just remembered again tonight why it had to be so unwieldy... its because .system does not keep track of the energy values, only power, so I had no choice. That... and it was written after lunch on boxing day

 

Thanks for the reply.

Is there a chance I can "give it a value" in some way to get over this issue?   or get rid of this check

Regards

Mark

[___]

1 hour ago, Mark said:

Is there a chance I can "give it a value" in some way to get over this issue?   or get rid of this check

It's strange that that call fails. If you run manually on the commandline, for example on a cerbo:

root@einstein:~# dbus -y com.victronenergy.system /VebusService GetValue
'com.victronenergy.vebus.ttyS4'

That is the service name of the vebus, and that is the value you should get in the vebus variable. So you could skip that and just hardcode it if you know where yours is, eg:

# Instead of...
vebus = str(query(conn, "com.victronenergy.system", "/VebusService"))

# do
vebus = 'com.victronenergy.vebus.ttyS4'

But that still does not explain why systemcalc is not seeing your Multi...

On a CCGX, the last bit is ttyO1, on a Venus-GX and an Octo-GX it is ttyO5, and on a Cerbo it is ttyS4. On a Raspberry Pi it is ttyUSBsomething, depending on order it was detected.

Edited July 30, 2020 by plonkster

[Mark]

Hi @plonkster

Thanks for reply... to clarify and I see above I forgot to mention I have no multi connected (sorry for misinfo).

2x MPPT's and a battery (Pylon) on the BMS CAN.

A Cerbo received yesterday and I would like to capture MPPT production to PVO and EMONCMS.

Using your PVO code and Paul's EMONCMS modification of your code...

I have tried the hard coding of the above: 

vebus = 'com.victronenergy.vebus.ttyS4'

But I guess it still needs the multi to respond... 

Regards

Mark

 

[___]

Comment out these two lines:

    track(conn, consumers, vebus, "/Energy/InverterToAcOut", "c1")
    track(conn, consumers, vebus, "/Energy/InverterToAcIn1", "c2")

This will cause the consumers dictionary to be empty, cause you also have no meter. energy_consumed will then be zero in the _upload function, so it will end up logging only the generation.

The script was never really improved after the initial version, so it has always required the Multi to be there to work.

[Mark]

Making some progress... I think...   (full disclosure... I know nothing about python )

root@einstein:/data# python dbus-pvoutput.py
INFO:__main__:Found solarchargers at com.victronenergy.solarcharger.ttyS6, com.victronenergy.solarcharger.ttyS7
INFO:__main__:Found grid meters at
INFO:__main__:Found vebus at com.victronenergy.vebus.ttyS4
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 734, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 465, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 329, in getMessage
    msg = msg % self.args
TypeError: float argument required, not NoneType
Logged from file dbus-pvoutput.py, line 117
Traceback (most recent call last):
  File "dbus-pvoutput.py", line 148, in <module>
    main()
  File "dbus-pvoutput.py", line 141, in main
    _upload()
  File "dbus-pvoutput.py", line 127, in _upload
    "v4": int(stats.pc),
TypeError: int() argument must be a string or a number, not 'NoneType'
root@einstein:/data#

[___]

7 hours ago, Mark said:

    "v4": int(stats.pc),

Again, that is maintained by the line that tracks the energy value on systemcalc, but that is unset cause you have no Multi. So just replace stats.pc with a zero... and comment out any places it is tracked...

[Mark]

Thanks for the help @plonkster - much appreciated.  I have a feed working and hope to refine a improve it a bit!

[Kalito]

On 2020/07/31 at 2:53 PM, Mark said:

Thanks for the help @plonkster - much appreciated.  I have a feed working and hope to refine a improve it a bit!

@Mark I would love to export to PVO too. I can SSH into the Cerbo. Where is the script you use and how do you copy it to the Cerbo? Complete newbie so some patience.

[Mark]

9 hours ago, Kalito said:

@Mark I would love to export to PVO too. I can SSH into the Cerbo. Where is the script you use and how do you copy it to the Cerbo? Complete newbie so some patience.

Plonkster has it here in GitHub.

"This is something I've wanted to do for a long time. This is very raw still, probably needs a lot more work, and of course you need a CCGX, Venus-GX, or you need to run Venus on your Raspberry Pi. Nevertheless, here it is."

 

Enjoy!

[Kalito]

11 hours ago, Mark said:

Plonkster has it here in GitHub.

"This is something I've wanted to do for a long time. This is very raw still, probably needs a lot more work, and of course you need a CCGX, Venus-GX, or you need to run Venus on your Raspberry Pi. Nevertheless, here it is."

 

Enjoy!

Thanks Mark. I have added my PVO details in line 16 and 17. where on the CerboGX do I copy the file to? Any command line syntax example?

[Mark]

Hi

I created a folder on my CerboGX and then run the script by doing the following.

>   nohup /usr/bin/python /data/dbus-pvoutput.py & >/data/pvoutput_output.log

Please note - I have little clue about how or why etc.  It has worked and I don't fiddle...
 

 

[Kalito]

50 minutes ago, Mark said:

Hi

I created a folder on my CerboGX and then run the script by doing the following.

>   nohup /usr/bin/python /data/dbus-pvoutput.py & >/data/pvoutput_output.log

Please note - I have little clue about how or why etc.  It has worked and I don't fiddle...
 

 

This is the classic case of two blind men leading each other - Learning. The fact that you are not yet in a ditch is a good sign-lol. I managed to copy dbus-pvoutput.py to the data folder of my CerboGX. I was thinking what next till I read your message. I think I have an idea what that script does. The nohup command is "NO HUNGUP" after you exit then you are telling the script to log into the log file. Smart. Will try this thanx

[Kalito]

@Mark@___ I am getting errors now so progress: Line 103. Is there a version with cleaned out issues to save me time? I see you had to make some changes. Any chance you share your working code?

 

[Mark]

5 hours ago, Kalito said:

@Mark@___ I am getting errors now so progress: Line 103. Is there a version with cleaned out issues to save me time? I see you had to make some changes. Any chance you share your working code?

 

Hi

See mine attached.

dbus-pvoutput new.py

[Kalito]

On 2021/05/03 at 6:33 AM, Mark said:

Hi

See mine attached.

dbus-pvoutput new.py 4.47 kB · 1 download

Thanx lots. Lemme try it out

[Kalito]

@Mark you are a . Your script worked:

root@einstein:~# nohup python /data/dbus-pvoutput.py > /data/pvoutput.log &
[2] 20631
root@einstein:~# cat /data/pvoutput.log
INFO:__main__:Found solarchargers at com.victronenergy.solarcharger.ttyS5
INFO:__main__:Found grid meters at
INFO:__main__:Found vebus at com.victronenergy.vebus.ttyS4
INFO:__main__:EG: 2278.39, EC: 42.56, PG: 0.00, PC: 618.00
 

[Mark]

1 hour ago, Kalito said:

@Mark you are a . Your script worked:

root@einstein:~# nohup python /data/dbus-pvoutput.py > /data/pvoutput.log &
[2] 20631
root@einstein:~# cat /data/pvoutput.log
INFO:__main__:Found solarchargers at com.victronenergy.solarcharger.ttyS5
INFO:__main__:Found grid meters at
INFO:__main__:Found vebus at com.victronenergy.vebus.ttyS4
INFO:__main__:EG: 2278.39, EC: 42.56, PG: 0.00, PC: 618.00
 

Thanks to the author on github

[Kalito]

On 2021/05/04 at 8:55 PM, Mark said:

Thanks to the author on github

Question though: What happens when the GX device restarts?

[Mark]

12 hours ago, Kalito said:

Question though: What happens when the GX device restarts?

You have to restart the script or do it on reboot...