PowerForum Solar Data Collection - using a Raspberry Pi 3 :-)

[___]

Man, I'm so excited about how this is progressing. Tonight I finally took the home-made can board for a test drive, and it worked first time. This is essentially the same as the shield, except for one important difference: The shield uses an mcp2551 as can transceiver, while this one uses an ISO1050 with galvanic isolation. It doubles the cost of the chip, but I figure it is worth it. This is why there is an extra 5V regulator on the board: The input side is actually powered from the can bus, while the can controller is powered from the 5V line on the arduino.

[Guest]

AWESOME Plonkster!!!

[Guest]

Nice.

[___]

I did a bit more work on the server side of my project. I've removed the development dependency on nodejs, or at least, it's now optional and you only need to install those tools if you intend hacking on angular and additional libraries. The source ships with an uglified and compressed JS library containing everything you need to run it.

That means the only two things you need now is an MQTT broker, python, and virtualenv. Any takers yet, or do I need to write instructions for Windows? :-P

[Guest]

Plonkster ... lets do as out brothers do. Not chat Zupta, but rather Windows ... Windows ... Windows ... :-) 

Okaaaay, compromise, whatever native OS Rpi has, other than Windows?

I simply cannot pull my developers off what they are currently busy with, but we will contribute one of the days, and share back.

[___]

Lol :-) I'm almost feeling a little guilty, barging in here with an OS/language/inverter combo that's apparently a little unsupported, completely under my own steam, reporting to the crickets. Insert Zuma laugh here. But I'm perfectly happy writing instructions for windows if it might help. I'm doing the old open source thing, releasing it in the hope that it might be useful to someone, and that someone else is going to write the Axpert interface... :-P

Edit: The next step is going to be to make a distribution for the Rpi. Believe it or not, the biggest current obstacle is actually that I need to go to a shop and buy an SD-card... and I don't actually get around to doing it. Once I have the Rpi distro going, in theory you can just burn the image to an SD-card on your side and check it out.

[Guest]

26 minutes ago, plonkster said:

Edit: The next step is going to be to make a distribution for the Rpi. Believe it or not, the biggest current obstacle is actually that I need to go to a shop and buy an SD-card... and I don't actually get around to doing it. Once I have the Rpi distro going, in theory you can just burn the image to an SD-card on your side and check it out.

Send me your address, make and model of SD card, and I will have it delivered. 

[Guest]

7 hours ago, plonkster said:

Lol :-) I'm almost feeling a little guilty, barging in here with an OS/language/inverter combo that's apparently a little unsupported, completely under my own steam, reporting to the crickets. Insert Zuma laugh here. But I'm perfectly happy writing instructions for windows if it might help. I'm doing the old open source thing, releasing it in the hope that it might be useful to someone, and that someone else is going to write the Axpert interface... :-P

Plonkster, we are the odd ones here yes, but once we have our stuff sorted, the rest is easy as Pi (pun intended). 

All that is needed is our to be read (Victron inverters / MPPT controllers and my Morningstar) by a Rpi, then stored per device output, in a comma delimited csv (per device) on said Rpi.

Then a very clever plan can be set in motion that me and someone else came up with.

Then the other inverters can be added that the guys are asking for, using the same logic.

Going the web route can then follow based on discussions that will follow after the above has been bedded down.

[___]

I added instructions for a windows build today :-) I found an old SD card in an old phone, so sorted there too. And I already have the Rasbian image and even cross compilers and a kernel tree... Because I do this for work. But I also need to build a bare bones arduino with the spare atmega328 I have... because that old Duemillanove is too good for this :-) Slow but steady.

Sent from my GT-I9195 using Tapatalk

[Gnome]

Just a quick update, I know I said I would post my source code.

I'm almost done. I've added some things that I realized I need while playing with Web User Interface (WUI) development.

Needed a way to bucket my data. I poll for data every 2 seconds using serial and store it in a database but I wanted a way to aggregate data points for a duration (eg. 1 minute, 5 minutes, hourly based on percentile or average).

Anyway, means I had to make a bit of change, but I'm just doing some final testing and will release it soon.

I'll start out with Linux/Mac install scripts (I develop on a Mac and run on RPi which is Linux). Windows isn't really much more difficult but I'll get this out there first.

[Guest]

BTW plonky you know you get a can bus shield for the Arduino

http://www.communica.co.za/Catalog/Details/P1865739722

 

 

[___]

1 hour ago, jdp said:

BTW plonky you know you get a can bus shield for the Arduino

http://www.communica.co.za/Catalog/Details/P1865739722

 

 

You can get them all over the place :-) They are all pretty similar, they use the MCP2515 controller, and the MCP2551 transceiver. Pretty much the only difference is which pin is used for Chip Select, and it's not difficult to see why: You can't choose which pins are used for SPI, that is fixed :-) Anyway, I do have a shield. I got it from Ebay. It took a month and a day to get here (which isn't bad for the SAPO) and I used it for the initial version of the code. But I really wanted the galvanic isolation, and though I briefly considered retrofitting a transceiver (because they too are all pretty similar) there would just be no way to power the input side of the new chip without serious hacking :-)

[___]

So, another boring progress report. Found an old 4GB SD-card in a 6-year old Nokia phone... but the Raspbian image was about a hundred megabyte larger than the sdcard, so it took some loopback acrobatics to resize the file system. Then once I had the Rpi booting, I found that the default image (around 3GB of the 4GB image size used) is just packed with stuff that won't be used in this application. So I stripped it all out, including X (can always add it back if I need it), and got it down to 870MB. That ought to compess to about 500MB or so, which is quite manageable as a download by today's standards.

[___]

And yet more progress. I now have the Rpi distribution working correctly, with the software actually running on it (and starting automatically at boot -- I really hate systemd sometimes). Tonight I hacked together the first script that  posts actual mk2 data onto MQTT, so the attached image is an actual picture of my present load (night time).

One issue I can see coming: Both my mk2 and the arduino registers as serial usb devices, so depending on the order they are detected/plugged in, one is going to be /dev/ttyUSB0 and the other /dev/ttyUSB1. I can employ some udev hackery to name them, but the mk2 comes up as a generic ftdi device and so do the arduino! I haven't checked usb ids yet, but even if they differ, there is a good chance that users might end up with two pieces of hardware that looks exactly the same to the OS.

This will likely become sort of useful by next week.

[Guest]

Plonkster, I presume you are reading from your inverter, MPPT and BMV?

Or that all would be read via the Rpi in the end?

[___]

12 minutes ago, The Terrible Triplett said:

Plonkster, I presume you are reading from your inverter, MPPT and BMV?

Or that all would be read via the Rpi in the end?

I don't have a BMV. I really should have one... I know... but I don't :-) So the plan is just reading from the inverter (using mk2) and reading from the MPPT (using the canbus->serial project), then doing some newtonian integration that approaches ∫Pdt, and that will give me KWh in and out.

I don't have the MPPT hooked up yet, and the mk2 hookup was extremely hacky, with half the code not even running on the Pi yet :-)

Edit: I do plan on supporting readings from the BMV at some point. They will simply represent absolute values rather than a delta. Easy enough to code around.

[___]

Also, I left avahi on the Rpi, so this thing supports multicast dns. That's how come raspberrypi.local works. Not sure if windows supports mdns. Mac does... I think they call the service Bonjour.

[___]

Something visible that's actually working. I have the Pi set up and reading data from the inverter, and it is available on the web, at least for now while there are no critical administrable stuff there.

http://ispburger.ddns.net:6543/

The charge numbers are simulated. The load numbers are real. I still need to write the ve-direct mppt interface, and build a bare bones arduino to complete that half.

[Guest]

Plonkster, check this out: http://powerforum.co.za/topic/504-victron-battery-monitor-bmv-700/#comment-6074

You said you really should have one. 

[___]

1 hour ago, The Terrible Triplett said:

Plonkster, check this out: http://powerforum.co.za/topic/504-victron-battery-monitor-bmv-700/#comment-6074

You said you really should have one. 

Saw it, but he's down in the Eastern Cape iirc? :-) Besides... I'm as arm as a kerkmuis here... I just finished painting the house and the surrounding wall, on a corner plot! :-)

[Guest]

Eastern Cape, China, Europe ... couriers are all over them shows, dime a dozen. 

But I feel your pain re. house maintenance. Needs versus wants ... yet again.

[SilverNodashi]

I would like to join this project, if possible. There's a Raspberry Pi in the drawer, along with some Linkit One devices, some arduino's, and a bunch of other electronics. HTML, PHP and MySQL isn't foreign to me and I have access to a bunch of Linux web servers (my hosting company, essentially). My current inverter arsenal consists of some 3Kva Axpers, 5KVA Axperts, 10Kw InfiniSolar (though I don't have 3phase power), a 5Kw powerstar inverter, a steca 2020 MPPT charge controller, and another 20A MPPT charge controller. I can also get some other brands inverters, MPPT charge controllers, battery monitors, etc. 

 

IMO, a Rasberry Pi, or any other Iot device would be a very good option to use. An old PC / laptop would be too big / power hungry for this kind of project. A small dev board of sorts could be permanently installed close to the device(s) to be monitored and could either use network / 3G to upload data to a central DB, or alternatively one could access it via the internet to get stats as well. the amount of data to be uploaded, or viewed wouldn't be that much. 

[Guest]

 

4 minutes ago, SilverNodashi said:

IMO, a Rasberry Pi, or any other Iot device would be a very good option to use. An old PC / laptop would be too big / power hungry for this kind of project.

Don't let Viper hear you say this, he will just want you to use a Tablet.

Sorry VIPER, I could NOT stop myself!!! 

[Guest]

SilverNodashi, you are more than welcome. Plonkster is on the Victron's, I will do the Morningstar in due course, and one two others also showed interest.

Have weird feeling we are going to have a few versions, using different approaches.

For me the core issue was and always has been, reading as many devices as possible, using a Rpi.

Now, I have seen in my career you get min 2 types of developers. No 1: The hard core developer who does not care about the look and feel, as long as it bloody well works right.

No 2 is the more patient creative developer who make software sommer looking really professional.

 

Which bring me to a point I think must be mentioned again.

If we can get the data read from as many devices as possible with their related data stored in file named i.e. DEVICE_NAME_MODEL.CSV file stored in blocks of 24 hours per file on the Rpi, more software suppliers can then read that data and use it in their software. It is like splitting the hardcore stuff from the nice to have front ends.

I am pretty sure we will get quite a few takers of said Rpi software if more guys can integrate that with their own software.

[viper_za]

6 hours ago, The Terrible Triplett said:

 

Don't let Viper hear you say this, he will just want you to use a Tablet.

Sorry VIPER, I could NOT stop myself!!! 

I just like having a screen all the time