DEVELOPMENT SUSPENDED - KLEVA Solar Monitor Beta v0.2.1604.40

[KLEVA]

Please Note: As of 2018-07-10 Development and support of this product has ceased. The code below and and comments remain, but only for review and implementation into your own projects. This code is completely free of any license or restriction of any kind (other that the KLEVA name and Logos which are under copyright and Trademark laws)

 

Description:
KLEVA Solar Monitor is NOT a replacement for the other perfectly usable products out there for your home use. KLEVA Solar Monitor was born out of a need to log data to an internal SQL server, and to automate a scheduling system that could remain running with custom commands. You are expected to have at least basic database configuration knowledge to be able to use this product. Basically, KLEVA Solar Monitor should actually be one of your last resorts in getting data from your system.

I wanted something that could be run independantly for multiple devices and still use the same database. I wanted something that could run on practically any MS Windows version released in the last 15 years and I also wanted something that was flexible to be able to be added onto using a base framework, and to be able to give other developers a tool to communicate with their devices, without having to master all the communication protocols, etc (as I had to), just by including this open source Core file. (All source is available on request, and will be posted somewhere for future generations )

KLEVASolarMonitor will remain open/free, but support at a later stage (should this suddenly by magic become a hugely successful product), may need to be charged for by my current employers if it occupies my working/business hours.

Installation:
1. Extract the contents of the zip file to any directory on your hard drive
2. Read the currently slightly disorganised KLEVASolarMonitor.pdf file in the Docs folder for details
3. Contact me below regarding queries/support/issues or via the email address in that file

Other:
Also attached, but not really part of the KLEVA Solar Monitor per-se, is a sample of how I  occassionaly read the data into some nice little pretty graphs (courtesy of Google Graphs). This can always be seen live at http://www.kleva.co.za/KLEVASolarMonitor/All.html (Edited: 2018-07-10 Site no longer available)

KLEVASolarMonitor.zip

KSMWebsite.zip

KLEVASolarMonitor.pdf

Edited July 12, 2018 by KLEVA
typo in link

[viceroy]

Will give this a go, maybe this weekend if I can free up some time.

Being a SQL developer and data analyst myself, this could be what I'm looking for.

[Guest]

On 5/11/2016 at 6:05 PM, KLEVA said:

I wanted something that could be run independantly for multiple devices and still use the same database. I wanted something that could run on practically any MS Windows version released in the last 15 years and I also wanted something that was flexible to be able to be added onto using a base framework, and to be able to give other developers a tool to communicate with their devices, without having to master all the communication protocols, etc (as I had to), just by including this open source Core file.

This is what I read yesterday and got confused with all of us doing our own thing, where I said we all sing from the same hymn sheet as what I sang when the Pi3b thread started. 

And I am laughing at myself for I broke away from the rest as they broke away from each other ... and now more people are going into even more directions.

IDEAL would have been a reader for Power Forum users ... reading all devices users have ... but that idea seems to have been a non-starter.

For in my mind if you read ALL the data from ALL devices used by forum users, one for Linux and one for Windows, giving the users the ability to send the data read to a database of their choice, THAT would have stopped a lot of the same development.

We should all focus on the front ends and what we can do with the data, than every 2nd person who knows development developing yet another reader for the same device as what the previous 10 have done, all with "limited" front end abilities.

The front end and what you DO with the data, that is where it gets complicated, time consuming and costly, as many a developer has said.

So we all "waste" our time on our own readers instead of really clever software that adds value.

Wish @Energy could look into this for we need a leader. Have something that will benefit all Forum Members, irrespective of the device they chose to use.

And @viceroy, SolWEB is SQL, waar jy kan toor met SP's? What am I missing? 

 

@KLEVA  and @Manie - why are we all pulling in different directions?

MY reason was I have 5 devices that must be read, so no difference to sommer add a 6th one like Voltronic and then a case of, ag jy weet, whilst we are at it, make it 7, add Inifini's.

And the Windows platform logic?
Most if not all new end users probably have Windows and are not going to go out and buy a Pi nor setup a Linux Pc at home. So Windows for them is less intimidating, easier and no-one is "put off" with a Linux / Pi learning curve.

And the inevitable support when things do not work, see how easy @jdp sorted support calls for AICC, as did @edmundp on SolarMon. Both Windows based.

Moving on, When the end user smiles and wants to curb the readers consumption, after seeing what the batts have to dish out, THEN you say:No problem, Here, have a Pi.

Then the end user WANTS to change. Been there, done that. But it started with an easy to use Windows program.

Pi and Linux are predominately for the high end tech users, or new users who wants to, all whom are passionate about Pi's and Linux and free software ... the rest of us, the majority, really are not interested. Plug and Play. Take me, when I got the Pi from @plonkster, 1st thing I did was make it accessible using Windows Remote Desktop. Why? Because. I lost interest after spending 2 days learning to do things on the Pi. What was 2 seconds for Plonkster, was hours for me. AND I let the smoke out of a SD card to boot.

That is my logic for going Windows first.

So let me ask you all bluntly: Are we going to keep on pulling all over, or are we going to band together and devise a Power Forum device reader, or not?

[viper_za]

17 minutes ago, The Terrible Triplett said:

@KLEVA  and @Manie - why are we all pulling in different directions?

Dink hulle goed is heeltemal free

[Guest]

Just now, viper_za said:

Dink hulle goed is heeltemal free

VERY good point.

And JDP just answered why it will not last. 

[Guest]

1 minute ago, jdp said:

TTT I said what you are now saying 3 months ago and you gave some long explanation as to why you are doing what you are doing. As so many have done what they have done I lost interest in the project. I have moved on. Too much work to keep everybody happy for very little reward. It was a fun project but I am so busy with real work that will put bread on the table.

Let me put this in perspective from the other side.

JDP, you forget 3 things:
1)) I offered to pay you to include Victron and Morningstar. If you did that I would be on AICC today.
2)) Being unsuccessful on point 1, I offered to buy AICC from you, or buy in and add the above. You said no.
3)) Then you PM'ed the one day to say we must talk business ... only to give AICC out for free seconds later.

So from where I sit, I REALLY tried boet. I really did.

 

[Guest]

All good point here. Why are we all develop different software for the same devices ? Well all that i can say is that it means that the current software lack something the the user want. AICC does not have usb support. Ed software in the beginning also did not have usb support. Now with my inverter i only have usb, no serial connection. I had to develop something for myself.

To get to the Pi , what do we need ? I wanted the Pi to do all the thing with an interface that AICC do and it must be simple to install it, just like AICC works, all the source in one folder and the user just click on the app and it runs. Now the question , Do we have something like this where the users can download it and install it on the Pi ? If there is then we must get links to these software.  

[___]

1 hour ago, The Terrible Triplett said:

So we all "waste" our time on our own readers instead of really clever software that adds value.

I don't know. I was having a discusion with someone at Victron about how many times they've implemented the ve-direct protocol in some way or another (and now we're not even counting the times it was reimplemented by others on the internet), but as much as we try to stick to DRY (Don't Repeat Yourself), the truth is that the ve-direct protocol is a simple protocol that a good developer can actually reimplement in under an hour... so what the heck are we worried about?

Your initial idea was a non-started, and I think I even said so back in the day. Ain't no way I'm coding in C-hash :-P

[Guest]

Here is my software screens . This was things i wanted to have. Life trends , History trends on all values and different time switching

 

[Guest]

AWESOME MANIE!!!

Plonkster ... Matthijs Vader ... ja, just had a lekker chat with him also on another matter.
Non-starter ... explain some more?

JDP, agreed. I really tried to join forces so I cannot be put in the group of doing my own thing just because I can. I had the resources to make it work faster and better. 

Therein me now taking up the torch for working together ... BUT ... if  Watchpower / Solarpower is going to improve ... then as you, I may lose interest.

Moving on. 

 

Manie, SolarMon can read USB, for I understand why JDP had to call it a day. I really do.

And the Pi idea is where I started months ago for I really did not want a Pc or tablet to read the data 24/7 ... and I wanted to see my system on my phone. None of this Teamviewer or some such. 

 

Let me go out on a limb here ... . what happened to negotiations in SA!?   

There must be M$ SQL experts on this forum whom can help and make stats in 30 minutes that takes us a day, us being faster than 100's out there?
And then you have users who want to be pilot sites to test and compare and ensure the software works properly? The details people.
Then we have Pi experts and Linux boffins to move the reader over to Linux, for that is what THEY want.
And then we have Plonkster for Victron devices ... we need help getting this donnerse Victron inverter to behave.
And we have Manie who knows EXACTLY what he wants to see.

What if ALL contributors get to use SolWEB for free? Reader being a Pi, Linux, and they can see their systems on their phones?

Because when all of us add our collective ideas and use the software, for free, no not free, at the cost of some development / testing time, and we the owners, hosting it and whom have put in the brunt of the work, make our monies back selling it onto outside people, with Forum Support and references, paying a portion back to Power Forum that can maybe be put back as points to members who actively contribute?

Because guys, you cannot sit and get awesome software, give feed back for new ideas and support, all for free. That has and never will work ... and you ALL know that very well. 

Now I BET, none of you EVER thought of this, nor saw it coming, having free software at the cost of some of your expertise.

And don't lie ... for then you would have said it on the forum and I have missed it. 

[Guest]

Agree totally what you said Jaco. We should have started with what we want and what can be done and put all of that together. The end goal should be for me the pi reading the data and you can view it on your Phone , PC etc, but also that you have full control from a web base application to control the settings of the inverter, (When to switch)   

[___]

11 minutes ago, The Terrible Triplett said:

Non-starter ... explain some more?

The range of technologies, skills and opinions are too diverse. To take a leaf out of jdp's book, I'm spending family time here to write something. I'm scratching a personal itch, in other words. I'm going to do it in a way that takes the least time, or in other words, I'm going to use the technologies that I know. Hence... Raspberry Pi, Linux, Python. When someone asks me to collaborate on their project that uses 1 out of the 3 (replacing the other two with C-hash and Windows)... well, now there's an extra obstacle in the way... now I have to first learn C# and then I have to install Windows (joke below), something that I haven't done in about 7 years... effort to reward ratio becomes silly.

To be fair, I suppose, there are many windows/MS developers here, so perhaps you could have done something there. I, personally, was simply doomed not to be a part of it right from the start though, because of the mentioned obstacles :-)

Anyway, so someone once told me that if you take a Microsoft Windows CD and play it in reverse, you will hear Satanic messages. I told him that that is nothing compared to what it does if you play it in the forward direction: Then it installs windows!

[Guest]

3 minutes ago, Manie said:

but also that you have full control from a web base application to control the settings of the inverter, (When to switch)

That, SIR, is a MUST!!! 

And, I am sure, with some effort, that it can be done from SQL back to the device.

1 minute ago, plonkster said:

Anyway, so someone once told me that if you take a Microsoft Windows CD and play it in reverse, you will hear Satanic messages. I told him that that is nothing compared to what it does if you play it in the forward direction: Then it installs windows!

(ROFL) Brilliant!!!

I agree 100% with you.

I never thought of you going the Win route, ever, but I hoped you had more time to get the Pi reader operational, you came so close (I read the thread the other day again looking for your wisdoms), but for Voltronics also. But family time is precious to you. Respect!

@plonksterOn a side note, via email or Skype, can you consult with my developer on WHAT we are missing? I have only till Sunday to get the Victron inverter read, otherwise I need to call that one a day for the moment.

[Guest]

The reader for the pi is not difficult from the usb call . 

[Guest]

11 minutes ago, plonkster said:

The range of technologies, skills and opinions are too diverse

Yes, it can be or it can be used to our advantage, under the right circumstances.

Someone just needs to drive it, and drive it hard with collective support.

[___]

11 minutes ago, The Terrible Triplett said:

till Sunday to get the Victron inverter read

I suppose he's already looked at the ib.victron code on github? Between that and the Victron mk2 documentation, you should be able to figure out out, otherwise he can just email me about the part where he's stuck.

I actually want to redo that library (again! I even do it with my own stuff). I want it to use a callback. When you ask the inverter for info, you will pass a callback to the call, and when the info comes back from the inverter it will call your callback (instead of blocking for it, as it does now). But that is a different story.

The difficult part with the inverter, when you first start working with it, is that it spits out a version frame every second or so, so you have a race condition right off the bat. At any point when you start talking to it, it might throw out another version frame a split second before it responds to the query you just sent it. This means you can never rely on the response to be a response to the actual query you just sent, it might be a response to a different query, or a gratuitous version response.

The way I work around it is to send a version query, wait half a second, and then clear the serial buffer. This works because the inverter will shut up for a few seconds once you've made contact, giving you a window to sync up with it.

Now that I've had a look at the actual code used in the CCGX, I noticed that Victron also uses a callback architecture. Event driven. It actually works very well, together with DBUS, because the event that you are waiting for could come from a different piece of equipment controlled by a different piece of software. Some of it is C and some of it is Python. This really is what I envisioned for my own project, and I actually had a lot of stuff working: BMV, VE-direct charge controllers, mk2, and a csv logger. It was all there... :-)

Edit: I had precisely the same problem though. For anyone to join "my project" would require a switch into an effective no-mans-land where nothing is familiar. Linux. Python. Too much effort man! I'm rolling my own!

[___]

Another side note: The CCGX has a built-in mk2, but with a difference: They have a reset line going to it, so the CCGX can physically reset the mk2 and ensure a clean start. With the usb one, you don't have that.

[Guest]

39 minutes ago, plonkster said:

I suppose he's already looked at the ib.victron code on github? Between that and the Victron mk2 documentation,

Yes en yes. For DAYS on end. As well as any other we could find. It is something STUPID!

Lets cut the cr_p.  

Plonkster, would you be prepared, if we give you the specs, you supply the Pi reader, Linux and Python, for Victron, Morningstar AND Voltronics?

I give you a link to the SQL DB, the structure you need to format it into, and walla, you send the data there , we get it and take over from there.

[Guest]

3 minutes ago, jdp said:

I only see text here.

Pictures as text 

[viper_za]

4 minutes ago, jdp said:

I only see text here.

Mine works looks real good Manie

[Guest]

thanks Viper

[Guest]

I just sent Kleva a PM, apologizing for derailing his thread so thoughtlessly, yet very thoroughly. 

And not once have we spoken about cars, car repairs nor what is in Mart-Mari's fridge . (ROFL) 

[Energy-Jason]

@The Terrible Triplett , Wish @Energy could look into this for we need a leader. Have something that will benefit all Forum Members, irrespective of the device they chose to use.

Hi Guys the forum is a freedom of expression and designs of all types contribute.  I feel narrowing the creative ideas will limit new developments and new fun concepts. Also I feel members get self satisfaction from making their own software and sharing. 

I do however feel at the rate you guys are going there will be so many software options to choose from for forummembers. 

The fact that they can buy and choose what they like the most is the foundation of freedom to design,  invent.

I don't won't to stunt that by governing it,

Sincerely 

Jay

[___]

21 minutes ago, The Terrible Triplett said:

Plonkster, would you be prepared, if we give you the specs, you supply the Pi reader, Linux and Python, for Victron, Morningstar AND Voltronics?

Don't have the time, sorry. If you go back, you'll see that I was worried about this from the start. When it's just me building my own thing, I work at my own pace. If it works like the normal Open Source project I'm involved with, then it is ready when it is ready. If you are in a hurry, you do it yourself (which is what you did). Throwing money at the problem doesn't help either: Money can't add hours to the day. I also have a massive inbound project that involves a Lithium Ion battery with an unsupported BMS (it's a loan, but I'd love to have one for myself). Also, why on earth would I write support for the voltronic? :-P

I'm not even sure I understand what I'm rambling about. Simply, having been in open source as long as I have, I know that cooperation only happens if there is a significant benefit to using the other guys' platform. That's why it hasn't happened: None of the platforms here provided a significant benefit.

In other news: Victron is porting Venus to the Beagleboard, and someone has made a cape (BB speak for what arduino people call a "shield") that has the can bus and serial connectors on it, so very very soon you will be able to run CCGX software on an affordable board. Twice the price of the Rpi, for sure, but really not bad at all.

[Guest]

Good points Energy, but as JDP, in the end it boils down to being rewarded for ones efforts so I am worried that in the end, there is going to be a few people using this, few that software and all slowly dwindling into nothingness because, again, as JDP said, whilst it is new and fun, no-one worries. It is when it is not fun anymore, becomes WORK, that is when the problems start.

For as Plonkster said: I know that cooperation only happens if there is a significant benefit to using the other guys' platform. That's why it hasn't happened: None of the platforms here provided a significant benefit.