Also, I'm really beginning to think the bus should by MQTT. It runs on most operating systems, its free, it's meant for telemetry and IoT stuff, it has pubsub... etc.

I have to read more about MQTT.

I dont know it at all

Sent from my SM-G920F using Tapatalk

For the guys who want to try the Raspberry Pi usb have a look at

 

http://sourceforge.net/p/pyusb/wiki/Home/

So after this discussion, and ending up with a somewhat slow day at work, I banged out a quick json service in python, with the angular bit on top. No real values at the moment, it just creates random values, but the idea is there. It already depends on paho_mqtt (for future mqtt integration) and the README explains how to get mosquitto going.

 

This is pretty much hacked together and not polished at all.

 

https://github.com/izak/ib.bluelantern

Is it avail in SA

Sent from my SM-G920F using Tapatalk

Very interesting, the Pine64.

 

Found this: https://www.raspberrypi.org/forums/viewtopic.php?f=62&t=128128

Still coming ti seems to me: https://www.pine64.com/product#features

Pi is so 2013. I would use the Pine 64 

 

https://www.pine64.com/

 

What I like about the Pine is that it has a RTC. You need a RTC if you want to do things against time.

 

Let me tell you a funny story about the lack of a RTC.

 

So the company I work for builds some embedded systems for research purposes, and we make a companion product that's based on the Rpi. One of the things it does is flash firmware onto said embedded hardware.

 

So one day someone boots the Rpi, plugs in his embedded device, and a new firmware starts flashing. About halfway through the flashing, ntp wakes up, notices the clock is 20 minutes slow, and reset it. Of course at this point the flashing process notices that it has been 20 minutes since it's had a response from the hardware, times out, and bricks the device.

 

So now we disable ntp while we flash.

 

:-)

Haha, this is an interesting thread.

 

I actually wrote an application to do some of the things you guys suggest but lost patience with HTML & AJAX and stopped while doing other things.

 

I have actually written the following tho:

Rest API written using Ruby (natively supported on RPi)

The app runs using WEBrick, which is a very lightweight HTTP server built into Ruby

I made it really simple: You send a command using POST and it is sent to the inverter.

All the results are stored in SQLite db

I also allow GET requests with time based (just used unix timestamps) input to show the history of the commands run.

 

It actually works pretty well and I was able to do a simple fire and forget NC (Netcat), every 2 seconds to run commands like QPIGS.

The output is then stored in the DB.

 

I had the output of the commands parsed using HTML.

 

Motivations:

Storing raw input and output in the DB as is: Initially I didn't do this, but it actually makes it much, much easier to integrate new features (just requires HTML knowledge)

SQLite: My company runs a fleet of a few hundred MySQL servers and I have a LOT of experience with them, SQLite is pretty close and concurrent access is actually pretty trivial which is pretty awesome. (It is thread safe and consistent)

I used Ruby because it is super lightweight and I have plenty of experience writing apps in it. It is also cross platform (including the serial port stuff I did).

 

All in all works incredible well on my RPi.

 

I can post the REST API if you guys are interested?

I don't quite have the motivation right now to work on the HTML part further but my backend is fairly simple and solid

 

Gnome . Glad to see some one already got feedback from the inverter. It will be awesome if you can share what you have

Sent from my SM-G920F using Tapatalk

Wow. Pi, Pine, Python, AJAX, Ruby, Web service, Gnome, SQL, bla bla bla.

 

Doesn't anyone work in C and assembler, directly on a MCU, no OS or anything, anymore? Are we a dying breed?

Wow. Pi, Pine, Python, AJAX, Ruby, Web service, Gnome, SQL, bla bla bla.

 

Doesn't anyone work in C and assembler, directly on a MCU, no OS or anything, anymore? Are we a dying breed?

 

OomD, you forgot to mention the 'vieslike one', Windows! 

Jeez, and all these years I thought it was called windoze.....

Wow. Pi, Pine, Python, AJAX, Ruby, Web service, Gnome, SQL, bla bla bla.

 

Doesn't anyone work in C and assembler, directly on a MCU, no OS or anything, anymore? Are we a dying breed?

 

Not at all. When required, I will still code C. The low cost of hardware compared to the relative high cost of software development does however push developers towards higher level languages.

 

The arduino language makes sense to me (and I think the Atmel family has a processing variant too). It's a simplification for developers, but it still compiles to fast machine code and allows users to use low level code.

 

Some of it borders on the ridiculous, at least at face value. For example, there is this thing called espruino that runs on the arm cortex m4. It's a javascript interpreter. Now if I wasn't introduced to nodejs BEFORE I came across this I would consider it all foolishness, but with some nodejs experience (and coffeescript!) I've learned to love some parts of the language. It's a lot less messy than back in the day with Netscape 4 and IE 5, and the scoping/closure stuff does grow on you. I probably wouldn't do anything serious in it though.

 

Second example, Micropython. I like python. I'm still not sure about this one. Probably not for a serious product.

Let me tell you guys something. Writing at the lowest possible level, i.e. C and assembler, allows one to extract and use all the available power from a microcontroller. While there's not really aything a high level language can't do it's more about the wastage for me. Using a high level language means you now need more RAM, ROM (or Flash/Codespce), more horsepower form the CPU, and more help from the OS to get the task done. While the same task can be achieved much faster*, more efficiently, and much cheaper by sticking to efficient means of coding.

 

Yes, it needs more hours of coding. But, it also needs more brain power, more forethought in the design, and a better understanding of the underlying work. So it needs s higher skill level, which again translates to more money. That is so, yes.

 

I understand the need for higher level languages, off course. But those belong on a PC, not in an embedded environment. Putting high level langaages on an embedded environment raises the cost. Let me explain in terms that is easier to understand: Imagin you have an inverter that can generate 5KW, but 2KW of that is used for itself... you only get 3KW of usable energy out of it. But, it is very easy to use, and even novices can install and use it without an incling as to the complexities of battery management, PV MPPT tracking, etc. So you're trading efficiency for ease of use. What a waste!

 

This reminds me of the arguments between iPhones and Android devices, where people gloat about the amount of RAM in their phone. Android fans have long used "more RAM" as a one-up against the iPhone, without realising that the iPhone achieves the same performance, if not better, but just using the RAM it has efficiently. Ergo, less RAM needed, thus lower manufacturing cost.

 

Anyway, I'm not negating the benefits of using high level systems. I'm merely rueing the fact that proper efficient use of resourcs seems to be on the way out. It's a mindset thing.

 

Gotta go, have some code to finish...

 

*Years ago a colleague challenged me, saying whatever I wrote in assembler he could do faster, more efficiently, and with less (executable) code in C. I'm sure you all know how that ended.

You know, I've had this same debate a bit higher up in the language tree, some years ago when this new-fangled thing called Java came out. I was horrified at the amount of RAM it took to run a jvm, and it was so slow! Some years later I ended up in a job where they deployed their apps on JBoss (enterprise Java server)... and I was even more horrified. This beat was impossible to get stable. What you would do, whenever it started using too much RAM, is to kill it and restart it.

 

I sat there thinking how my apache web server just sits there week on week using almost no RAM and several times the volume.

 

In each case, however, the argument was how developer time is so expensive and computer hardware is now so cheap... and I agree with all of that, but I could not shake the feeling that we have become... well... LAZY!

 

But then on the other side, I've also come to believe that it is wasteful to use the wrong tool for the job, and after I started coding in python the sheer speed advantage convinced me that interpreted languages do have a place in this world: Especially cool ones like python that can be easily extended in C if you need it!

 

(Check out numpy. We use it extensively where I work, and it is literally an order faster for heavy mathematical work).

 

So here I am, sort of precariously balanced between the two worlds, because I understand the arguments from both sides. So my rule of thumb, for now, is to put the stuff that is critical and/or needs to be fast into C code, and do the rest in python.

 

As I've indicated earlier, I'm a bit concerned about using a RPi to manage batteries. Linux is stable... but it's not a certified RTOS and it does fall over. If I use it, there's going to be a watchdog or similar to take care of it :-)

That, and I'm still curious how much effort it would be to make an STM32 chip act as a USB host, so that it can talk MK2 to my inverter. If I can get that done, the RPi can go away, but that is probably goint to take a long long time. In the interim, the Rpi is going to take center stage. In addition, it's probably going to stick around, because it's just easier to use someone else's http server implementation rather than writing my own. I promise to use one written in C :-P

We have been pulled onto some other work for the moment, so the RPi development is subtly delayed.

 

Step one was simply to just read the data,quickly. All went well till we ran head-on into Morningstar requirements. 

Step two would be to make software to run on the RPi to read the data from the USB ports and store that in CSV format, per device, on the RPi.

 

Then move on from there.

 

Now, to add onto Plonkster and OomD, yes, we can do it the right way, go deep, dot it properly, or we can do it the quick way, at the cost of a little bit more memory or processing power.

 

That debate becomes null and void when doing experimental development at ones own risk and cost. 

My present status:

 

1. My can bus shield for the arduino arrived yesterday. I have to solder on the headers still.

2. While I was waiting, I spent half an hour or so every night building a can bus interface on strip board. This is also one header away from prime time. Took a bit more time as it uses the Texas Instruments ISO1050 which needs a powered can bus. The Victron BlueSolar has a powered can bus, but it runs at 13V, so there I went scrounging for a 7805 and a fuse to do it safely.

3. I have a can bus example running on the STM32F4 board. So far untested.

4. I have a mock framework up and running. It's called ib.bluelantern, derives from DC comics' Blue Lantern Corps :-) It uses mqtt for message passing, angular for the display on a web browser, python and pyramid for the app server bit. This will eventually run on the Rpi.

 

So next steps are:

 

1. Example can bus app on the arduino.

2. Wire the arduino and the stm32 together and see if they can talk to each other.

3. Once that works, wire one of them to the Victron MPPT.

4. Figure out the can bus messages, ie NMEA 2000, which seems simple enough if you look at the canboat project.

5. Read info from can bus and publish on mqtt.

6. Write code to pull mqtt published charger into the app server.

7. Adapt my existing code that reads the Victron inverter data to pass that info onto the mqtt bus.

8. Read inverter info from mqtt into app server, and display.

9. Write service that connects to mqtt and logs to database and/or remote server.

 

Then... when that long list is done...

 

10. Write something that reads stats from the lower end VE.Direct hardware using USB.

11. Bully someone into writing something for the Axpert.

12. Steal TTT's half-written Morningstar process to do the same.

 

After all that, I'll have a generic product with plugible data importers. This should all be done some time after 2023 :-P

... Now, to add onto Plonkster and OomD, yes, we can do it the right way, go deep, dot it properly, or we can do it the quick way, at the cost of a little bit more memory or processing power.

 

That debate becomes null and void when doing experimental development at ones own risk and cost. 

Yes, off course. Sorry my posts were in no way meant to negate or degrade the effort you guys are putting in. There are certainly a lot of factors one needs to consider when developing such a system. I was wondering out loud, more than anything else.

 

Anyway, I am busy developing an embedded monitor/control system, as per my related thread some weeks back. But it will be embedded, not running on top of a commercial/free OS or anything like that. I'll post more details another day (on another thread), I don't want to interfere with your project. Good luck!

One upside from my elaborate way of doing this, is that things can be distributed across a network, with a sensor pushing data onto the mqtt bus even if it is not on the same physical piece of hardware. Unfortunately that requires a TCP stack on the sensor side... so this is hardly perfect, but in theory it is decoupled. My main reason for this was mostly because MQTT is the defacto standard now for IoT telemetry sort of things, and it means separation of concerns, if you want to implement an interface for your China special inverter in PHP... then in theory you can do that :-)

One upside from my elaborate way of doing this, is that things can be distributed across a network, with a sensor pushing data onto the mqtt bus even if it is not on the same physical piece of hardware. Unfortunately that requires a TCP stack on the sensor side... so this is hardly perfect, but in theory it is decoupled. My main reason for this was mostly because MQTT is the defacto standard now for IoT telemetry sort of things, and it means separation of concerns, if you want to implement an interface for your China special inverter in PHP... then in theory you can do that :-)

Where and how will the mqtt server be hosted?

A programming language is a tool. You use the best tool for the job which is evaluated based on requirements.

A good software engineer isn't tied to any specific language, so generally I like to ignore these why aren't you using <insert my favorite/only hardcore language here statement>

 

If C is your language and you like doing everything in that one language that is awesome.

Personally I've worked in many languages and each has its strong suits and weak points.

 

Ruby and Python are both good solutions here. Both are portable, include a vast library that is standard to the language.

Both are easy to install and use. Both have low upfront development cost. Performance is more than adequate for the problem space.

 

If the project were better suited to C/C++ or Java, I would use that instead.

But really there are no benefits worth mentioning doing it in those languages.

Certainly not performance as it hasn't even been a constraint up to now.

Processing power is cheap.

 

Personally I don't often use a non-portable language like .NET so that is out for me.

 

I considered doing this in Go and did a prototype in Go but the standard library didn't do everything I wanted.

JVM is too heavy.

C/C++ is far too high cost in terms of development effort and maintenance costs. It doesn't bring any positives to the table except "performance" which isn't even a problem

 

#JustSaying

Are you still going to post the API ?

Sent from my SM-G920F using Tapatalk

Are you still going to post the API ?

Sent from my SM-G920F using Tapatalk

Yep, just fixing up some bugs. Working on it again today.

I haven't worked on it for over 5 months.

Work has been busy and doing some home improvements in my free time that isn't spent with friends family.

Thanks Gnome appreciate it

Sent from my SM-G920F using Tapatalk