Introduction

From the last weekend of july i've started to backport features from the new rewrite code to development. For several reasons.

1. I have been running the rewrite code for some time now, but haven't finished it enough to replace the current development code. So i consider 99% of the code to be stable.
2. I still want to test this new code in public to test if my assumptions are right.
3. I don't want to keep these new cool features away from the general user any longer.

This blog is meant to inform you all about the changes that are slowly being backported while development on the rewrite code is still going on.

Some parts of this blog are copied from the forum posts i already had about it.

New core changes already backported

All the changes below are already being backported from rewrite. This however means that the new code has been implemented besides the old code, until all the old code has been removed.

Internal event driven actions

All pilight code was intertwited in one big knot. Much like an old telephone central. When new module was written, a cable had to be plugged in somewhere in another older piece of code.

The new event driven approach overcomes this by implementing a producer/consumer pattern. So the config update functionality now triggers the REASON_CONFIG_UPDATE event. On the other side various parts of pilight react on this event. So, when new modules are released, they only have to listen to certain events. The code that triggers the events doesn't have to know that someone is or is not listening.

Threadpool

Since the beginning of pilight, all parallel code was run in a separate thread. This could lead to over 20 threads in complex setups. The new pilight code uses a threadpool. Functions are added to the threadpool awaiting execution and a fixed number of thread (workers) will execute them in parallel. At the moment pilight uses a fixed number of 4 threads to handle all the work.

Centralized socket handling

Since the beginning of pilight, all sockets communication was implemented on its own. Because, pilight uses non-blocking sockets, each implementation also used its own code to wait for events. The new approach uses a centralized socket handling library that processes all sockets at once. So all seperate threads were reduced to just one, which grealy reduces resources.

Custom webserver implementation

The latest webserver implementation used by pilight (Mongoose) had some big limitations:
- SSL could not be easily ran on mbedtls but only on OpenSSL.
- Adapting Mongoose to use a centralized socket pool was quite impossible.
- Adapting Mongoose to use the threadpool was quite impossible.
- Mongoose had a big overhead in functionality already present in pilight.

I thereby decided to drop Mongoose and created my own webserver (by reusing some of Mongoose and others code). This new webserver perfectly fits the new pilight codebase and also implements SSL by default.

All backported code has already been actively unit tested

All new code written is being tested constantly to be sure no new bugs are introduced when writing new code. The status of this check can be seen on the rewrite code page:

https://github.com/pilight/pilight/tree/rewrite

What's changed

28-07-2017

At this moment the http, mail, and webserver module and the full pilight-sha256 program has been backported from rewrite. The asynchronous I/O library libuv has been added as well as the new SSL and eventpool module. The openweathermap and weather underground protocols have been adapted to use this new code as well as the pushbullet and pushover event actions.

• pilight now supports a HTTPS webserver which can be configured in the settings using the webserver-https-port.
• New gpio-platform setting. pilight does not try to discover the GPIO platform it is being ran on (like Raspberry Pi, Hummingboard, Odroid), but wants you to explicitly tell this to pilight. The platforms supported can be retrieved through pilight-daemon -H
• wiringX and mbedtls are not staticly compiled anymore and therefor require seperate installtion. So make sure you have the stable repository enabled (besides the nightly).
• pilight also stopped detecting if the mailserver you have configures requires an SSL connection. To tell pilight about the SSL requirement of a mail server the smtp-ssl (1 or 0) setting has been added. Servers that switch from a plain connection to SSL require a 0 value here.
• The ram resource usage has been removed, because this wasn't reliable.