Automatic Lighting Adjustment with LIFX Bulbs

Two weeks ago i got my new LIFX bulbs and they are really superb. They are so great because of 4 things:

1. You can control the lights color. It got 3 RGB leds, so you can pretty much choose any color you wish
2. Light Intensity can be controlled as well
3. It's all done Remotely, where the bulbs are linked to your WiFi network
4. You can automate them through programming.

Obviously, the last one on the list is the most intriguing to me. What I am going to show is how to use the bulbs, together with my IP Camera (One Edimax IC3030), to implement a system that can automatically turn the lights on or off based on the lighting conditions. It will also automatically adjust the lights intensity so that you can get a fairly constant lighting level. Since the system is completely based on computer vision, and not time-based, it will work both when it's dark because of sundown, and when it is cloudy, and will turn off if another artificial light source is used in the scenery.
In this article I will describe the system components, algorithm, and setup. The whole system is tied together by my computer and some NodeJS script that i wrote for it to run the algorithm. You can find the script and all the supporting software in this github repo. The script is lightAdjustment.js. You will possibly also want to look into the relatively trivial algorithm determining a brightness factor for a given image, here implemented as a NodeJS C++ module - ConstantLight.cpp

Ingredients

Well, to carry out the task you need 3 components:

• One LIFX bulb

 

The LIFX bulb can be controlled via TCP packets. Although to this date LIFX had not yet released an API SDK for the bulbs, one MagicMonkey wrote a NodeJS module to control them and it provides all the basic functions quite wonderfully. you can find it in NPM. The script can turn the lights on and off, control the color and intensity, as well as provide info on the bulb current state.

• One IP camera

 

Well, we need a camera to learn what's the current lighting level. We will use it both to determine the initial level of lighting, as well as test how changes in the bulb light intensity change it. You can use any other sensor that can provide a simple "Brightness" level. I used the my IP camera and some software that I wrote to take snapshots and analyze the brightness value for them.

• One computer

You need a computer to control the camera and bulb, as well as to run the algorithm software. The software can either run one time or continuously. In this implementation i'm using XPlatform principles, so the OS doesn't matter. As long as it can run NodeJS we are happy.

Algorithm

The computer will hold a software to maintain the light adjustment. The algorithm is fairly straightforward and goes like this:
As a first step determine the desired brightness level. This can be done by waiting to a time of day where you'd like that anything darker should turn the lights on, and determining this as the desired brightness level. Then it continues:

1. Wait for bulb to become available
2. Once available, determine the current brightness level
3. If brighter than desired: 
1. If bulb is off, can't improve, exit
2. Otherwise, turn the bulb one level lower. If it's very low, simply turn the lights off 
3. Go back to 2
4. Otherwise, if darker than desired: 
1. If bulb is at maximum level, can't improve, exit 
2. Otherwise, turn the bulb one level higher. If was off, turn on. 
3. Go back to 2
5. Otherwise, it's good, done, exit. When used in a continuous mode, this would be a good time to wait for a couple of minutes, and go back to 1.

Fairly simple. Some elements of the algorithm need discussion. Like, how is brightness level determined. This comes up next.

How is brightness Determined

To determine a brightness level i'm using my camera to take a snapshot. The particular model i'm using provides a URL to take a current snapshot in a JPG file. I am then using a node module that I wrote to calculate it's brightness level.
Since i'm using the same measure algorithm for determining the desired level and the current level, there's no need for special standards for this level. Just make sure it's coordinated with how you want things to work - low level for dark, high level for light.
The way it's done is by taking some samples from the image in RGB color space and calculating for each its HSB color space values.
RGB and HSB color spaces are easily converted from one to another, and there are plenty of resources providing howtos.
HSB stands for Hue, Saturation and Brightness. That "Brightness" is pretty much equivalent to what we think of as brightness, so i'm simply doing the conversion and taking it as a measure.
That's all. Simple.
B.T.W, in my implementation grabbing the samples is done using LibJPEG, a simple and good tool in analyzing JPG images.
Now that i got "B" samples i simply take an average of them, and this is the brightness level.
There are two additional elements in my algorithm that are worth to take note:

1. To get the result quickly, i'm not taking all samples of the image. Rather, i am taking up to around 50 per row and 50 per column. They are evenly spaced.
2. So that the value is not interfered by local special lighting conditions, like a small light source which doesn't really have an effect at large, i'm dropping all samples that are too distant from others. The way this is done is by calculating for each sample its distance from other samples, and taking only 2/3 of the samples, which have the lowest sum of distances.

You can find the implementation here.

Why no binary search

You'll note, that this problem is a simple monotonous search, calling for an efficient binary search. Here i'm using simple 1 up 1 down method, however. Less efficient, but more natural to the eye. You don't want the bulb adjustment to freak you out going up and down and up and down like crazy.

Threshold and Steps

The algorithm can change the direction of light adjustment, as is clearly seen. This is good for when the lights are turned on because it's too dark, but than it becomes too light, and the bulb should adjust itself by going darker.
For this facet of the algorithm, it is important to determine a threshold for the desired brightness level, so that anything within the threshold distance is considered good. Hence, when checking "lower" or "higher" comparisons bring into the account also a factor of threshold. Seems like in my implementation "3" is a good choice (for 0..255 possible brightness levels). In addition, it's important that any step to either up or down, does not normally change the brightness to more than twice this threshold, so that the algorithm won't keep circling around the desired level.

The script

The script that I wrote and I have working at home is here. It is a simple node script that can be ran through command line with node lightAdjustment. There are 2 optional arguments for it:

• node lightAdjustment -r : this should be used for the initial setup. Running this will determine the current brightness level and set it up as the target level.
• node lightAdjustment -c : this will run the script continuously, making adjustments every 2 minutes. Quite wonderful to see it run :)

With no options arguments the script will run adjustment once and exit.
Note that in order to run it there's a couple of things to do:

1. Setup the camera and bulb 
2. Compile the C++ module (just cd to the root of the solution and go 'node-gyp rebuild') 
3. Go into tests folder and go 'npm install' to get the required additional node modules. 
4. There's a config file here. Setup the name of the bulb to control and url for grabbing snapshots with the camera. 
5. Run the script from this folder

One thing to note in the script is that working with he bulbs and camera is highly asynchronous. A big deal of the script is to adjust the lights and then wait till they are adjusted in order to later take a picture and see how to continue.

Setup

The main items to make sure in a good setup has to do with the camera. So note the following:

• Put the camera where changes in light show in where it's looking at. Yes...test with the bulb, adjusting the lights. 
• If you are lighting the street, point the camera at the street, this will achieve the most fitting adjustment
• If you are lighting your living room, the last thing you want is your picture on the web that some wise-ass hacker took from your cam. Point it to the wall 
• If you do point it to the wall MAKE SURE THAT IT IS NOT HAVING A SHADOW WHEN THE LIGHTS ARE ON on the area that you are taking picture from. Otherwise the algorithm will be really confused. If you do get shadow...worry not, just tilt the camera so that its taking picture from where there are no shadows.

Also, when setting up the desired level, do it when the sun is not completely up. Otherwise there's a good chance that after the first night the lights will be on for the whole time. Wait till the evening to when you'd normally turn the lights on, do the reset then.

Summary

So, we saw a simple algorithm for automatic lighting adjustment LIFX bulbs. That was fun.
Bye
Gal.