Home Automation on steroids
This is the first in a series of blog posts about my adventures into home automation (HA) land with openHAB and it will be primarily about building and preparing the Linux (Debian) based system. In next articles we will dive into the details of setting up a HA system with openHAB and some other tools. This is focused on my particular setup, but you will most likely pick-up a thing or two to apply to your own environment. Let's go to the process of getting a Linxu system up and running.
I use Debian Buster (10) for this project. Although it is still in testing phase, we all know how rigid the testing and release procedures for Debian are. And we don't need any special features, so for our HA project it will be more than adequate. Over the last few months I've had zero issues related to Debian 10 or its core packages.
Install Linux base system
This is rather straight foreward and not very specific for an HA setup. But, as we need it anyway, let's go and dive into the boring details quickly (and briefly). First, download the Debian Buster image here and put it on a bootable device, like a USB stick. Installation is quite straightforward and you can define a fixed IP address for the local network interface during install or use DHCP for now and change the network settings later.
Although it depends on your needs and preferences, some of the basic choices and settings to make during Debian initial setup tool are:
1. Use LVM for flexibility in disk layout and expansion. 2. See some of the parameters to use during setup in the table below. 3. Create an additional user named `myuser` (replace with your choice) for login, instead of `root`. 4. Define local repository images. 5. Only install SSH server and core packages.
Collect the basic configuration parameter for the setup, like:
|NIC||enp3s0 (if you have multiple)|
|Name servers||192.168.1.1, 188.8.131.52|
Our openHAB server gets IP address 192.168.1.2 and hostname ha.openhab.org in the description below. Replace with your own parameters.
sudo and add user
myuser (replace with your own selected username) to the
apt install sudo usermod -a -G sudo myuser
Start the installed SSH daemon (if not already started), check the IP address and logout.
systemctl status sshd systemctl start sshd ip a exit
Start PuTTY, or another SSH terminal application you prefer, and login as user
Since my server has Realtik NIC's and the Realtek firmware is part of the non-free packages repository in Debian, we must update the sources list for the Realtek NIC firmware and any other non-free stuff we want to install later, like unrar.
sudo nano /etc/apt/sources.list
Content of the sources.list file after being update:
deb http://ftp.nl.debian.org/debian/ stretch main contrib non-free deb-src http://ftp.nl.debian.org/debian/ stretch main contrib non-free deb http://security.debian.org/debian-security stretch/updates main contrib non-free deb-src http://security.debian.org/debian-security stretch/updates main contrib nonfree deb http://ftp.nl.debian.org/debian/ stretch-updates main contrib non-free deb-src http://ftp.nl.debian.org/debian/ stretch-updates main contrib non-free
Now update the repository index and install the Realtek firmware.
sudo apt update sudo apt install firmware-realtek
Upgrade the distribution and install some utilitities we will need later on.
sudo apt upgrade sudo apt dist-upgrade sudo apt install -y mc screen git zip unrar htop sysstat curl net-tools sudo apt install -y apt-transport-https software-properties-common tcpdump
Setup the SSH service to use key authentication.
mkdir .ssh chmod 700 .ssh nano .ssh/authorized_keys
Paste the private key (can be generated by PuTTYgen) and change the security for the key file.
chmod 600 .ssh/authorized_keys
Edit the SSH daemon configuration to disable root login and password authentication.
sudo nano /etc/ssh/sshd_config
Port nnnn <=== replace with ypour port# PermitRootLogin no PubkeyAuthentication yes PasswordAuthentication no ChallengeResponseAuthentication no
Restart the SSH daemon
sudo systemctl restart ssh
Test login with myuser via SSH (PuTTY).
Screen autodetach setting might not be enabled by default so it make sense to put a corresponding directive in your .screenrc
Setup the network interface. First check what interfaces we have and which name has been assigned.
sudo ip a
To configure the network interface(s), type:
sudo nano /etc/network/interfaces
Note that we must use auto enp3s0 (replace with your NIC alias) because the systemd network-online.target only cares about interfaces that are listed as "auto" in /etc/network/interfaces, not allow hotplug.
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). source /etc/network/interfaces.d/* # The loopback network interface auto lo iface lo inet loopback # The local network interface auto enp3s0 iface enp3s0 inet static address 192.168.1.2/24 gateway 192.168.1.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 192.168.1.1 dns-search moerman.online
sudo service networking restart
Verify the configuration of the network and DNS settings.
sudo ip a sudo cat /etc/resolv.conf
Define your networks and give the local network a recognizable name, I used mordor in this example.
sudo nano /etc/networks
default 0.0.0.0 loopback 127.0.0.0 link-local 169.254.0.0 mordor 192.168.1.0
Install additional tools
Although not needed to run an HA system, I like to be prepared and always install some additional tools to be able to detect potential issues. Let's start with iperf 3.1.3.
sudo apt-get remove iperf3 libiperf0 wget https://iperf.fr/download/ubuntu/libiperf0_3.1.3-1_amd64.deb wget https://iperf.fr/download/ubuntu/iperf3_3.1.3-1_amd64.deb sudo dpkg -i libiperf0_3.1.3-1_amd64.deb iperf3_3.1.3-1_amd64.deb rm libiperf0_3.1.3-1_amd64.deb iperf3_3.1.3-1_amd64.deb
Optionally install some simple network monitoring tools.
sudo apt-get install iptraf iptraf-ng
Install and configure Samba, so we can access and edit the openHAB configuration remotely from a Windows PC.
sudo apt install -y samba cifs-utils
To configure Samba and allow creating simple shares later, type:
sudo nano /etc/samba/smb.conf
Most of the entries in smb.conf are not relevant for our configuration. You can cleanup if you like. The essential settings are:
[global] workgroup = MYWORKGROUP server role = standalone server obey pam restrictions = yes unix password sync = yes passwd program = /usr/bin/passwd %u passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\spassword:* %n\n *password\supdated\ssuccessfully* . pam password change = yes map to guest = bad user
To test the configuration run the following command:
If all is ok, restart the smb service.
sudo systemctl restart smbd sudo systemctl status smbd
Create a Samba password file for accessing the shares we will create later.
sudo nano /root/.smbcredentials
Set the permissions more strict.
sudo chmod 640 /root/.smbcredentials
Just to be sure, reboot the system and check if there are no issues.
Install openHAB prerequisites
OpenHAB is built on the Eclipse Smart Home platform and runs on Java. So our first order of business is to install Java. Also we will be doing a lot of communication from sensors using MQTT which requires a MQTT broker to be installed. And since openHAB has limited built-in persistence functionality for state and history, we will install InfluxDB to store timeseries of sensor values.
OpenHAB currently supports Java 8, not Java 9 or 10. There are multiple options available, including Oracle Java, but I opt for the Zulu open source Java SDK as it is a completely open source, fully certified Java SE compliant build of OpenJDK. For compatibility with openHAB Cloud, you should install at minimal revision 101. The current Java 8 version for Linux 64-bit can be found here. At the time of writing, openJDK version 1.8.0_163 was the current version.
Add the Zulu repository for Ubuntu.
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0xB1998361219BD9C9 sudo apt-add-repository 'deb http://repos.azulsystems.com/debian stable main' sudo apt update
Install the Zulu package.
sudo apt install -y zulu-8
Check the installed Java version.
$ java -version openjdk version "1.8.0_181" OpenJDK Runtime Environment (Zulu 184.108.40.206-linux64) (build 1.8.0_181-b02) OpenJDK 64-Bit Server VM (Zulu 220.127.116.11-linux64) (build 25.181-b02, mixed mode) $ javac -version javac 1.8.0_181
Install Mosquitto MQTT Server
Mosquitto is one of the more popular open source MQTT brokers and is well supported in the openHAB community. Install the Debian Mosquitto package from the repository.
sudo apt install -y mosquitto mosquitto-clients
By default, Debian will start the Mosquitto daemon after install. Let's first test the default configuration. We'll use one of the Mosquitto clients we just installed to subscribe to a topic on the broker. Topics are labels that you publish messages to and subscribe to. They are arranged as a hierarchy, so you could have sensors/outside/temp` and sensors/outside/humidity, for example. How you arrange topics is up to you and your needs. For now we will use a simple topic to test our configuration changes.
Open a second terminal (using
screen). In the new terminal, use
mosquitto_sub to subscribe to the test topic.
mosquitto_sub -h localhost -t test
-h option is used to specify the hostname of the MQTT server, and
-t specifies the topic name. There is no output after hitting ENTER because
mosquitto_sub is waiting for messages to arrive. Switch back to the other terminal and publish a message:
mosquitto_pub -h localhost -t test -m "hello world"
The options for
mosquitto_pub are the same as
mosquitto_sub, though this time we use the additional
-m option to specify the message to send. Hit ENTER, and the "hello world" message should pop up in the other terminal.
Enter CTRL+C in the second terminal to exit out of
mosquitto_sub. Next, we'll secure the configuration using password-based authentication. Mosquitto includes a utility to generate a special password file called
mosquitto_passwd. This command will prompt you to enter a password for the specified username, and place the results in /etc/mosquitto/passwd.
sudo mosquitto_passwd -c /etc/mosquitto/passwd <USERNAME>
Provide the chosen username and enter the password (twice) when asked. Next, open up a new configuration file for Mosquitto and tell it to use this password file to require logins for all connections.
sudo nano /etc/mosquitto/conf.d/default.conf
This should open an empty file. Paste in the following:
/etc/mosquitto/conf.d/default.conf allow_anonymous false password_file /etc/mosquitto/passwd
Note: be sure to put a trailing newline at the end of the file!
allow_anonymous false will disable all non-authenticated connections, and the
password_file option tells Mosquitto where to look for user and password information. Restart Mosquitto and test the changes.
sudo systemctl restart mosquitto
Switch to the second terminal window and subscribe to the 'test' topic, using the username and password just provided.
mosquitto_sub -h localhost -t test -u "<USERNAME>" -P "<PASSWORD>"
It should connect and sit, waiting for messages. You can leave this terminal open and connected for the rest of the tutorial, as we'll periodically send it test messages. Publish a message with the other terminal session, using the username and password.
mosquitto_pub -h localhost -t "test" -m "hello world" -u "<USERNAME>" -P "<PASSWORD>"
Be aware that we're sending passwords unencrypted over the local network, to connect over the internet, either VPN or SSL is recommended.
The configuration of Mosquitto is now ready to work with openHAB. Later on we'll configure openHAB to connect to it.
Install InfluxDB for persistence
Add the InfluxDB APT repository.
curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add - echo "deb https://repos.influxdata.com/debian buster stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
Install and start the service.
sudo apt update && sudo apt install influxdb sudo systemctl start influxdb
InfluxDB uses the configuration file
/etc/influxdb/influxdb.conf and optional environment variables. If you do not uncomment a configuration option, the system uses its default setting. All the default settings can be viewed with the influxdb config command.
We will disable reporting of usage statistics and change the API port in the
[http] section of the configuration file to
8886. Also, enable API request logging for now to debug any potential setup issues (we'll disable it later on). And finally change the backup RPC endpoint to
8899, so it won't interfere with other services on the same host. The changed settings are automatically picked up by the InfluxDB service. But the first order of business is to create the admin user in InfluxDB:
$ influx -port '8086' -host localhost > CREATE USER admin WITH PASSWORD '<PASSWORD>' WITH ALL PRIVILEGES > EXIT
/etc/influxdb/influxdb.ini configuration file.
... reporting-disabled = true # Bind address to use for the RPC service for backup and restore. bind-address = "127.0.0.1:8899" [http] # Determines whether HTTP endpoint is enabled. enabled = true # The bind address used by the HTTP service. bind-address = ":8886" # Determines whether user authentication is enabled over HTTP/HTTPS. auth-enabled = true ...
From then on, we need to specify the port, username and password on the commandline for InfluxDB, like this:
influx -port '8886' -username admin -password <PASSWORD> -host localhost
Restart the service and create a database and users for openHAB and Grafana.
> CREATE DATABASE openhab_db > CREATE USER openhab_user WITH PASSWORD '<PASSWORD>' > GRANT ALL ON openhab_db TO openhab_user > CREATE USER grafana_user WITH PASSWORD '<PASSWORD>' > GRANT READ ON openhab_db TO grafana_user > EXIT
Grafana does all the graphing stuff for us. First, the Grafana repository must be added to enable installing the Grafana package for Debian. Note that we use the Debian stretch repository for now, as there is no newer version available at the time of this writing.
echo "deb https://packagecloud.io/grafana/testing/debian/ stretch main" | sudo tee /etc/apt/sources.list.d/grafana.list
Then add the Package Cloud key. This allows you to install the signed Grafana packages.
curl https://packagecloud.io/gpg.key | sudo apt-key add -
Update the Debian Apt repositories and install Grafana.
sudo apt update && sudo apt install grafana
Enable and start Grafana by running:
sudo systemctl daemon-reload sudo systemctl start grafana-server sudo systemctl status grafana-server sudo systemctl enable grafana-server
After successful installation you should be able to reach the Grafana dashboard at http://192.168.1.2:3000 with the default login admin:admin. You will be prompted to change the password. Change it to something more secure.
To connect Grafana to InfluxDB, go to “Add Data Source” and create a new source pointing to the InfluxDB database, providing the database name (openhab_db) and credentials chosen before (grafana_user).
Disable user signup and enable anonymous access (for later image export) in the configuration file
/etc/grafana/grafana.ini and restart the Grafana service.
[users] # disable user signup / registration allow_sign_up = false [auth.anonymous] # enable anonymous access enabled = true [auth.basic] # disable basic authentication (enabled by default) enabled = false
Install openHAB 2
After al these preparations we can finaly start the installation of openHAB. Let's first create a user
openhab for running the daemon and make it safer by not allowing interactive login.
sudo adduser --system --no-create-home --group --disabled-login openhab
I choose to install openHAB2 manually, not from a .deb package. Download the platform independent archive file and extract it to the path
/opt/openhab2. Choose between the latest Beta release or a Snapshot with all incoming contributions. As openHAB 2 is still in an evolving state, the snapshot may be the preferred choice. Download and extract the latest snapshot version of openHAB 2 from this download page.
cd /tmp wget -O openhab-download.zip "https://openhab.ci.cloudbees.com/job/openHAB-Distribution/lastSuccessfulBuild/artifact/distributions/openhab/target/openhab-2.4.0-SNAPSHOT.zip" sudo unzip openhab-download.zip -d /opt/openhab2 rm openhab-download.zip
The extracted openHAB files should belong to the earlier created openhab user. Execute:
sudo chown -hR openhab:openhab /opt/openhab2
Everything is ready for a first test run. Execute openHAB and you should be able to reach the openHAB 2 Dashboard at http://192.168.1.2:8080 (or use it's FQDN) after a few minutes:
sudo su -s /bin/bash -c '/opt/openhab2/start.sh' openhab
You will see the openHAB console in your terminal session and can directly interact with it. Please be aware, that openHAB 2 will need a few minutes so finish the first start, even after the openHAB console is visible. Let openHAB settle for 10-15 minutes. If the portal is not reachable by then, restart openHAB once.
Launching the openHAB runtime __ _____ ____ ____ ____ ___ ____ / / / / | / __ ) / __ \/ __ \/ _ \/ __ \/ /_/ / /| | / __ | / /_/ / /_/ / __/ / / / __ / ___ |/ /_/ / \____/ .___/\___/_/ /_/_/ /_/_/ |_/_____/ /_/ 2.4.0-SNAPSHOT Build #1379 Hit '<tab>' for a list of available commands and '[cmd] --help' for help on a specific command. Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown openHAB. openhab>
Stop openHAB using the command
system:shutdown. This will take several seconds.
To enable openHAB to run as a daemon and start automatically at boot, we leverage systemd and register openHAB as a service, so that it runs at startup and automatically restarts if openHAB crashes. The service will be running with the privileges of the user openhab and expects the openHAB files under
Create the file
/lib/systemd/system/openhab2.service with the following content:
[Unit] Description=The openHAB 2 Home Automation Bus Solution Documentation=http://docs.openhab.org Wants=network-online.target After=network-online.target [Service] Type=simple User=openhab Group=openhab GuessMainPID=yes WorkingDirectory=/opt/openhab2 ExecStart=/opt/openhab2/start.sh server ExecStop=/bin/kill -SIGINT $MAINPID Restart=on-failure [Install] WantedBy=multi-user.target
Next, enable the service to be executed on system startup, start the service and retrieve status information:
sudo systemctl daemon-reload sudo systemctl enable openhab2.service sudo systemctl start openhab2.service sudo systemctl status openhab2.service
The output of status after a successful execution should be similar to:
openhab2.service - The openHAB 2 Home Automation Bus Solution Loaded: loaded (/lib/systemd/system/openhab2.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2018-10-03 20:47:19 CEST; 7s ago Docs: http://docs.openhab.org ...
To stay up to date with new releases, you should do regular upgrades of your manual installation. This is especially important if you are working with the latest snapshot as changes and fixes are incorporated constantly.
OpenHAB uses a script to update to any other version, or from stable to snapshot and visa-versa. Your personal configuration will be retained on upgrades, but you should stop openHAB and perform a backup first. From version 2.1.0 onwards, openHAB is distributed with the update script included. This script should be called from within openHAB's root directory. Assuming the openHAB base directory is
/opt/openhab2/, simply run the following commands to update to the next major version of openHAB:
cd /opt/openhab sudo runtime/bin/update
You can also specify any version as a parameter. For example, to switch back to 2.3.0:
sudo runtime/bin/update 2.3.0
You may also change to openHAB's more frequent, but a little less stable, snapshot builds. Just append -SNAPSHOT to the target version, like:
sudo runtime/bin/update 2.4.0-SNAPSHOT
OpenHAB file locations and backups for manual install
|Additional add-on files||/opt/openhab2/addons|
|Userdata like rrd4j db||/opt/openhab2/userdata|
OpenHAB supports the use of environment variables. These include:
It is recommended to make a backup of your configuration before any major change. To make a backup of openHAB, you need to retain your configuration and userdata files. OpenHAB 2 comes with scripts for storing your configuration in a zip file which is saved in
/opt/openhab2/backups/ by default. You can change the default path by setting the
$OPENHAB_BACKUPS environment variable.
To create a backup, stop openHAB and run the command:
To restore from the backup files, stop openHAB and run the command:
sudo $OPENHAB_RUNTIME/bin/restore $OPENHAB_BACKUPS/myBackup.zip
You can learn more about openHAB and how it works by looking at the log files. Execute the following command in one session or have both files separated in sessions side by side:
tail -f /opt/openhab2/userdata/logs/openhab.log -f /opt/openhab2/userdata/logs/events.log
openHAB basic configuration
To make openHAB work properly and interact with its environment, some basic tasks must be performed, including changing security settings and creating Samba shares to allow easy editing of the configuration from your remote PC. I recommend Visual Studio Code with the openHAB extension; just make sure you define the openHAB hostname and port in the extension settings and restart VS Code.
Java network access
The Java Virtual Machine hosting openHAB is restricted in it's permissions to interact on network level for security reasons. Some openHAB add-ons, like the Network or AmazonDash bindings, need elevated permissions to work. If needed, grand these permissions by executing the following command:
sudo setcap 'cap_net_raw,cap_net_admin=+eip cap_net_bind_service=+ep' $(realpath /usr/bin/java)
openHAB configuration fileshares
To allow remotely editing of the different configuration files and looking at the runtime files like logs, we'll setup two shares in Samba. Edit the Samba configiuration file
/etc/samba/smbd.conf and add two share definitions.
#======================= Share Definitions ======================= [openHAB2-user] comment=openHAB2 userdata path=/opt/openhab2/userdata browseable=Yes writeable=Yes only guest=no public=no create mask=0777 directory mask=0777 [openHAB2-conf] comment=openHAB2 site configuration path=/opt/openhab2/conf browseable=Yes writeable=Yes only guest=no public=no create mask=0777 directory mask=0777
The shares are configured to be not open for guests nor to the public. Let's activate the openhab user as a samba user and set his password next.
sudo smbpasswd -a openhab
Now we can access the openHAB configuration through the file share from a remote PC.
Setup the Karaf Console
With the Karaf console we can monitor the log in real-time, manage bundles and execute runtime commands. The console can be accessed locally on the system where openHAB is running with the command
$OPENHAB_RUNTIME/bin/client or via SSH. Out of the box, only the localhost interface is allowed. The default user for the console is openhab and default password is habopen. You should change the default password to something more secure.
sudo sed -i -e "s/openhab = .*,/openhab = <PASSWORD>,/g" /var/lib/openhab2/etc/users.properties
Next we will enable access to the console from the local network, disable automatic link creationg and auto inbox approval, and also set InfluxDB as the default persistence service. Enter the following in
# The region that should be used. org.eclipse.smarthome.core.localeprovider:region="NL" # The persistence service to use if no other is specified. org.eclipse.smarthome.persistence:default=influxdb # The karaf sshHost parameter configures the bind address for the ssh login to karaf. # Default is 127.0.0.1 (localhost), so it is only possible to login from the local machine. # Setting this to 0.0.0.0 will allow login from all network interfaces. org.apache.karaf.shell:sshHost = 0.0.0.0 # Setting this to true will automatically approve all inbox entries and create Things for them, # so that they are immediately available in the system (default is false) org.eclipse.smarthome.inbox:autoApprove=false # This setting allows to switch between a "simple" and an "advanced" mode for item management. # In advanced mode (autoLinks=false), gives full control over which items channels are linked to. # Existing links will remain untouched. (default is true) org.eclipse.smarthome.links:autoLinks=false
Start the console locally with the following command:
$ $OPENHAB_RUNTIME/bin/client Logging in as openhab __ _____ ____ ____ ____ ___ ____ / / / / | / __ ) / __ \/ __ \/ _ \/ __ \/ /_/ / /| | / __ | / /_/ / /_/ / __/ / / / __ / ___ |/ /_/ / \____/ .___/\___/_/ /_/_/ /_/_/ |_/_____/ /_/ 2.4.0-SNAPSHOT Build #1379 Hit '<tab>' for a list of available commands and '[cmd] --help' for help on a specific command. Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown openHAB. openhab>
You can access the console from another system in the local network using SSH with the following command:
ssh -p 8101 firstname.lastname@example.org
We now have a basic openHAB system running, anxious to perform all kinds of home automation stuff. But for now it's just a useless box making noise and generating heat. Only through addons we can unleash the real power of openHAB. We will dive into that the next time.