Home Automation on steroids

This is the first in a series of blog posts about my adventures into home automation (HA) land with openHAB and it will be primarily about building and preparing the Linux (Debian) based system. In next articles we will dive into the details of setting up a HA system with openHAB and some other tools. This is focused on my particular setup, but you will most likely pick-up a thing or two to apply to your own environment. Let's go to the process of getting a Linxu system up and running.

I use Debian Buster (10) for this project. Although it is still in testing phase, we all know how rigid the testing and release procedures for Debian are. And we don't need any special features, so for our HA project it will be more than adequate. Over the last few months I've had zero issues related to Debian 10 or its core packages.

Install Linux base system

This is rather straight foreward and not very specific for an HA setup. But, as we need it anyway, let's go and dive into the boring details quickly (and briefly). First, download the Debian Buster image here and put it on a bootable device, like a USB stick. Installation is quite straightforward and you can define a fixed IP address for the local network interface during install or use DHCP for now and change the network settings later.

Although it depends on your needs and preferences, some of the basic choices and settings to make during Debian initial setup tool are:

1. Use LVM for flexibility in disk layout and expansion.
2. See some of the parameters to use during setup in the table below.
3. Create an additional user named `myuser` (replace with your choice) for login, instead of `root`.
4. Define local repository images.
5. Only install SSH server and core packages.

Collect the basic configuration parameter for the setup, like:

Parameter Value
Domain name openhab.org
Host name ha
NIC enp3s0 (if you have multiple)
IP address 192.168.1.2
Subnet 192.168.1.0/24
Gateway 192.168.1.1
Broadcast 192.168.1.255
Name servers 192.168.1.1, 8.8.8.8

Our openHAB server gets IP address 192.168.1.2 and hostname ha.openhab.org in the description below. Replace with your own parameters.

Login as root, install sudo and add user myuser (replace with your own selected username) to the sudo group.

apt install sudo
usermod -a -G sudo myuser

Start the installed SSH daemon (if not already started), check the IP address and logout.

systemctl status sshd
systemctl start sshd
ip a
exit

Start PuTTY, or another SSH terminal application you prefer, and login as user myuser.

Since my server has Realtik NIC's and the Realtek firmware is part of the non-free packages repository in Debian, we must update the sources list for the Realtek NIC firmware and any other non-free stuff we want to install later, like unrar.

sudo nano /etc/apt/sources.list

Content of the sources.list file after being update:

deb http://ftp.nl.debian.org/debian/ stretch main contrib non-free
deb-src http://ftp.nl.debian.org/debian/ stretch main contrib non-free

deb http://security.debian.org/debian-security stretch/updates main contrib non-free
deb-src http://security.debian.org/debian-security stretch/updates main contrib nonfree

deb http://ftp.nl.debian.org/debian/ stretch-updates main contrib non-free
deb-src http://ftp.nl.debian.org/debian/ stretch-updates main contrib non-free

Now update the repository index and install the Realtek firmware.

sudo apt update
sudo apt install firmware-realtek

Upgrade the distribution and install some utilitities we will need later on.

sudo apt upgrade
sudo apt dist-upgrade
sudo apt install -y mc screen git zip unrar htop sysstat curl net-tools
sudo apt install -y apt-transport-https software-properties-common tcpdump

Setup the SSH service to use key authentication.

mkdir .ssh
chmod 700 .ssh
nano .ssh/authorized_keys

Paste the private key (can be generated by PuTTYgen) and change the security for the key file.

chmod 600 .ssh/authorized_keys

Edit the SSH daemon configuration to disable root login and password authentication.

sudo nano /etc/ssh/sshd_config
Port nnnn       <=== replace with ypour port#
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
ChallengeResponseAuthentication no

Restart the SSH daemon

sudo systemctl restart ssh

Test login with myuser via SSH (PuTTY).

Screen autodetach setting might not be enabled by default so it make sense to put a corresponding directive in your .screenrc

nano ~/.screenrc

Add:

autodetach on

Setup the network interface. First check what interfaces we have and which name has been assigned.

sudo ip a

To configure the network interface(s), type:

sudo nano /etc/network/interfaces

Note that we must use auto enp3s0 (replace with your NIC alias) because the systemd network-online.target only cares about interfaces that are listed as "auto" in /etc/network/interfaces, not allow hotplug.

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The local network interface
auto enp3s0
iface enp3s0 inet static
    address 192.168.1.2/24
    gateway 192.168.1.1
    # dns-* options are implemented by the resolvconf package, if installed
    dns-nameservers 192.168.1.1
    dns-search moerman.online
sudo service networking restart

Verify the configuration of the network and DNS settings.

sudo ip a
sudo cat /etc/resolv.conf

Define your networks and give the local network a recognizable name, I used mordor in this example.

sudo nano /etc/networks
default        0.0.0.0
loopback       127.0.0.0
link-local     169.254.0.0
mordor         192.168.1.0

Install additional tools

Although not needed to run an HA system, I like to be prepared and always install some additional tools to be able to detect potential issues. Let's start with iperf 3.1.3.

sudo apt-get remove iperf3 libiperf0
wget https://iperf.fr/download/ubuntu/libiperf0_3.1.3-1_amd64.deb
wget https://iperf.fr/download/ubuntu/iperf3_3.1.3-1_amd64.deb
sudo dpkg -i libiperf0_3.1.3-1_amd64.deb iperf3_3.1.3-1_amd64.deb
rm libiperf0_3.1.3-1_amd64.deb iperf3_3.1.3-1_amd64.deb

Optionally install some simple network monitoring tools.

sudo apt-get install iptraf iptraf-ng

Install and configure Samba, so we can access and edit the openHAB configuration remotely from a Windows PC.

sudo apt install -y samba cifs-utils

To configure Samba and allow creating simple shares later, type:

sudo nano /etc/samba/smb.conf

Most of the entries in smb.conf are not relevant for our configuration. You can cleanup if you like. The essential settings are:

[global]
   workgroup = MYWORKGROUP
   server role = standalone server
   obey pam restrictions = yes
   unix password sync = yes
   passwd program = /usr/bin/passwd %u
   passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\spassword:* %n\n *password\supdated\ssuccessfully* .
   pam password change = yes
   map to guest = bad user

To test the configuration run the following command:

sudo testparm

If all is ok, restart the smb service.

sudo systemctl restart smbd
sudo systemctl status smbd

Create a Samba password file for accessing the shares we will create later.

sudo nano /root/.smbcredentials
username=myuser
password=mypassword

Set the permissions more strict.

sudo chmod 640 /root/.smbcredentials

Just to be sure, reboot the system and check if there are no issues.

Install openHAB prerequisites

OpenHAB is built on the Eclipse Smart Home platform and runs on Java. So our first order of business is to install Java. Also we will be doing a lot of communication from sensors using MQTT which requires a MQTT broker to be installed. And since openHAB has limited built-in persistence functionality for state and history, we will install InfluxDB to store timeseries of sensor values.

Install Java

OpenHAB currently supports Java 8, not Java 9 or 10. There are multiple options available, including Oracle Java, but I opt for the Zulu open source Java SDK as it is a completely open source, fully certified Java SE compliant build of OpenJDK. For compatibility with openHAB Cloud, you should install at minimal revision 101. The current Java 8 version for Linux 64-bit can be found here. At the time of writing, openJDK version 1.8.0_163 was the current version.

Add the Zulu repository for Ubuntu.

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0xB1998361219BD9C9
sudo apt-add-repository 'deb http://repos.azulsystems.com/debian stable main'
sudo apt update

Install the Zulu package.

sudo apt install -y zulu-8

Check the installed Java version.

$ java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (Zulu 8.31.0.1-linux64) (build 1.8.0_181-b02)
OpenJDK 64-Bit Server VM (Zulu 8.31.0.1-linux64) (build 25.181-b02, mixed mode)
$ javac -version
javac 1.8.0_181

Install Mosquitto MQTT Server

Mosquitto is one of the more popular open source MQTT brokers and is well supported in the openHAB community. Install the Debian Mosquitto package from the repository.

sudo apt install -y mosquitto mosquitto-clients

By default, Debian will start the Mosquitto daemon after install. Let's first test the default configuration. We'll use one of the Mosquitto clients we just installed to subscribe to a topic on the broker. Topics are labels that you publish messages to and subscribe to. They are arranged as a hierarchy, so you could have sensors/outside/temp` and sensors/outside/humidity, for example. How you arrange topics is up to you and your needs. For now we will use a simple topic to test our configuration changes.

Open a second terminal (using screen). In the new terminal, use mosquitto_sub to subscribe to the test topic.

mosquitto_sub -h localhost -t test

The -h option is used to specify the hostname of the MQTT server, and -t specifies the topic name. There is no output after hitting ENTER because mosquitto_sub is waiting for messages to arrive. Switch back to the other terminal and publish a message:

mosquitto_pub -h localhost -t test -m "hello world"

The options for mosquitto_pub are the same as mosquitto_sub, though this time we use the additional -m option to specify the message to send. Hit ENTER, and the "hello world" message should pop up in the other terminal.

Enter CTRL+C in the second terminal to exit out of mosquitto_sub. Next, we'll secure the configuration using password-based authentication. Mosquitto includes a utility to generate a special password file called mosquitto_passwd. This command will prompt you to enter a password for the specified username, and place the results in /etc/mosquitto/passwd.

sudo mosquitto_passwd -c /etc/mosquitto/passwd <USERNAME>

Provide the chosen username and enter the password (twice) when asked. Next, open up a new configuration file for Mosquitto and tell it to use this password file to require logins for all connections.

sudo nano /etc/mosquitto/conf.d/default.conf

This should open an empty file. Paste in the following:

/etc/mosquitto/conf.d/default.conf
allow_anonymous false
password_file /etc/mosquitto/passwd

Note: be sure to put a trailing newline at the end of the file! allow_anonymous false will disable all non-authenticated connections, and the password_file option tells Mosquitto where to look for user and password information. Restart Mosquitto and test the changes.

sudo systemctl restart mosquitto

Switch to the second terminal window and subscribe to the 'test' topic, using the username and password just provided.

mosquitto_sub -h localhost -t test -u "<USERNAME>" -P "<PASSWORD>"

It should connect and sit, waiting for messages. You can leave this terminal open and connected for the rest of the tutorial, as we'll periodically send it test messages. Publish a message with the other terminal session, using the username and password.

mosquitto_pub -h localhost -t "test" -m "hello world" -u "<USERNAME>" -P "<PASSWORD>"

Be aware that we're sending passwords unencrypted over the local network, to connect over the internet, either VPN or SSL is recommended.

The configuration of Mosquitto is now ready to work with openHAB. Later on we'll configure openHAB to connect to it.

Install InfluxDB for persistence

Add the InfluxDB APT repository.

curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add -
echo "deb https://repos.influxdata.com/debian buster stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

Install and start the service.

sudo apt update && sudo apt install influxdb
sudo systemctl start influxdb

InfluxDB uses the configuration file /etc/influxdb/influxdb.conf and optional environment variables. If you do not uncomment a configuration option, the system uses its default setting. All the default settings can be viewed with the influxdb config command. We will disable reporting of usage statistics and change the API port in the [http] section of the configuration file to 8886. Also, enable API request logging for now to debug any potential setup issues (we'll disable it later on). And finally change the backup RPC endpoint to 8899, so it won't interfere with other services on the same host. The changed settings are automatically picked up by the InfluxDB service. But the first order of business is to create the admin user in InfluxDB:

$ influx -port '8086' -host localhost
> CREATE USER admin WITH PASSWORD '<PASSWORD>' WITH ALL PRIVILEGES
> EXIT

Update the /etc/influxdb/influxdb.ini configuration file.

...
reporting-disabled = true

# Bind address to use for the RPC service for backup and restore.
bind-address = "127.0.0.1:8899"

[http]
  # Determines whether HTTP endpoint is enabled.
  enabled = true
  # The bind address used by the HTTP service.
  bind-address = ":8886"
  # Determines whether user authentication is enabled over HTTP/HTTPS.
  auth-enabled = true
...

From then on, we need to specify the port, username and password on the commandline for InfluxDB, like this:

influx -port '8886' -username admin -password <PASSWORD> -host localhost

Restart the service and create a database and users for openHAB and Grafana.

> CREATE DATABASE openhab_db
> CREATE USER openhab_user WITH PASSWORD '<PASSWORD>'
> GRANT ALL ON openhab_db TO openhab_user
> CREATE USER grafana_user WITH PASSWORD '<PASSWORD>'
> GRANT READ ON openhab_db TO grafana_user
> EXIT

Install Grafana

Grafana does all the graphing stuff for us. First, the Grafana repository must be added to enable installing the Grafana package for Debian. Note that we use the Debian stretch repository for now, as there is no newer version available at the time of this writing.

echo "deb https://packagecloud.io/grafana/testing/debian/ stretch main" | sudo tee /etc/apt/sources.list.d/grafana.list

Then add the Package Cloud key. This allows you to install the signed Grafana packages.

curl https://packagecloud.io/gpg.key | sudo apt-key add -

Update the Debian Apt repositories and install Grafana.

sudo apt update && sudo apt install grafana

Enable and start Grafana by running:

sudo systemctl daemon-reload
sudo systemctl start grafana-server
sudo systemctl status grafana-server
sudo systemctl enable grafana-server

After successful installation you should be able to reach the Grafana dashboard at http://192.168.1.2:3000 with the default login admin:admin. You will be prompted to change the password. Change it to something more secure.

To connect Grafana to InfluxDB, go to “Add Data Source” and create a new source pointing to the InfluxDB database, providing the database name (openhab_db) and credentials chosen before (grafana_user).

Define InfluxDB as Grafana source

Disable user signup and enable anonymous access (for later image export) in the configuration file /etc/grafana/grafana.ini and restart the Grafana service.

[users]
# disable user signup / registration
allow_sign_up = false

[auth.anonymous]
# enable anonymous access
enabled = true

[auth.basic]
# disable basic authentication (enabled by default)
enabled = false

Install openHAB 2

After al these preparations we can finaly start the installation of openHAB. Let's first create a user openhab for running the daemon and make it safer by not allowing interactive login.

sudo adduser --system --no-create-home --group --disabled-login openhab

I choose to install openHAB2 manually, not from a .deb package. Download the platform independent archive file and extract it to the path /opt/openhab2. Choose between the latest Beta release or a Snapshot with all incoming contributions. As openHAB 2 is still in an evolving state, the snapshot may be the preferred choice. Download and extract the latest snapshot version of openHAB 2 from this download page.

cd /tmp
wget -O openhab-download.zip "https://openhab.ci.cloudbees.com/job/openHAB-Distribution/lastSuccessfulBuild/artifact/distributions/openhab/target/openhab-2.4.0-SNAPSHOT.zip"
sudo unzip openhab-download.zip -d /opt/openhab2
rm openhab-download.zip

The extracted openHAB files should belong to the earlier created openhab user. Execute:

sudo chown -hR openhab:openhab /opt/openhab2

Everything is ready for a first test run. Execute openHAB and you should be able to reach the openHAB 2 Dashboard at http://192.168.1.2:8080 (or use it's FQDN) after a few minutes:

sudo su -s /bin/bash -c '/opt/openhab2/start.sh' openhab

You will see the openHAB console in your terminal session and can directly interact with it. Please be aware, that openHAB 2 will need a few minutes so finish the first start, even after the openHAB console is visible. Let openHAB settle for 10-15 minutes. If the portal is not reachable by then, restart openHAB once.

Launching the openHAB runtime

                          __  _____    ____
  ____  ____  ___  ____  / / / /   |  / __ )
 / __ \/ __ \/ _ \/ __ \/ /_/ / /| | / __  |
/ /_/ / /_/ /  __/ / / / __  / ___ |/ /_/ /
\____/ .___/\___/_/ /_/_/ /_/_/  |_/_____/
    /_/                        2.4.0-SNAPSHOT
                               Build #1379

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown openHAB.

openhab>

Stop openHAB using the command system:shutdown. This will take several seconds.

To enable openHAB to run as a daemon and start automatically at boot, we leverage systemd and register openHAB as a service, so that it runs at startup and automatically restarts if openHAB crashes. The service will be running with the privileges of the user openhab and expects the openHAB files under /opt/openhab2/.

Create the file /lib/systemd/system/openhab2.service with the following content:

[Unit]
Description=The openHAB 2 Home Automation Bus Solution
Documentation=http://docs.openhab.org
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
User=openhab
Group=openhab
GuessMainPID=yes
WorkingDirectory=/opt/openhab2
ExecStart=/opt/openhab2/start.sh server
ExecStop=/bin/kill -SIGINT $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.target

Next, enable the service to be executed on system startup, start the service and retrieve status information:

sudo systemctl daemon-reload
sudo systemctl enable openhab2.service
sudo systemctl start openhab2.service
sudo systemctl status openhab2.service

The output of status after a successful execution should be similar to:

  openhab2.service - The openHAB 2 Home Automation Bus Solution
   Loaded: loaded (/lib/systemd/system/openhab2.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2018-10-03 20:47:19 CEST; 7s ago
     Docs: http://docs.openhab.org
  ...

Upgrade openHAB

To stay up to date with new releases, you should do regular upgrades of your manual installation. This is especially important if you are working with the latest snapshot as changes and fixes are incorporated constantly.

OpenHAB uses a script to update to any other version, or from stable to snapshot and visa-versa. Your personal configuration will be retained on upgrades, but you should stop openHAB and perform a backup first. From version 2.1.0 onwards, openHAB is distributed with the update script included. This script should be called from within openHAB's root directory. Assuming the openHAB base directory is /opt/openhab2/, simply run the following commands to update to the next major version of openHAB:

cd /opt/openhab
sudo runtime/bin/update

You can also specify any version as a parameter. For example, to switch back to 2.3.0:

sudo runtime/bin/update 2.3.0

You may also change to openHAB's more frequent, but a little less stable, snapshot builds. Just append -SNAPSHOT to the target version, like:

sudo runtime/bin/update 2.4.0-SNAPSHOT

OpenHAB file locations and backups for manual install

Files Location
openHAB application /opt/openhab2
Additional add-on files /opt/openhab2/addons
Site configuration /opt/openhab2/conf
Log files /opt/openhab2/userdata/logs
Userdata like rrd4j db /opt/openhab2/userdata
Backups folder /opt/openhab2/backups

OpenHAB supports the use of environment variables. These include:

Variable Default value
$OPENHAB_BACKUPS /opt/openhab2/backups
$OPENHAB_RUNTIME /opt/openhab2/runtime
$OPENHAB_CONF /opt/openhab2/conf
$OPENHAB_GROUP openhab
$OPENHAB_HOME /opt/openhab2
$OPENHAB_LOGDIR /opt/openhab2/userdata/logs
$OPENHAB_USER openhab
$OPENHAB_USERDATA /opt/openhab2/userdata
$OPENHAB_HTTP 8080
$OPENHAB_HTTPS 8443

It is recommended to make a backup of your configuration before any major change. To make a backup of openHAB, you need to retain your configuration and userdata files. OpenHAB 2 comes with scripts for storing your configuration in a zip file which is saved in /opt/openhab2/backups/ by default. You can change the default path by setting the $OPENHAB_BACKUPS environment variable.

To create a backup, stop openHAB and run the command:

sudo $OPENHAB_RUNTIME/bin/backup

To restore from the backup files, stop openHAB and run the command:

sudo $OPENHAB_RUNTIME/bin/restore $OPENHAB_BACKUPS/myBackup.zip

You can learn more about openHAB and how it works by looking at the log files. Execute the following command in one session or have both files separated in sessions side by side:

tail -f /opt/openhab2/userdata/logs/openhab.log -f /opt/openhab2/userdata/logs/events.log

openHAB basic configuration

To make openHAB work properly and interact with its environment, some basic tasks must be performed, including changing security settings and creating Samba shares to allow easy editing of the configuration from your remote PC. I recommend Visual Studio Code with the openHAB extension; just make sure you define the openHAB hostname and port in the extension settings and restart VS Code.

Java network access

The Java Virtual Machine hosting openHAB is restricted in it's permissions to interact on network level for security reasons. Some openHAB add-ons, like the Network or AmazonDash bindings, need elevated permissions to work. If needed, grand these permissions by executing the following command:

sudo setcap 'cap_net_raw,cap_net_admin=+eip cap_net_bind_service=+ep' $(realpath /usr/bin/java)

openHAB configuration fileshares

To allow remotely editing of the different configuration files and looking at the runtime files like logs, we'll setup two shares in Samba. Edit the Samba configiuration file /etc/samba/smbd.conf and add two share definitions.

#======================= Share Definitions =======================
[openHAB2-user]
  comment=openHAB2 userdata
  path=/opt/openhab2/userdata
  browseable=Yes
  writeable=Yes
  only guest=no
  public=no
  create mask=0777
  directory mask=0777

[openHAB2-conf]
  comment=openHAB2 site configuration
  path=/opt/openhab2/conf
  browseable=Yes
  writeable=Yes
  only guest=no
  public=no
  create mask=0777
  directory mask=0777

The shares are configured to be not open for guests nor to the public. Let's activate the openhab user as a samba user and set his password next.

sudo smbpasswd -a openhab

Now we can access the openHAB configuration through the file share from a remote PC.

Setup the Karaf Console

With the Karaf console we can monitor the log in real-time, manage bundles and execute runtime commands. The console can be accessed locally on the system where openHAB is running with the command $OPENHAB_RUNTIME/bin/client or via SSH. Out of the box, only the localhost interface is allowed. The default user for the console is openhab and default password is habopen. You should change the default password to something more secure.

sudo sed -i -e "s/openhab = .*,/openhab = <PASSWORD>,/g" /var/lib/openhab2/etc/users.properties

Next we will enable access to the console from the local network, disable automatic link creationg and auto inbox approval, and also set InfluxDB as the default persistence service. Enter the following in /opt/openhab2/conf/services/runtime.cfg:

# The region that should be used.
org.eclipse.smarthome.core.localeprovider:region="NL"

#  The persistence service to use if no other is specified.
org.eclipse.smarthome.persistence:default=influxdb

# The karaf sshHost parameter configures the bind address for the ssh login to karaf.
# Default is 127.0.0.1 (localhost), so it is only possible to login from the local machine.
# Setting this to 0.0.0.0 will allow login from all network interfaces.
org.apache.karaf.shell:sshHost = 0.0.0.0

# Setting this to true will automatically approve all inbox entries and create Things for them,
# so that they are immediately available in the system (default is false)
org.eclipse.smarthome.inbox:autoApprove=false

# This setting allows to switch between a "simple" and an "advanced" mode for item management.
# In advanced mode (autoLinks=false), gives full control over which items channels are linked to.
# Existing links will remain untouched. (default is true)
org.eclipse.smarthome.links:autoLinks=false

Start the console locally with the following command:

$ $OPENHAB_RUNTIME/bin/client

Logging in as openhab

                          __  _____    ____
  ____  ____  ___  ____  / / / /   |  / __ )
 / __ \/ __ \/ _ \/ __ \/ /_/ / /| | / __  |
/ /_/ / /_/ /  __/ / / / __  / ___ |/ /_/ /
\____/ .___/\___/_/ /_/_/ /_/_/  |_/_____/
    /_/                        2.4.0-SNAPSHOT
                               Build #1379

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown openHAB.

openhab>

You can access the console from another system in the local network using SSH with the following command:

ssh -p 8101 openhab@192.168.1.2

We now have a basic openHAB system running, anxious to perform all kinds of home automation stuff. But for now it's just a useless box making noise and generating heat. Only through addons we can unleash the real power of openHAB. We will dive into that the next time.

Add a comment

Next Post Previous Post

Add a comment