Categories
Linux

SSH Key Tutorial

SSH Key Tutorial

This post will be a basic SSH key tutorial. In my Securing this WordPress blog from evil hackers! post, I recommended turning off password based SSH authentication and moving exclusively to SSH key based authentication. This post will go over the steps to set up SSH key authentication. The outline is:

  1. Generate SSH public/private key pair
  2. Transfer the public key to the host in question
    1. manual method (Windows)
    2. automatic method (Linux/Mac)
  3. Verify the SSH key authentication is functional.

At the end of this post is my user creation script which does all of this automatically.

I do go over this in a good amount of detail in my first ever YouTube video which you can view here if you’d prefer a video instead of text.

I will be generating keys on my new laptop and transferring them to the Raspberry Pi I set up in my second YouTube video (Austin’s Nerdy Things Ep 2 – Setting up a Raspberry Pi) – https://youtu.be/u5dHvEYwr9M.

Step 1 – Generate the SSH key pair

The first thing we need to do is generate the SSH key pair. On all major OSes these days, this command is included (Windows finally joined the other “real” OSes in 2018 with full SSH support out of the box). We will be using ED25519, which is advised for basically anything build/updated since 2014.

ssh-keygen -t ed25519

This will start the generation process. I hit enter when it prompts for the location (it defaults to C:/Users/<user>/.ssh/id_<type> which is fine). I also hit enter again at the passphrase prompt twice because I don’t want to use a passphrase. Using a passphrase increase security but it can’t be used for any automated processes because it’ll prompt every time for the passphrase. This is the full output from the process:

C:\Users\Austin>ssh-keygen -t ed25519
Generating public/private ed25519 key pair.
Enter file in which to save the key (C:\Users\Austin/.ssh/id_ed25519): [press enter here]
Enter passphrase (empty for no passphrase): [press enter here]
Enter same passphrase again: [press enter here]
Your identification has been saved in C:\Users\Austin/.ssh/id_ed25519.
Your public key has been saved in C:\Users\Austin/.ssh/id_ed25519.pub.
The key fingerprint is:
SHA256:t0FIIk<snip.......................snip>6Rx4 austin@DESKTOP-3IPSAN3
The key's randomart image is:
+--[ED25519 256]--+
|o++++++ .        |
|*  + * o .       |
|o.o o   . .      |
| . .     .       |
|..... o S o      |
| . + .  .o.o..   |
|  = . o.+.+=o..  |
|   +.o.  +a..o   |
|  ..ooo o+...    |
+----[SHA256]-----+

Step 2 – Transfer the SSH public key

For Windows, they included all the useful SSH utilities except one: ssh-copy-id. This is the easy way to transfer SSH keys. It is possible to transfer the key without using this utility. For this SSH key tutorial, I will show both.

Step 2a – Transfer SSH key (Windows)

First we need to get the contents of the SSH public key. This will be done with the little-known type command of Windows. In Step 1, the file was placed at C:\Users\Austin\.ssh\id_ed25519 so that’s where we will read:

C:\Users\Austin>type C:\Users\Austin\.ssh\id_ed25519.pub
ssh-ed25519 AAAAC3NzaC1lZD2PT/VIK3KyYRliA0jgqmn7yzkVjuDiFK67Cio austin@DESKTOP-3IPSAN3

The contents of the file are the single line of the SSH public key (starting with the type of key “ssh-ed25519” and then continuing with the key contents “AAAAC3” and ending with the username and host the key was generated on “austin@DESKTOP-3IPSAN3”). This is the data we need to transfer to the new host.

I will first log onto the Raspberry Pi with the username/password combo from the default installation:

ssh pi@raspberrypi

I am now logged into the Pi:

Raspberry Pi Log In Prompt
Logged into the Raspberry Pi

Now on the target machine we need to create the user account and set up some initial things. We use sudo on all of these since root level permissions are required.

# create user austin with home directory (-m means create the home directory)
sudo useradd -m austin

# set the password for user austin
sudo passwd austin

# add user austin to sudo group
sudo usermod -aG sudo austin

# make the .ssh directory for user austin
sudo mkdir /home/austin/.ssh
sudo useradd
User created, password set, sudo added, and directory created

With those four commands run, the user is created, has a password, can run sudo, and has an empty .ssh directory ready for the keys.

Now we need to create the authorized_keys file:

sudo nano /home/austin/.ssh/authorized_keys

In this file, paste (by right-clicking) the single line from the id_ed25519 file:

nano authorized_keys
line pasted and ready to save

To save the file, press CTRL+O and a prompt will appear. Press enter again to save. Now to exit, press CTRL+X. There should be no “are you sure” prompt because the contents are saved.

Lastly, we need to change the owner of the .ssh directory and all of it’s contents to the new user (sudo means they were created as root):

sudo chown -R austin:austin /home/austin/.ssh

Now we can test. From the machine that generated the SSH key, SSH to the target with the new user specified:

ssh austin@raspberrypi

You should get in without having to type a password!

logged in with SSH key
Successfully logged in without typing a password (the SSH key worked)
Step 2b – Transfer SSH key (Linux)

This is much easier. The ssh-copy-id utility does what we just did in a single command.

# make sure you've already generated keys on your Linux/Mac machine
ssh-copy-id austin@raspberrypi
linux ssh-copy-id command
ssh-copy-id copied the keys and did everything for us in a single command

Microsoft – will you please add ssh-copy-id to default Windows installations?

Step 3 – Verify

We did this at the end of each of the other methods.

User creation script

This is the script I use on any new Linux virtual machine or container I spin up. It updates packages, installs a few things, sets the timezone, does all the user creation stuff and sets the SSH key. Lastly, it changes the shell to bash. Feel free to modify this for your own use. It is possible to copy the data from /etc/shadow so you don’t have to type in the password. I haven’t got that far yet.

apt update
apt upgrade -y 
apt install -y net-tools sudo htop sysstat git curl wget zsh
timedatectl set-timezone America/Denver
useradd -m austin
passwd austin
adduser austin sudo
usermod -aG sudo austin
usermod -aG adm austin
groups austin
mkdir /home/austin/.ssh
touch /home/austin/.ssh/authorized_keys
echo 'ssh-rsa AAAAB  <snip>  bhgqXzD austin@EARTH' >> /home/austin/.ssh/authorized_keys
chown -R austin:austin /home/austin/.ssh
usermod -s /bin/bash austin
Categories
Linux Python Weather

Python service to consume Ambient Weather API data

Python service to consume Ambient Weather API data

Continuing from the last post (Handling data from Ambient Weather WS-2902C to MQTT), we have a working script that reads the data coming from the Ambient Weather WS-2902 base station (Ambient Weather API) and sends it to a MQTT broker. In this post, we will turn that script into a Linux service that starts at boot and runs forever. This type of thing is perfect for Linux. Non-server Windows versions aren’t really appropriate for this since they reboot often with updates and such. If you want to run Windows Server, more power to you, you probably don’t need this guide. We will thus be focusing on Linux for consuming the Ambient Weather API. A Raspberry Pi is a perfect device for this (plenty of power, as big as a credit card, and less than $100).

Linux Services

Linux is made up of many individual components. Each component is designed to handle a single task (more or less). This differs from Windows where there are large executables/processes that handle many tasks. We will be taking advantage of Linux’s single task theory to add a new service that will run the Python script for ingesting and consuming the Ambient Weather API. In this instance for Austin’s Nerdy Things, the weather data is being provided by the Ambient Weather WS-2902C weather station.

Creating the service

For Ubuntu/Debian based distributions, service files live under /etc/systemd/system. Here is a list of services on the container I’m utilizing.

List command:

ls -1 /etc/systemd/system
dbus-org.freedesktop.resolve1.service
dbus-org.freedesktop.timesync1.service
default.target.wants
getty.target.wants
graphical.target.wants
multi-user.target.wants
socket.target.wants
sockets.target.wants
sshd.service
sys-kernel-debug.mount
sysinit.target.wants
syslog.service
timers.target.wants

Since this is a LXC container, there aren’t many services. On a standard Raspbian or Ubuntu full install, there will be 100+.

We will be creating a new service file using the nano text editor:

nano /etc/systemd/system/ambient-weather-api.service

In this file we need to define our service. The After lines mean don’t start this up until the listed services are running. The rest is pretty straight forward. I’m not 100% sure what the WantedBy line is for but it’s present in most of my service files. The contents of ambient-weather-api.service are as follows:

[Unit]
Description=Python script to ingest Ambient Weather API data
After=syslog.target
After=network.target

[Service]
Type=simple
User=root
Group=root
ExecStart=/usr/bin/python3 /srv/ambient-weather-api/main.py
ExecStartPre=/bin/sleep 5
Restart=always
RestartSec=5s
# Give the script some time to startup
TimeoutSec=10

[Install]
WantedBy=multi-user.target

Save the file. The service definition is looking for Python at /usr/bin/python3 and the python script at /srv/ambient-weather-api/main.py. You will probably be fine with the Python executable, but be sure to mv or cp the main.py file to /srv/ambient-weather-api/main.py.

We will need to reload the service definitions:

systemctl daemon-reload

Now we can start the service:

systemctl start ambient-weather-api.service

And verify it is running:

systemctl status ambient-weather-api.service
Ambient Weather API service status
Ambient Weather API service status

The above screenshot shows it is indeed running and active. It is still showing the print messages in the log as well, which we should disable by commenting out the lines by adding a # in front of the print line. In this case, it is coming from line 56 (the status check in the publish function).

#print(f"Sent {msg} to topic {topic}")

 

 

 

 

Categories
Home Assistant Weather

Handling data from Ambient Weather WS-2902C API to MQTT

Handling data from Ambient Weather WS-2902C API to MQTT

As I mentioned in my initial Ambient Weather WS-2902C post, there is a new feature that allows sending data to a custom server. I coded up a python script to take the sent data, and publish it to MQTT. This allows for super easy data ingestion with Home Assistant and other similar solutions. I should probably publish on GitHub but I’ll post here first.

Installation

This script is reliant on Paho-MQTT. EDIT: during the creation of the service to run this at boot, I discovered version 1.5.1 will throw errors. Use version 1.5.0. Install it with pip:

sudo pip install paho-mqtt==1.5.0

Create a python file and name it main.py. Paste in the following:

# Python script to decode Ambient Weather data (from WS-2902C and similar)
# and publish to MQTT.
# original author: Austin of austinsnerdythings.com
# publish date: 2021-03-20

# some resources I used include
#https://askubuntu.com/questions/29152/how-do-i-use-python-with-apache2
#https://www.toptal.com/python/pythons-wsgi-server-application-interface
#https://www.emqx.io/blog/how-to-use-mqtt-in-python

from urllib.parse import urlparse, parse_qs
import paho.mqtt.client as mqtt
import time, os

# set MQTT vars
MQTT_BROKER_HOST  = os.getenv('MQTT_BROKER_HOST',"mqtt")
MQTT_BROKER_PORT  = int(os.getenv('MQTT_BROKER_PORT',1883))
MQTT_CLIENT_ID    = os.getenv('MQTT_CLIENT_ID',"ambient_weather_decode")
MQTT_USERNAME     = os.getenv('MQTT_USERNAME',"")
MQTT_PASSWORD     = os.getenv('MQTT_PASSWORD',"")

# looking to get resultant topic like weather/ws-2902c/[item]
MQTT_TOPIC_PREFIX = os.getenv('MQTT_TOPIC_PREFIX',"weather")
MQTT_TOPIC 		  = MQTT_TOPIC_PREFIX + "/ws-2902c"

# mostly copied + pasted from https://www.emqx.io/blog/how-to-use-mqtt-in-python and some of my own MQTT scripts
def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print(f"connected to MQTT broker at {MQTT_BROKER_HOST}")
    else:
        print("Failed to connect, return code %d\n", rc)

def on_disconnect(client, userdata, flags, rc):
    print("disconnected from MQTT broker")

# set up mqtt client
client = mqtt.Client(client_id=MQTT_CLIENT_ID)
if MQTT_USERNAME and MQTT_PASSWORD:
    client.username_pw_set(MQTT_USERNAME,MQTT_PASSWORD)
    print("Username and password set.")
client.will_set(MQTT_TOPIC_PREFIX+"/status", payload="Offline", qos=1, retain=True) # set LWT     
client.on_connect = on_connect # on connect callback
client.on_disconnect = on_disconnect # on disconnect callback

# connect to broker
client.connect(MQTT_BROKER_HOST, port=MQTT_BROKER_PORT)
client.loop_start()

def publish(client, topic, msg):
    result = client.publish(topic, msg)
    # result: [0, 1]
    status = result[0]

    # uncomment for debug. don't need all the success messages.
    if status == 0:
        #print(f"Sent {msg} to topic {topic}")
        pass
    else:
        print(f"Failed to send message to topic {topic}")


def application(environ, start_response):
    # construct a full URL from the request. HTTP_HOST is FQDN, PATH_INFO is everything after
    # the FQDN (i.e. /data/stationtype=AMBWeatherV4.2.9&&tempinf=71.1&humidityin=35)
    url = "http://" + environ["HTTP_HOST"] + environ["PATH_INFO"]

    # unsure why I need to parse twice. probably just need to do it once with variable url.
    parsed = urlparse(url)
    result = parse_qs(parsed.geturl())

    # send to our other function to deal with the results.
    # result is a dict 
    handle_results(result)

    # we need to return a response. HTTP code 200 means everything is OK. other HTTP codes include 404 not found and such.
    start_response('200 OK', [('Content-Type', 'text/plain')])

    # the response doesn't actually need to contain anything
    response_body = ''

    # return the encoded bytes of the response_body. 
    # for python 2 (don't use python 2), the results don't need to be encoded
    return [response_body.encode()]


def handle_results(result):
    """ result is a dict. full list of variables include:
    stationtype: ['AMBWeatherV4.2.9'], PASSKEY: ['<station_mac_address>'], dateutc: ['2021-03-20 17:12:27'], tempinf: ['71.1'], humidityin: ['36'], baromrelin: ['29.693'],	baromabsin: ['24.549'],	tempf: ['58.8'], battout: ['1'], humidity: ['32'], winddir: ['215'],windspeedmph: ['0.0'],	windgustmph: ['0.0'], maxdailygust: ['3.4'], hourlyrainin: ['0.000'],	eventrainin: ['0.000'],	dailyrainin: ['0.000'],
    weeklyrainin: ['0.000'], monthlyrainin: ['0.000'], totalrainin: ['0.000'],	solarradiation: ['121.36'],
    uv: ['1'],batt_co2: ['1'] """

    # we're just going to publish everything. less coding.
    for key in result:
        # skip first item, which is basically a URL and MQTT doesn't like it. probably resulting from my bad url parsing.
        if 'http://' in key:
            continue

        #print(f"{key}: {result[key]}")
        # resultant topic is weather/ws-2902c/solarradiation
        specific_topic = MQTT_TOPIC + f"/{key}"

        # replace [' and '] with nothing. these come from the url parse
        msg = str(result[key]).replace("""['""", '').replace("""']""", '')
        #print(f"attempting to publish to {specific_topic} with message {msg}")
        publish(client, specific_topic, msg)

# this little guy runs a web server if this python file is called directly. if it isn't called directly, it won't run.
# Apache/Python WSGI will run the function 'application()' directly
# in theory, you don't need apache or any webserver. just run it right out of python. would need
# to improve error handling to ensure it run without interruption.
if __name__ == '__main__':
    from wsgiref.simple_server import make_server

    # probably shouldn't run on port 80 but that's what I specified in the ambient weather console
    httpd = make_server('', 80, application)
    print("Serving on http://localhost:80")

    httpd.serve_forever()

Execute with python3:

python3 main.py

Watch the results populate in the console window:

PS C:\Users\Austin\source\repos\ambient-weather-decode> python3 c:\Users\Austin\source\repos\ambient-weather-decode\main.py
connected to MQTT broker at mqtt
Serving on http://localhost:80
Sent E0:98:06:A3:42:65 to topic weather/ws-2902c/PASSKEY
Sent 2021-03-20 17:57:31 to topic weather/ws-2902c/dateutc
Sent 70.7 to topic weather/ws-2902c/tempinf
Sent 36 to topic weather/ws-2902c/humidityin
Sent 29.675 to topic weather/ws-2902c/baromrelin
Sent 24.531 to topic weather/ws-2902c/baromabsin
Sent 66.2 to topic weather/ws-2902c/tempf
Sent 1 to topic weather/ws-2902c/battout
Sent 26 to topic weather/ws-2902c/humidity
Sent 207 to topic weather/ws-2902c/winddir
Sent 0.2 to topic weather/ws-2902c/windspeedmph
Sent 1.1 to topic weather/ws-2902c/windgustmph
Sent 3.4 to topic weather/ws-2902c/maxdailygust
Sent 0.000 to topic weather/ws-2902c/hourlyrainin
Sent 0.000 to topic weather/ws-2902c/eventrainin
Sent 0.000 to topic weather/ws-2902c/dailyrainin
Sent 0.000 to topic weather/ws-2902c/weeklyrainin
Sent 0.000 to topic weather/ws-2902c/monthlyrainin
Sent 0.000 to topic weather/ws-2902c/totalrainin
Sent 697.92 to topic weather/ws-2902c/solarradiation
Sent 6 to topic weather/ws-2902c/uv
Sent 1 to topic weather/ws-2902c/batt_co2

Verification

Verify in MQTT by subscribing to topic ‘weather/#’. The # is a wildcard and will include all subtopics:

homeassistant ws-2902c mqtt messages
homeassistant ws-2902c mqtt messages

Conclusion

I am stoked the Ambient Weather WS-2902C makes it so easy to work the the data.

My next post will show how to turn this Python script into a persistent Linux service – Python service to consume Ambient Weather API data.

A further post will demonstrate incorporating these MQTT messages into Home Assistant sensors.

Categories
Weather

The Ambient Weather WS-2902C weather station!

The Ambient Weather WS-2902C

My new weather station came in the mail today. It is a Ambient Weather WS-2902C WiFi-Enabled Smart Weather Station. It was supposed to arrive tomorrow but I’m glad it showed up a day early, thanks Amazon! The newest firmware version also has a feature that blew me away – read on to find out what. I’m sure many others will appreciate it once they realize how powerful it is.

What I was looking for in a weather station

Before I bought a weather station, I decided on a few key requirements:

  • Basic weather data – outdoor temperature and humidity, wind speed and direction, and rain
  • The sensor station doesn’t need batteries replaced sooner than twice a year
  • A base station that shows the outdoor info with some additional inside info (inside temperature and humidity mostly)
  • The remote sensor station emits signals that could be decoded by rtl_433 so I could integrate it into my Home Automation/Smart Home system
  • Less than $200 (it is amazing how expensive some of these weather stations are! Looking at you, Davis Vantage Pro2, which is 3x the cost of the Ambient Weather WS-2902C as of writing this post)
Why I picked the WS-2902C

The Ambient Weather WS-2902C fit all of my requirements…. AND it has a few bonuses:

  • Additional sensors include: solar radiation (measured in watts per square meter) and pressure
  • Super capacitor charged by a small solar panel to really cut down on battery changes
  • Ability to interface with and send weather data to various weather services such as Weather Underground, Ambient Weather’s own service, and a few others
Initial thoughts

The sensor station is a nice little unit. I need to order a pole mount but it will be fine for a few days in my backyard. I am a but unsure why they put the solar panel on the north side of the solar sensor – it will surely block some sun. Here is what it looks like in my back yard (this snow is from the 4th largest snowstorm in Denver recorded history):

Ambient Weather WS-2902C testing
Ambient Weather WS-2902C sensor base in my backyard

The base station display looks crisp and the buttons are responsive and work as expected:

Ambient Weather WS-2902C base station
Ambient Weather WS-2902C base station in my kitchen

Nice bonus is the packaging and other packaging contents are almost entirely recyclable – nice work Ambient Weather!

Ambient Weather WS-2902C recyclable packaging
Ambient Weather WS-2902C recyclable packaging

Connecting it to my WiFi was straight-forward. The iOS app is showing 2 stars out of 5 in the Apple App Store. Not sure why – it worked fine for me.

The bonus feature

When loading up the app for the first time it listed firmware v4.2.8 as the latest. In the firmware release notes was an entry – Version 4.2.8 – adds path for custom server setup. This immediately caught my eye so I updated the firmware. Indeed the app did show an entry for a custom server that I immediately filled out even though I didn’t have hosts on my network named “weather”:

Ambient weather custom server
Ambient weather custom server screen

I had no idea what the format was but I guessed it was either an HTTP POST or GET. I created a quick LXC container in Proxmox named “weather”, installed apache2, and checked the access logs. Sure enough, it was hitting my web server! Further, it was valid, usable data!

10.98.1.132 - - [19/Mar/2021:20:20:13 +0000] "GET /data/stationtype=AMBWeatherV4.2.9&PASSKEY=<MAC_ADDRESS>&dateutc=2021-03-19+20:20:12&tempinf=70.3&humidityin=29&baromrelin=29.900&baromabsin=24.756&tempf=62.8&battout=1&humidity=31&winddir=188&windspeedmph=1.1&windgustmph=3.4&maxdailygust=5.8&hourlyrainin=0.000&eventrainin=0.000&dailyrainin=0.000&weeklyrainin=0.000&monthlyrainin=0.000&totalrainin=0.000&solarradiation=622.94&uv=6&batt_co2=1 HTTP/1.1" 404 467 "-" "ESP8266"
10.98.1.132 - - [19/Mar/2021:20:20:32 +0000] "GET /data/stationtype=AMBWeatherV4.2.9&PASSKEY=<MAC_ADDRESS>&dateutc=2021-03-19+20:20:31&tempinf=70.3&humidityin=29&baromrelin=29.894&baromabsin=24.750&tempf=62.8&battout=1&humidity=32&winddir=189&windspeedmph=2.5&windgustmph=3.4&maxdailygust=5.8&hourlyrainin=0.000&eventrainin=0.000&dailyrainin=0.000&weeklyrainin=0.000&monthlyrainin=0.000&totalrainin=0.000&solarradiation=620.73&uv=6&batt_co2=1 HTTP/1.1" 404 467 "-" "ESP8266"

This means I don’t even need to reconfigure my rtl_433 set up to add a listener for the Ambient Weather WS-2902C. It will send the data directly to my server! People have been poking around these for a while, trying to connect to the base stations (port 45000 is open on Telnet) which are ESP devices (likely ESP8266 or similar). Ambient Weather needs to promote this. This is a slam dunk for a ton of people in the Home Automation/weather nerd categories. This post is long enough, I’ll document how to parse the data out in a later post. Update: I have put together a python script to parse the data and send it to MQTT here – Handling data from Ambient Weather WS-2902C to MQTT

In summary

In summary, the Ambient Weather WS-2902C appears to check all my boxes for a weather station, be reasonable priced, and connects directly to my own server to post data without needing to decode the radio traffic. Initial score for this device is 10/10 (I was going to say 9/10 until I discovered the custom server feature. That easily adds +2 points).

Disclaimer

Austin’s Nerdy Things is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to amazon.com.

Categories
ADS-B

Receiving aircraft ADS-B (position) signals – part 3 (antenna reposition)

Welcome back from Receiving aircraft ADS-B (position) signals – part 2!

Yesterday I moved the antenna up a couple feet from a “very temporary” position to a “less temporary” position. I still need to get it up on the roof. Either way, my reception and max range have increased by at least 20%. It is still in my garage, which is a terrible location, but at least is elevated.

New antenna placement

ADS-B antenna in garage
ADS-B antenna in garage (we need a shed for all that gardening stuff)

The antenna needs to be vertical for maximum reception and upside down works just as well as right side up!

New antenna placement results

I saw up to 82 aircraft being tracked this morning. You can see the big planes lining up for southerly landings at KDEN on three separate runways and a few planes taking off to the southeast.

piaware updated antenna KDEN focus
PiAware updated antenna KDEN focus

Here is a screenshot zoomed out. I’ve got quite a few position reports from >100 nm out.

piaware ADS-B zoomed out KDENProposed final placement of the FlightAware ADS-B antenna

Proposed placement of ADS-B antenna on chimney
Proposed placement of ADS-B antenna on chimney (1 is where it currently is inside the garage and 2 is desired final placement)

I have a new weather station coming this weekend so I’ll try to combine roof trips to install the weather station and the 1090 MHz Flightaware antenna at the same time. I’ll also need to drill a hole in the side of the house to run the cable. Or maybe I do it all solar powered and use WiFi ?? I also have a 25W solar panel arriving this weekend. Keeping it physically isolated from the house would eliminate some lightning risk. Decisions, decisions. Be on the lookout for a part 4 with the results from the new roof placement! Part 4 – moving the antenna to the roof, has produced great results! I’m now seeing 150+ planes at once and getting over 700 messages per second.