Tag: mqtt

Background

If you are coming from the Handling data from Ambient Weather WS-2902C API to MQTT post, you are ready to proceed! If not, you’ll need to follow the steps in that post to get your Ambient Weather WS-2902C data into MQTT.

In short, we set up a Python script that listens on port 80 for data coming from the Ambient Weather base station. It then takes the data and publishes it to various MQTT topics. We can subscribe to those topics to receive new data as soon as it arrives.

Available Data

I’ll re-post the list of available data/topics from my last post:

Topic	Value	Comment
weather/ws-2902c/PASSKEY	aa:bb:cc:dd:ee:ff	MAC address
weather/ws-2902c/dateutc	5/15/2021	date
weather/ws-2902c/tempinf	70.7	temp at the base station
weather/ws-2902c/humidityin	36	humidity at the base station
weather/ws-2902c/baromrelin	29.675	adjusted barometric pressure (in Hg)
weather/ws-2902c/baromabsin	24.531	absolute barometric pressure (in Hg)
weather/ws-2902c/tempf	66.2	temp at the weather station
weather/ws-2902c/battout	1	battery status at the weather station?
weather/ws-2902c/humidity	26	humidity at the weather station
weather/ws-2902c/winddir	207	wind direction in degrees azimuth
weather/ws-2902c/windspeedmph	0.2	wind speed
weather/ws-2902c/windgustmph	1.1	wind gust (shows peaks between updates)
weather/ws-2902c/maxdailygust	3.4	max daily wind gust
weather/ws-2902c/hourlyrainin	0	hourly rain fall
weather/ws-2902c/eventrainin	0	event rain fall (resets after 24 hours of no rain)
weather/ws-2902c/dailyrainin	0	daily rain fall
weather/ws-2902c/weeklyrainin	0	weekly rain fall
weather/ws-2902c/monthlyrainin	0	monthly rain fall
weather/ws-2902c/totalrainin	0	total rain fall since power on?
weather/ws-2902c/solarradiation	697.92	solar radiation in watts per square meter
weather/ws-2902c/uv	6	UV intensity index
weather/ws-2902c/batt_co2	1	?

This is a lot of data. How much/little you want to use is up to you! I believe I added every topic to my Home Assistant so they’d would be available if I ever wanted to use them.

Adding the MQTT topics to Home Assistant

If you don’t have the MQTT line in your base configuration, make sure you add it. I also am using secrets here so it goes and grabs the broker IP address from my secrets file.

For my configuration.yaml file, showing the relevant lines (MQTT sensors and the sensor file)

homeassistant@ha-new:~/.homeassistant$ cat configuration.yaml

mqtt:
  broker: !secret mqtt_broker

sensor: !include sensor.yaml

For the secrets.yaml file:

homeassistant@ha-new:~/.homeassistant$ cat secrets.yaml
mqtt_broker: mqtt.home.fluffnet.net

The way the secrets file works means it looks like this in configuration.yaml:

mqtt:
  broker: mqtt.home.fluffnet.net

If you haven’t worked with .yaml in Home Assistant before, it is very picky about spacing. Ensure the spacing is correct (usually 2 spaces per indentation).

With the MQTT line added, we can turn to the sensors file (sensors.yaml). This is where the magic happens! I’ve only added a subset of the topics:

homeassistant@ha-new:~/.homeassistant$ cat sensor.yaml
[snip non-ambient weather sensors]
- platform: mqtt
  state_topic: "weather/ws-2902c/tempinf"
  name: "real kitchen temp"
  unit_of_measurement: "F"
- platform: mqtt
  state_topic: "weather/ws-2902c/tempf"
  name: "real outside temp"
  unit_of_measurement: "F"
- platform: mqtt
  state_topic: "weather/ws-2902c/humidityin"
  name: "real kitchen hum"
  unit_of_measurement: "%"
- platform: mqtt
  state_topic: "weather/ws-2902c/humidity"
  name: "real outside hum"
  unit_of_measurement: "%"
- platform: mqtt
  state_topic: "weather/ws-2902c/solarradiation"
  name: "solar radiation"
  unit_of_measurement: "W/m2"
- platform: mqtt
  state_topic: "weather/ws-2902c/dailyrainin"
  name: "daily rain"
  unit_of_measurement: "in"
- platform: mqtt
  state_topic: "weather/ws-2902c/windspeedmph"
  name: "wind speed"
  unit_of_measurement: "mph"
- platform: mqtt
  state_topic: "weather/ws-2902c/windgustmph"
  name: "wind gust"
  unit_of_measurement: "mph"

Restarting Home Assistant

With those lines added to the sensor.yaml file, restart Home Assistant. I love Home Assistant but needing to restart it for basically any configuration change is a huge pain.

sudo systemctl restart [email protected]

Adding the new Ambient Weather WS-2902C sensors to your Home Assistant screens

With the new sensors activated, you can add them to any of your Home Assistant pages!

First click the edit button then Add Card:

Next up we need to select what kind of card we want to add. For most of these, they’re time series, so History Graph will be the best choice. I do not know why Home Assistant is recommending the sun position in this screenshot.

Now that the History Graph is selected, we can pick any of the new sensors we added in the entity drop down. In this screenshot we see most of what I added. The others are sorted elsewhere (there are 100+ entities available in my entities drop down).

With sensor.real_outside_temp selected, I added 60 for the refresh interval, which means the graph will refresh itself every 60 seconds.

Now the graph is added to the page! You can repeat with all the other sensors you want to view. In the below screenshot, we have successfully added the outside temperature from the Ambient Weather WS-2902C to Home Assistant.

My full weather tab

I’ve added a number of sensors from my Ambient Weather WS-2902C to my Home Assistant. Below you can see I have solar radiation, daily rain, real outside humidity, real outside temp (I have another sensor labeled “outdoor temp” that is a floating sensor that is no longer outdoors), and the wind data. I also have the badges up top with just the current numeric value. You can add more or less, it’s totally up to you!

Conclusion

With this series, we have connected the Ambient Weather WS-2902C to our own Linux container to read the data, publish it to MQTT, and then view it in Home Assistant. I hope you’ve found this helpful!

Handling data from Ambient Weather WS-2902C API to MQTT

As I mentioned in my initial Ambient Weather WS-2902C post, there is a new feature that allows sending data to a custom server. I coded up a python script to take the sent data, and publish it to MQTT. This allows for super easy data ingestion with Home Assistant and other similar solutions. I should probably publish on GitHub but I’ll post here first.

Installation

This script is reliant on Paho-MQTT. EDIT: during the creation of the service to run this at boot, I discovered version 1.5.1 will throw errors. Use version 1.5.0. Install it with pip:

sudo pip install paho-mqtt==1.5.0

Create a python file and name it main.py. Paste in the following:

# Python script to decode Ambient Weather data (from WS-2902C and similar)
# and publish to MQTT.
# original author: Austin of austinsnerdythings.com
# publish date: 2021-03-20

# some resources I used include
#https://askubuntu.com/questions/29152/how-do-i-use-python-with-apache2
#https://www.toptal.com/python/pythons-wsgi-server-application-interface
#https://www.emqx.io/blog/how-to-use-mqtt-in-python

from urllib.parse import urlparse, parse_qs
import paho.mqtt.client as mqtt
import time, os

# set MQTT vars
MQTT_BROKER_HOST  = os.getenv('MQTT_BROKER_HOST',"mqtt")
MQTT_BROKER_PORT  = int(os.getenv('MQTT_BROKER_PORT',1883))
MQTT_CLIENT_ID    = os.getenv('MQTT_CLIENT_ID',"ambient_weather_decode")
MQTT_USERNAME     = os.getenv('MQTT_USERNAME',"")
MQTT_PASSWORD     = os.getenv('MQTT_PASSWORD',"")

# looking to get resultant topic like weather/ws-2902c/[item]
MQTT_TOPIC_PREFIX = os.getenv('MQTT_TOPIC_PREFIX',"weather")
MQTT_TOPIC 		  = MQTT_TOPIC_PREFIX + "/ws-2902c"

# mostly copied + pasted from https://www.emqx.io/blog/how-to-use-mqtt-in-python and some of my own MQTT scripts
def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print(f"connected to MQTT broker at {MQTT_BROKER_HOST}")
    else:
        print("Failed to connect, return code %d\n", rc)

def on_disconnect(client, userdata, flags, rc):
    print("disconnected from MQTT broker")

# set up mqtt client
client = mqtt.Client(client_id=MQTT_CLIENT_ID)
if MQTT_USERNAME and MQTT_PASSWORD:
    client.username_pw_set(MQTT_USERNAME,MQTT_PASSWORD)
    print("Username and password set.")
client.will_set(MQTT_TOPIC_PREFIX+"/status", payload="Offline", qos=1, retain=True) # set LWT     
client.on_connect = on_connect # on connect callback
client.on_disconnect = on_disconnect # on disconnect callback

# connect to broker
client.connect(MQTT_BROKER_HOST, port=MQTT_BROKER_PORT)
client.loop_start()

def publish(client, topic, msg):
    result = client.publish(topic, msg)
    # result: [0, 1]
    status = result[0]

    # uncomment for debug. don't need all the success messages.
    if status == 0:
        #print(f"Sent {msg} to topic {topic}")
        pass
    else:
        print(f"Failed to send message to topic {topic}")


def application(environ, start_response):
    # construct a full URL from the request. HTTP_HOST is FQDN, PATH_INFO is everything after
    # the FQDN (i.e. /data/stationtype=AMBWeatherV4.2.9&&tempinf=71.1&humidityin=35)
    url = "http://" + environ["HTTP_HOST"] + environ["PATH_INFO"]

    # unsure why I need to parse twice. probably just need to do it once with variable url.
    parsed = urlparse(url)
    result = parse_qs(parsed.geturl())

    # send to our other function to deal with the results.
    # result is a dict 
    handle_results(result)

    # we need to return a response. HTTP code 200 means everything is OK. other HTTP codes include 404 not found and such.
    start_response('200 OK', [('Content-Type', 'text/plain')])

    # the response doesn't actually need to contain anything
    response_body = ''

    # return the encoded bytes of the response_body. 
    # for python 2 (don't use python 2), the results don't need to be encoded
    return [response_body.encode()]


def handle_results(result):
    """ result is a dict. full list of variables include:
    stationtype: ['AMBWeatherV4.2.9'], PASSKEY: ['<station_mac_address>'], dateutc: ['2021-03-20 17:12:27'], tempinf: ['71.1'], humidityin: ['36'], baromrelin: ['29.693'],	baromabsin: ['24.549'],	tempf: ['58.8'], battout: ['1'], humidity: ['32'], winddir: ['215'],windspeedmph: ['0.0'],	windgustmph: ['0.0'], maxdailygust: ['3.4'], hourlyrainin: ['0.000'],	eventrainin: ['0.000'],	dailyrainin: ['0.000'],
    weeklyrainin: ['0.000'], monthlyrainin: ['0.000'], totalrainin: ['0.000'],	solarradiation: ['121.36'],
    uv: ['1'],batt_co2: ['1'] """

    # we're just going to publish everything. less coding.
    for key in result:
        # skip first item, which is basically a URL and MQTT doesn't like it. probably resulting from my bad url parsing.
        if 'http://' in key:
            continue

        #print(f"{key}: {result[key]}")
        # resultant topic is weather/ws-2902c/solarradiation
        specific_topic = MQTT_TOPIC + f"/{key}"

        # replace [' and '] with nothing. these come from the url parse
        msg = str(result[key]).replace("""['""", '').replace("""']""", '')
        #print(f"attempting to publish to {specific_topic} with message {msg}")
        publish(client, specific_topic, msg)

# this little guy runs a web server if this python file is called directly. if it isn't called directly, it won't run.
# Apache/Python WSGI will run the function 'application()' directly
# in theory, you don't need apache or any webserver. just run it right out of python. would need
# to improve error handling to ensure it run without interruption.
if __name__ == '__main__':
    from wsgiref.simple_server import make_server

    # probably shouldn't run on port 80 but that's what I specified in the ambient weather console
    httpd = make_server('', 80, application)
    print("Serving on http://localhost:80")

    httpd.serve_forever()

Execute with python3:

python3 main.py

Watch the results populate in the console window:

PS C:\Users\Austin\source\repos\ambient-weather-decode> python3 c:\Users\Austin\source\repos\ambient-weather-decode\main.py
connected to MQTT broker at mqtt
Serving on http://localhost:80
Sent E0:98:06:A3:42:65 to topic weather/ws-2902c/PASSKEY
Sent 2021-03-20 17:57:31 to topic weather/ws-2902c/dateutc
Sent 70.7 to topic weather/ws-2902c/tempinf
Sent 36 to topic weather/ws-2902c/humidityin
Sent 29.675 to topic weather/ws-2902c/baromrelin
Sent 24.531 to topic weather/ws-2902c/baromabsin
Sent 66.2 to topic weather/ws-2902c/tempf
Sent 1 to topic weather/ws-2902c/battout
Sent 26 to topic weather/ws-2902c/humidity
Sent 207 to topic weather/ws-2902c/winddir
Sent 0.2 to topic weather/ws-2902c/windspeedmph
Sent 1.1 to topic weather/ws-2902c/windgustmph
Sent 3.4 to topic weather/ws-2902c/maxdailygust
Sent 0.000 to topic weather/ws-2902c/hourlyrainin
Sent 0.000 to topic weather/ws-2902c/eventrainin
Sent 0.000 to topic weather/ws-2902c/dailyrainin
Sent 0.000 to topic weather/ws-2902c/weeklyrainin
Sent 0.000 to topic weather/ws-2902c/monthlyrainin
Sent 0.000 to topic weather/ws-2902c/totalrainin
Sent 697.92 to topic weather/ws-2902c/solarradiation
Sent 6 to topic weather/ws-2902c/uv
Sent 1 to topic weather/ws-2902c/batt_co2

Verification

Verify in MQTT by subscribing to topic ‘weather/#’. The # is a wildcard and will include all subtopics:

Conclusion

I am stoked the Ambient Weather WS-2902C makes it so easy to work the the data.

My next post will show how to turn this Python script into a persistent Linux service – Python service to consume Ambient Weather API data.

A further post will demonstrate incorporating these MQTT messages into Home Assistant sensors.