Author: Austin

Categories
Home Assistant Weather

Viewing Ambient Weather WS-2902C data in Home Assistant

Background

If you are coming from the Handling data from Ambient Weather WS-2902C API to MQTT post, you are ready to proceed! If not, you’ll need to follow the steps in that post to get your Ambient Weather WS-2902C data into MQTT.

In short, we set up a Python script that listens on port 80 for data coming from the Ambient Weather base station. It then takes the data and publishes it to various MQTT topics. We can subscribe to those topics to receive new data as soon as it arrives.

Available Data

I’ll re-post the list of available data/topics from my last post:

TopicValueComment
weather/ws-2902c/PASSKEYaa:bb:cc:dd:ee:ffMAC address
weather/ws-2902c/dateutc5/15/2021date
weather/ws-2902c/tempinf70.7temp at the base station
weather/ws-2902c/humidityin36humidity at the base station
weather/ws-2902c/baromrelin29.675adjusted barometric pressure (in Hg)
weather/ws-2902c/baromabsin24.531absolute barometric pressure (in Hg)
weather/ws-2902c/tempf66.2temp at the weather station
weather/ws-2902c/battout1battery status at the weather station?
weather/ws-2902c/humidity26humidity at the weather station
weather/ws-2902c/winddir207wind direction in degrees azimuth
weather/ws-2902c/windspeedmph0.2wind speed
weather/ws-2902c/windgustmph1.1wind gust (shows peaks between updates)
weather/ws-2902c/maxdailygust3.4max daily wind gust
weather/ws-2902c/hourlyrainin0hourly rain fall
weather/ws-2902c/eventrainin0event rain fall (resets after 24 hours of no rain)
weather/ws-2902c/dailyrainin0daily rain fall
weather/ws-2902c/weeklyrainin0weekly rain fall
weather/ws-2902c/monthlyrainin0monthly rain fall
weather/ws-2902c/totalrainin0total rain fall since power on?
weather/ws-2902c/solarradiation697.92solar radiation in watts per square meter
weather/ws-2902c/uv6UV intensity index
weather/ws-2902c/batt_co21?
table showing available data topics from our Ambient Weather WS-2902C weather station

This is a lot of data. How much/little you want to use is up to you! I believe I added every topic to my Home Assistant so they’d would be available if I ever wanted to use them.

Adding the MQTT topics to Home Assistant

If you don’t have the MQTT line in your base configuration, make sure you add it. I also am using secrets here so it goes and grabs the broker IP address from my secrets file.

For my configuration.yaml file, showing the relevant lines (MQTT sensors and the sensor file)

homeassistant@ha-new:~/.homeassistant$ cat configuration.yaml

mqtt:
  broker: !secret mqtt_broker

sensor: !include sensor.yaml

For the secrets.yaml file:

homeassistant@ha-new:~/.homeassistant$ cat secrets.yaml
mqtt_broker: mqtt.home.fluffnet.net

The way the secrets file works means it looks like this in configuration.yaml:

mqtt:
  broker: mqtt.home.fluffnet.net

If you haven’t worked with .yaml in Home Assistant before, it is very picky about spacing. Ensure the spacing is correct (usually 2 spaces per indentation).

With the MQTT line added, we can turn to the sensors file (sensors.yaml). This is where the magic happens! I’ve only added a subset of the topics:

homeassistant@ha-new:~/.homeassistant$ cat sensor.yaml
[snip non-ambient weather sensors]
- platform: mqtt
  state_topic: "weather/ws-2902c/tempinf"
  name: "real kitchen temp"
  unit_of_measurement: "F"
- platform: mqtt
  state_topic: "weather/ws-2902c/tempf"
  name: "real outside temp"
  unit_of_measurement: "F"
- platform: mqtt
  state_topic: "weather/ws-2902c/humidityin"
  name: "real kitchen hum"
  unit_of_measurement: "%"
- platform: mqtt
  state_topic: "weather/ws-2902c/humidity"
  name: "real outside hum"
  unit_of_measurement: "%"
- platform: mqtt
  state_topic: "weather/ws-2902c/solarradiation"
  name: "solar radiation"
  unit_of_measurement: "W/m2"
- platform: mqtt
  state_topic: "weather/ws-2902c/dailyrainin"
  name: "daily rain"
  unit_of_measurement: "in"
- platform: mqtt
  state_topic: "weather/ws-2902c/windspeedmph"
  name: "wind speed"
  unit_of_measurement: "mph"
- platform: mqtt
  state_topic: "weather/ws-2902c/windgustmph"
  name: "wind gust"
  unit_of_measurement: "mph"

Restarting Home Assistant

With those lines added to the sensor.yaml file, restart Home Assistant. I love Home Assistant but needing to restart it for basically any configuration change is a huge pain.

sudo systemctl restart [email protected]

Adding the new Ambient Weather WS-2902C sensors to your Home Assistant screens

With the new sensors activated, you can add them to any of your Home Assistant pages!

First click the edit button then Add Card:

home assistant screenshot to add card

Next up we need to select what kind of card we want to add. For most of these, they’re time series, so History Graph will be the best choice. I do not know why Home Assistant is recommending the sun position in this screenshot.

select History Graph

Now that the History Graph is selected, we can pick any of the new sensors we added in the entity drop down. In this screenshot we see most of what I added. The others are sorted elsewhere (there are 100+ entities available in my entities drop down).

New sensors available to add so we can view data from the WS-2902C
60 means it will refresh every minute (60 seconds)

With sensor.real_outside_temp selected, I added 60 for the refresh interval, which means the graph will refresh itself every 60 seconds.

Now the graph is added to the page! You can repeat with all the other sensors you want to view. In the below screenshot, we have successfully added the outside temperature from the Ambient Weather WS-2902C to Home Assistant.

My full weather tab

I’ve added a number of sensors from my Ambient Weather WS-2902C to my Home Assistant. Below you can see I have solar radiation, daily rain, real outside humidity, real outside temp (I have another sensor labeled “outdoor temp” that is a floating sensor that is no longer outdoors), and the wind data. I also have the badges up top with just the current numeric value. You can add more or less, it’s totally up to you!

Conclusion

With this series, we have connected the Ambient Weather WS-2902C to our own Linux container to read the data, publish it to MQTT, and then view it in Home Assistant. I hope you’ve found this helpful!

Categories
DIY Offgrid Solar

DIY solar with battery backup – up and running!

Update

As of 5/13/2021, I have everything up and running! My DIY solar system with battery backup has produced 5.9 kWh over the last few days. It has been very wet and rainy this spring here in the Front Range of the Rockies and I don’t think we’ve had a full day of sunshine in a couple weeks. In terms of dollars, that 5.9 kWh is worth $0.649. Not much, but knowing I can power the whole house for 6 hours (based on average consumption) is a pretty good feeling. Check out my last post for some detail on how I got the battery bank set up – DIY Solar System with battery backup update.

Battery bank

The battery bank has performed great so far. I am in the process of doing a full discharge test. These cells did each come with a discharge test sheet from Battery Hookup, all showing more than 260Ah (most around 262Ah). Total for the batteries was $1082.

8x260Ah LiFePO4 cell battery bank ready for BMS

Under discharge in the flat part of the curve, the cells are extremely close in voltage. Below is a screenshot of my BMS app (XiaoxiangBMS, which is a super handy app. the fastest I’ve ever spent $6 on the pro version of an app) showing minor discharge current and the cells are within 6 millivolts of each other. It really doesn’t get much better than that.

BMS app (I am not creative enough to name my BMS the U1timatron) showing closely matched cell voltages during discharge

BMS

The BMS is a JBD 8s 100A model I got from eBay that comes with bluetooth monitoring ability. It has worked flawlessly so far. I love it when technology just works. Every setting can be configured. It keeps track of every alarm event. It has temperature protection. Great BMS.

JBD 8S 100A Battery monitoring system (BMS) keeping tabs on my LiFePO4 cells
BMS showing a modest discharge. I’ve taken it to 1200W so far.

Inverter

The all-in-one inverter (inverter, battery charger, MPPT solar controller) is a MPP Solar LV2424 hybrid model. Everything works as expected. I’ve tested all the features successfully. I even got the monitoring software going. The main downside so far to this inverter (and I think this is common) is a relatively high idle power consumption. The BMS reports ~56-60W draw with the inverter on and nothing plugged in. The “idle” power draw decreases as a proportion of the total load on the inverter but it never drops to 0. For example, at 0 load, the inverter draws 60W. At 120W load (based on a questionable Kill-A-Watt), the inverter is drawing 160W, meaning the “idle” power dropped to 40W. This inverter was $665.

Inverter mounted above the battery bank. The solar breaker is to the left.

Solar panels

My $100 each solar panels from Craigslist have been hooked up on and off over the last few days. I haven’t seen more than 480W from the 2 of them combined but they are laying completely flat (they should be tilted towards the sun for maximum power). Overall, happy for the price. I need to get them mounted on the roof.

The main testing location on the south side of our house. They’re basically flat, maybe tilted a degree or two to the south. I haven’t seen more than 480W from them (rated for 2×310=620W) yet but they were cheap so I’m not too disappointed.
Inverter showing the panels putting out 26W on a very cloudy day recently. It has apparently decided that 73V was optimal for whatever conditions were present.

What’s next

My DIY solar system with battery backup is commissioned! Things are functional. Things aren’t located optimally. I need to get the solar panels mounted on the roof and do some tidying around the batteries/inverter. I plan on mounting some drywall above the battery cells to protect from whatever and covering up all the battery terminals. The rear of the cells are covered. The mains are covered. Some of the intermediate terminals are not.

I’m already looking on Craigslist for more solar panels, but I need to 100% finish this project before adding on to it according to my lovely wife (I am sure some of you know exactly what I’m talking about!). With that, I’ll be signing off for the night. The baby is having a very hard time falling asleep tonight. Until next time!

Categories
Blog Admin

Fixing a NGINX WordPress HTTP 302 redirect loop

Causes of a HTTP 302 redirect loop

HTTP code 302 redirect loops can be caused by a number of things. Most of them tend to be caused by a misconfigured forward or reverse proxy (Apache, NGINX, HAProxy, etc.). In my case, I am in the process of migrating my austinsnerdythings.com WordPress blog from a single tier (NGINX) stack to a 3-tier application stack. This 3-tier stack consists of HAProxy in the front, Varnish in the middle, and NGINX in the back. A simple header setting missing from my NGINX config caused me to spend about an hour figuring out what was causing my HTTP 302 redirect loop.

I created a video version of how to fix a HTTP 302 redirect loop if you’d like to view that instead of reading on – https://www.youtube.com/watch?v=wRxfZX4Wzzo.

Symptoms of a HTTP 302 redirect loop

Upon the first visit to my dev site immediately after enabling SSL in the HAProxy config, I was presented with a Firefox error stating “The page isn’t redirecting properly – An error occurred during a connection to dev.austinsnerdythings.com. *this problem can sometimes be caused by disabling or refusing to accept cookies.”. A screenshot of the error (and associated network requests) is presented below. Firefox apparently retried the request 20 times after the initial redirect.

screenshot showing firefox error code and 302 redirect network requests
screenshot of HTTP 302 code with associated redirect loop

What’s really going on here

The 3 tiers I’ve selected for my stack each play a distinct role in serving you this webpage. The 3 tiers are:

  • HAProxy – HAProxy is the first application to see any request to austinsnerdythings.com. It handles SSL (the s in https) and that’s about it. In the future, I can use it to make my site highly available (the HA in HAProxy) and fully redundant but that’s a definitely overkill for now. After dealing with the SSL, it hands off the regular http request to Varnish. HAProxy is fast.
  • Varnish – is a caching application. That’s all it does. It doesn’t do SSL, which is why we need to stick HAProxy in front. If a page or asset (.js, .css, etc.) hasn’t been accessed recently, Varnish sends the http request to the webserver and stores the result and forwards it back to the original requester (via HAProxy). If a page or asset has been accessed recently, it is stored in memory and is flipped back to HAProxy without even touching the webserver. Varnish is fast.
  • NGINX – is a event-driven webserver. It takes the http request and fulfills it according to the configuration. For any .php file (basically anything in WordPress), this means sending it to PHP-FPM so the Wordprss code can be executed and a result produced to hand back to NGINX and sent up the tiers.

The request is technically http (not SSL) from exiting HAProxy, through Varnish, and into NGINX. WordPress has at least two features that attempt to send http requests to the equivalent https request. WordPress became unhappy that it was receiving a http request from Varnish (via NGINX) and it turned around and said “don’t use http://dev.austinsnerdythings.com, use https://dev.austinsnerdythings.com”! Varnish and NGINX don’t want https requests. The competing requests turned into the redirect loop.

How to fix the HTTP 302 redirect loop

The solution is pretty easy: you just need to add a single line to your NGINX site config file inside the php block:

fastcgi_param HTTPS 1;

This forces the headers to show the request is in fact a HTTPS request. WordPress is perfectly happy with that and doesn’t try any funny business returning 302 codes.

My full NGINX php block (with credit to where I got this from, nyxi.eu) looks like this:

location ~ \.php$ {
	# https://nyxi.eu/wordpress-nginx-redirect-loop.html
	fastcgi_param HTTPS 1;

	fastcgi_split_path_info ^(.+\.php)(/.+)$;
	fastcgi_pass unix:/run/php/php7.4-fpm.sock;
	fastcgi_index index.php;
	fastcgi_send_timeout 300s;
	fastcgi_read_timeout 300s;
	include fastcgi_params;
	fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
	fastcgi_buffer_size 128k;
	fastcgi_buffers 256 4k;
	fastcgi_busy_buffers_size 256k;
	fastcgi_temp_file_write_size 256k;
	fastcgi_intercept_errors on;
}

Results

After making the changes and reloading NGINX, I attempted to log in again and was presented with a normal dashboard view with happy, green HTTP 200 codes!

screenshot showing successful loading of the wp-admin/wordpress dashboard
expected screen after attempting to log in after solving the 302 redirect loop

References

Huge shout out to Emil Flink and his post WordPress Nginx redirect loop which really got me pointed in the right direction. He broke down exactly what was happening throughout WordPress’ underlying code and presented it in a a very easy to interpret method.

Categories
Blog Admin

My Fast WordPress Page Load Speeds – Part 1

How fast is fast enough?

People trust faster loading web sites than slower ones, all else being equal. There are many articles and studies saying so. It’s also better for search engine optimization (SEO). Personally, if a site takes more than 5 seconds to load, I wonder why and open the page source and start looking into it. I knew when I made austinsnerdythings.com that I wanted it to have a super fast WordPress page load speed.

Initial goal – less than 1 second for page load speed, and >90 for page speed tests

I decided on a initial target of loading in under a second. How did I achieve this target? It wasn’t scientific. Sites that load in 2 seconds or more are noticeable. For sites that load in under a second – it is hard to tell if the site loads in 0.6 seconds or 0.8. One second just felt like a good goal. I also wanted the 3 major page speed test sites to show a 90% or better.

Results – consistently fast WordPress page load speeds and 90+ on page speed tests

This site consistently loads in under a second. Here is a screenshot from the evening of writing this post showing a load time of 0.619 seconds. The server is located in New York City and I’m loading the page in Denver. That’s 50 milliseconds of ping by itself.

619 millisecond load time on my main PC

I also score 90+ on each of the big 3 page speed test sites – Google Page Speed Insights, GTMatrix, and Pingdom.

Here is the GTMatrix page speed result showing a strong 99% for performance. Note that this test was conducted from Vancouver, which is nearly 3000 miles away. That’s a lot of distance for the packets to travel.

99% for performance. Can’t argue with that

For Google Page Speed Insight, I score a perfect 100 for desktop load speed. Mobile is 90, not sure why they’re so different. I don’t think I need to worry about this anytime soon.

The elusive 100 for Google PageSpeed Insights
PageSpeed reporting 90 for mobile version? Must be some sort of artificial limit on performance or something.

For Pingdom, it is reporting I need to make less HTTP calls. I experimented with a couple different plugins and I think I need to go back to a different one for merging the site’s assets to reduce calls.

I’ve never seen E for a grade. Guessing it basically means F. I should improve on this.

It wasn’t particularly difficult to achieve the 90+ page speed scores and fast WordPress page load speeds. Read on to find out how I did this.

Austinsnerdythings.com stack

The stack driving this site is pretty standard. I use the following (ascending layer order)

  • Ramnode premium VPS ($12/month). I had a standard VPS but realized a faster CPU would provide better time to first byte and be just faster in general. It also has NVMe drives instead of SATA SSD, which further reduces latency.
  • Dual stack networking – IPv6 enabled as well as IPv4
  • MariaDB (MySQL drop in replacement) – zero tuning
  • PHP-FPM – interprets all dynamic requests, which is basically all the requests. zero tuning
  • NGINX webserver – this is the new hotness for webservers. It is event driven and runs fast. zero tuning
  • GZIP compression – much of the content can be compressed while being transferred which means faster load times
  • HTTP/2 – the newest generation of HTTP transfer protocol. not sure what’s faster about it than HTTP/1.1 but it is
  • Fast velocity minify plugin – combines javascript and CSS files into fewer entities which means faster load time
  • No extra plugins – don’t load up on plugins. some are really bad and will drastically increase page load times.
  • Lazy load pictures – no need to have pictures load until users get to them
  • Don’t embed videos – they just take forever to load up, which is a problem for page load speeds

Overall, this is a pretty standard stack. I haven’t done any manual tuning to any of the services/processes serving up my site. Starting with a fast VPS definitely helps get a fast WordPress page load speed. The rest just seemed logical to me. That doesn’t mean there isn’t room for improvement.

What’s next for an even faster WordPress page load speed

I have a development environment mocked up on my Proxmox virtual machine host that mimics almost exactly the production site (i.e. what served you this page). The page load speeds were roughly the same locally vs from NYC so it is a good comparison. I installed Varnish for caching and HAProxy for SSL termination (Varnish doesn’t do SSL/https). My homepage loaded consistently in a quarter of a second. The fastest I saw was 0.219 seconds. Sure Varnish and HAProxy are two more full-blown services to install and manage but is it worth it? I think it is. I’ll be migrating my production stack over to this 3 tier stack soon.

Further, I can add my content to a content delivery network (CDN), like Cloudflare, and have it cached there too for fast access anywhere they have a datacenter (they have like 200 datacenters across the planet). I did use Cloudflare CDN for a bit and didn’t like not seeing my statistics update in real-time so I backed off.

I am realizing this may be a good business opportunity – consistently super fast WordPress sites. Let me know if you’d be interested.

Check back for part 2 where I document the journey to the 3 tier stack.

Categories
DIY Offgrid Solar

DIY Solar System with battery backup update

Update

As of 5/1/2021, all of the main components of my DIY solar system with battery backup have arrived. I posted about the requirements, component select, and some fun with shipping from China in my initial solar post – Planning my 600W DIY solar system with 6 kWh battery backup. If you want some background of how we got here, head to that link then come on back.

Background

To recap, my DIY solar system with battery backup consists of a few main components:

  • 8x 260 amp hour prismatic LiFePO4 battery cells. They will be placed in series for a 24V nominal system with around 6.0 kilowatt hours of usable storage.
  • MPP LV2424 hybrid all-in-one inverter. This device handles converting direct current (DC, like a car battery) to alternating current (AC, like household outlets) and charging the batteries
  • 2x 310W Canadian solar panels. These will be wired in series for 72V maximum power point voltage.
  • 8S JBD 100A battery management system – to protect the batteries from a number of undesirable conditions

The other miscellaneous things that I need are: battery bus bars, wiring, ring terminals, and general connection things.

Materials arriving and resting battery voltages

We were on vacation when the batteries (and inverter) arrived so they sat for a few days before I got a chance to unbox them. The batteries were very well packaged and I can’t thank Battery Hookup enough for how fast they shipped after what I’ve been dealing with from China.

battery cells unpacked
battery cells unpacked

Upon unboxing, I made sure to record the resting voltage of each cell. Below were the resting voltages:

cellvoltage
13.341
23.345
33.435
43.339
53.351
63.376
73.343
83.363
min3.339
max3.435
avg3.362
delta0.096
Resting voltages of the 260 amp hour BYD LiFePO4 cells

Cell #2 had the lowest voltage and cell #3 the highest. This presented an easy chance to test out my bus bars for voltage equalization. I did not measure the cell internal resistance so I wasn’t really sure how much current would flow from cell 3 to 2 when shorted together so I did a quick estimate based on Ohm’s law (V=IR -> I=V/R). With an estimated internal resistance (IR) of 20 milliohms (I’ve had LiPo in this range after some degradation), and a voltage difference of 0.096V, that would mean a current of 0.096/0.020=4.8A. That wasn’t a huge number so I was comforable just connecting the cells with the bus bars. But first I wanted to actually measure with a multimeter.

I was expecting about 5A based on the calculation above. I’m not sure how I was 10x off on the estimate but after hooking up my multimeter, the equalization current was 0.6A. That was plenty low so I set my mind to balancing. But before that, I needed bus bars to connect all the cells in parallel.

Constructing copper bus bars

Bus bars are used to conduct high amounts of current in electrical applications. In essence, they are oversized, flat wires. I ordered 2x copper bars from McMaster Carr that are 1″x1/8″x3′. They arrived in two days with free shipping… Amazon is gaining competition. I got to work drilling holes.

bus bars ready for drilling
copper bus bars ready for drilling with marks spaced 2.5″ apart
drilling 2mm pilot hole
drilling 2mm pilot hole
drilling 6mm hole
drilling 6mm final hole
copper shards after drilling
copper shards after drilling
filing down rough edges
filing down rough edges

Heat shrinking the cells

With the bus bars ready, it was time to heat shrink the battery cells. Battery Hookup said they were uncovered and needed to be heat shrunk for every cell. Turns out 7 of 8 had a covering on them already. I heat shrunk them anyways for two reasons: 1) to further protect the cells from damage and short circuits and 2) so they look better. The cell on the left will be re-done with more heat shrink. I guestimated how much I needed and was short a few inches.

cells ready for heat shrink
cells ready for heat shrink
cells heat shrunk with BMS
cells heat shrunk with BMS on top

Balancing in 2 sets of 4 cells each

When making the bus bars, I put together a mental picture of what I needed to make and how many I needed. This worked fine for the final battery but I would need double for balancing. I had the full extra 3′ copper bar but it took forever to cut so I just decided to balance the cells in 2 groups of 4. Balancing is charging all cells in parallel as one low voltage, giant capacity battery to their rated voltage (3.65V for LiFePO4). Doing 4 cells at a time meant it was a 3.2V battery with a capacity of 1040 amp hours. The resting voltages were in the upper end of the voltage curve chart so it wouldn’t take super long. If the battery was fully depleted, it would take 104 hours at the 10 amps my bench power supply puts out.

So I got the bus bars hooked up and started balancing by setting my power supply to 3.65V and current to max.

battery cells hooked up in parallel
battery cells hooked up in parallel (4x cells in parallel means a single 3.2V 1000Ah “cell”)
Measurement discrepancies and voltage drop

The first thing I did when I started charging was take voltage measurements to make sure things were right. I noticed some irregularities.

The first irregularity was the fact that the bench power supply was over-reporting the voltage by 0.034V or so when comparing the display to the terminals. This isn’t a huge deal and is actually a pretty decent level of accuracy for a $40 bench power supply from Amazon.

verifying DC power supply output voltage
verifying DC power supply output voltage (set to 3.650V, actual output is 3.614V)

The next thing I noticed is that for a 10A power supply, it is putting out a lot less than 10A. So I measured the voltage at the bus bars.

measuring voltage at bus bars
measuring voltage at bus bars. this is a pretty big voltage drop through the charger wires. 3.616V at the terminals – 3.363V at the bus bars is 0.253V drop. At 3.962A, that is a resistance of (V=IR -> V/I=R) 0.063 ohms. the wires were warm.

By leaving the voltage set to 3.65V at the DC power supply, the voltage drop means I wouldn’t get anywhere the rated 10A. The actual drop would decrease proportionally as the battery voltage neared the terminal voltage.

Regardless, it only took a couple hours until the cells were topped off for each set of 4. They are currently resting. I will check their voltages again tomorrow morning. The amount the cells drop in voltage from where they left off indicates the strength of the cell, with larger drops meaning weaker cells.

balancing in progress
the very start of balancing the 2nd set of 4 cells. I do intend to rewrap cell 3 and wrap cell 1 when my next batch of shrinkwrap arrives.

Conclusion

This is where we will leave off for the day. The 8 cells have been balanced in 2 groups of 4 and are currently resting. After 24 hours I will recheck the voltages to see which cells are strong and which are weak. I need to buy more electrical tape to cover up the bus bar ends (I did start assembling the full battery but stopped because the potential for short-circuit was higher than I was comfortable with).

As of 5/17/2021, things are up and running! Check it out at DIY solar with battery backup – up and running!