I intend to use this site to document my journey down the path of nerdiness (past, present, and future). I’ve been learning over the years from various sites like what I hope this one becomes, and want to give back. I have a wide variety of topics I’d like to cover. At a minimum, posting about my activities will help me document what I learned to refer back in the future. I’ll also post about projects we do ourselves around the house instead of hiring professionals, saving big $$$$ in the process. Hope you enjoy the journey with me!
Below are some topic I plan on covering (I’ve already done something with every one of these and plan on documenting it):
RTL-SDRs (receiving signals from your electric meter, ADS-B, general radio stuff)
Virtual machines and my homelab setup
Home automation / smart home (Home Assistant, Tasmota, Phillips Hue bulbs, automating various tasks throughout the house)
My mini solar setup (2x300W panels) and not-so-mini battery backup (8x272Ah LiFePO4 batteries – should yield 7ish kWh of storage)
Remote control aircraft running Arduplane with video downlink and two-way telemetry
General computer stuff (building them, what I use mine for, Hyper-V)
Home network (Ubiquiti setup, VLANs, wiring the house with CAT6, IP security cameras on Blue Iris)
Formation of my LLC if anyone wants to hear about that
Continuing from the last post (Adding some polish to the X-Plane Python Autopilot with Flask, Redis, and WebSockets), I have a bit of momentum going on the Python X-Plane Autopilot stuff. There were a couple of items I wanted to complete before declaring the project “done”. The first is a “takeoff” button, which isn’t yet done. The other is the ability to fly along a track. That is now complete as of last night.
It is one thing to fly a bearing from A to B. That works fine as long as there is no wind in any direction. Flying a heading set by bearing is easy, and is part of the heading select & hold feature built out in a previous iteration of the code. To do so requires a “desired heading” and a heading error PID. The goal is to minimize the heading error, so we set the setpoint to 0. This controls a “roll” PID controller, which controls an aileron PID controller.
Each have limits in place to prevent excessive movement. For example, the roll PID controller is limited to +/- 30 degrees. Pitch is +/- 15 degrees.
To take this to the next step requires a few things:
A “track”, which is commonly defined as a start point and an end point. Both are simply lat/lon coordinate sets.
A current location, which is current lat/lon
A cross track distance (error), which is the distance the current location is off the track.
More PID loops, namely a cross track distance PID control, which, like the heading error PID, has a setpoint of 0 (i.e. the goal is to minimize the cross track distance).
Additionally, to make something actually useful, we need a “database” of navigation points. I parsed the fixed-width delimited text files of X-Plane for this, which was not fun.
To tie it all together, the web interface needs a way to type in a nav point, and a Direct To (D->To) button. Direct to is common in aviation GPS units to set a track from the location when the button is pushed to some point (VOR, fix, airport, etc). I’ve emulated that functionality.
Here’s the screenshot showing the example aircraft navigating to DVV, which is the KDEN VOR, from somewhere near KBJC. It shows a cross track error of 0.056 km, or 56 meters. ChatGPT helpfully generated the cross track error function with a resultant number in meters. I am comfortable with many kinds of units so I’ll leave this for now. The red line on the right map view is the aircraft’s interpretation of the direct to set at the same time as I clicked my autopilot’s Direct To button. There is a 4 kt wind coming from 029. I tested with greater, somewhat constant crosswinds in the 40-50 kt range with gusts of +/- 5 kts.
The cross track error settles down to < 10 m after a minute or so. It is a little “lazy”. If it is on a track that is due east, and I flip the track to due west, it’ll dutifully do the 180, then attempt to rejoin the track but it overshoots a bit and settles down after ~1 oscillation. I could probably turn up the P on the xte PID and that would help. Below is a track of tacking off from KBJC and the doing direct to DVV. The X is where I clicked Direct to back to the BJC VOR, it turned left and rejoined the track, overshooting, then settling back in nicely.
The Python Autopilot Code
I’m not going to pretend I wrote the cross track distance code, nor will I pretend to understand it. It works. The sign of the result depends on something along the lines of which side of the great circle line you are on. Luckily, aircraft (and boats and other things that follow tracks) don’t typically go from B to A. They go from A to B so this is consistent no matter which direction the track is facing. If they do need to go back to the start, the start becomes the end, if that makes sense.
This is the glorious cross track distance code along with some test code. Using Google Earth, the distance from the KBJC control tower to the centerline of 30R/12L should be ~0.44 km.
def cross_track_distance(point, start, end):
# Convert all latitudes and longitudes from degrees to radians
point_lat, point_lon = math.radians(point[0]), math.radians(point[1])
start_lat, start_lon = math.radians(start[0]), math.radians(start[1])
end_lat, end_lon = math.radians(end[0]), math.radians(end[1])
# Calculate the angular distance from start to point
# Ensure the argument is within the domain of acos
acos_argument = math.sin(start_lat) * math.sin(point_lat) + math.cos(start_lat) * math.cos(point_lat) * math.cos(point_lon - start_lon)
acos_argument = max(-1, min(1, acos_argument)) # Clamp the argument between -1 and 1
delta_sigma = math.acos(acos_argument)
# Calculate the bearing from start to point and start to end
theta_point = math.atan2(math.sin(point_lon - start_lon) * math.cos(point_lat),
math.cos(start_lat) * math.sin(point_lat) - math.sin(start_lat) * math.cos(point_lat) * math.cos(point_lon - start_lon))
theta_end = math.atan2(math.sin(end_lon - start_lon) * math.cos(end_lat),
math.cos(start_lat) * math.sin(end_lat) - math.sin(start_lat) * math.cos(end_lat) * math.cos(end_lon - start_lon))
# Calculate the cross track distance
cross_track_dist = math.asin(math.sin(delta_sigma) * math.sin(theta_point - theta_end))
# Convert cross track distance to kilometers by multiplying by the Earth's radius (6371 km)
cross_track_dist = cross_track_dist * 6371
return cross_track_dist
kbjc_runways = {
"30R/12L": {
"Runway 12L": {
"Latitude": 39.91529286666667,
"Longitude": -105.12841313333334
},
"Runway 30R": {
"Latitude": 39.901373883333335,
"Longitude": -105.10191808333333
}
}
}
kbjc_runway_30R_start = (kbjc_runways["30R/12L"]["Runway 30R"]["Latitude"], kbjc_runways["30R/12L"]["Runway 30R"]["Longitude"])
kbjc_runway_30R_end = (kbjc_runways["30R/12L"]["Runway 12L"]["Latitude"], kbjc_runways["30R/12L"]["Runway 12L"]["Longitude"])
kbjc_tower = (test_locations["kbjc_tower"]["lat"], test_locations["kbjc_tower"]["lon"])
def test_cross_track_distance():
print(f"start lat: {kbjc_runway_30R_start[0]}, start lon: {kbjc_runway_30R_start[1]}")
print(f"end lat: {kbjc_runway_30R_end[0]}, end lon: {kbjc_runway_30R_end[1]}")
print(f"tower lat: {kbjc_tower[0]}, tower lon: {kbjc_tower[1]}")
dist = cross_track_distance(kbjc_tower, kbjc_runway_30R_start, kbjc_runway_30R_end)
print(f"cross track distance: {dist}")
test_cross_track_distance()
And the rest of the magic happens in this block. If you recall from the last post (Adding some polish to the X-Plane Python Autopilot with Flask, Redis, and WebSockets), I am using Redis as a store to hold the setpoints from the web app controlling the autopilot. It is fast enough that I don’t need to worry about latency when running at 10 Hz (the loop durations are consistently less than 30 milliseconds, with the bulk of that time being used to get and set data from X-Plane itself).
# get the setpoints from redis
setpoints = get_setpoints_from_redis()
# check if we have just changed to direct-to mode and if so, update the direct to coords. same if the target waypoint has changed
if (setpoints["hdg_mode"] == "d_to" and previous_nav_mode != "d_to") or (setpoints["target_wpt"] != previous_nav_target):
print("reason for entering this block")
print(f"previous nav mode: {previous_nav_mode}, setpoints hdg mode: {setpoints['hdg_mode']}, previous nav target: {previous_nav_target}, setpoints target wpt: {setpoints['target_wpt']}")
# d_to_start_coords is the current position, in lat,lon tuple
d_to_start_coords = (posi[0], posi[1])
# this function does a lookup in the nav_points dataframe to get the lat, lon of the target waypoint
# it could certainly be optimized to use something faster than a pandas dataframe
d_to_target_coords = get_nav_point_lat_lon(setpoints["target_wpt"])
# reset xte PID
xte_PID.clear()
print(f"setting d_to_start_coords to {d_to_start_coords}")
# these are unchanged
desired_alt = setpoints["desired_alt"]
desired_speed = setpoints["desired_speed"]
if setpoints["hdg_mode"] == "hdg":
# if we're in heading mode, just use the desired heading. this is mostly unchanged from the previous iteration
desired_hdg = setpoints["desired_hdg"]
heading_error = get_angle_difference(desired_hdg, current_hdg)
heading_error_PID.update(heading_error)
elif setpoints["hdg_mode"] == "d_to":
# if we're in direct-to mode, calculate the cross-track error and update the xte_PID.
# I am using xte to mean cross-track error/distance
xte = cross_track_distance((posi[0], posi[1]), d_to_start_coords, d_to_target_coords)
xte_PID.update(xte)
# calculate the heading correction based on the xte_PID output
heading_correction = xte_PID.output
# this is essentially saying for 1 km of cross-track error, we want to correct by 30 degrees
heading_correction = heading_correction * 30
# limit the heading correction to -45 to 45 degrees
heading_correction = normalize(heading_correction, -45, 45)
# calculate the track heading to the target waypoint. the track heading is the heading we would
# need to fly to get to the target waypoint from the current position. it is used as an initial heading
track_heading = get_bearing((posi[0], posi[1]), d_to_target_coords)
# adjust the desired heading by the heading correction
adjusted_desired_hdg = track_heading + heading_correction
# make sure the adjusted desired heading is between 0 and 360
adjusted_desired_hdg = adjusted_desired_hdg % 360
# calculate the heading error based on the adjusted desired heading, this is no different than the hdg mode
adjusted_heading_error = get_angle_difference(adjusted_desired_hdg, current_hdg)
heading_error_PID.update(adjusted_heading_error)
# log the current values
print(f"track hdg: {track_heading:.1f}, heading corr: {heading_correction:.1f}, adj desired hdg: {adjusted_desired_hdg:.1f}, adj heading err: {adjusted_heading_error:.1f}")
# write to a log file so we can make nice plots for the blog post
log_line = f"{datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]},{posi[0]},{posi[1]},{posi[2]},{xte},{xte_PID.output},{track_heading},{heading_correction},{adjusted_desired_hdg},{adjusted_heading_error}"
with open(current_run_log_filename, "a") as log_file:
log_file.write(log_line + "\n")
Getting nav data from X-Plane data files
If you looked at the code closely, you will see the d_to_target_coords is set via a function called get_nav_point_lat_lon(nav_point). This looks up lat/lon in a file that was generated by parsing the X-Plane navigation data. In my previous job, I dealt with fixed width data formats. It is not fun. I originally tried to split based on spaces, but some of the nav point names have more than one space in them. I suppose I could just ignore the name but this is already written. This code parses the earth_nav.dat file, specifically for type 3, which is VOR/DME-like.
import pandas as pd
nav_filepath = r"C:\Users\Austin\Desktop\X-Plane 10\Resources\default data\earth_nav.dat"
raw_file_data = open(nav_filepath, 'r').readlines()
# remove first 3 lines
raw_file_data = raw_file_data[3:]
# remove last line
raw_file_data = raw_file_data[:-1]
# remove new line characters
raw_file_data = [line.replace('\n', '') for line in raw_file_data]
# Adjusting the function based on the new column map provided
def parse_nav_info(line):
column_map = {
'type': {'start': 0, 'end': 1},
'lat_sign': {'start': 2, 'end': 3},
'latitude': {'start': 3, 'end': 15},
'lon_sign': {'start': 15, 'end': 16},
'longitude': {'start': 16, 'end': 28},
'elevation': {'start': 29, 'end': 35},
'frequency': {'start': 36, 'end': 41},
'range': {'start': 42, 'end': 45},
'unknown': {'start': 46, 'end': 52},
'identifier': {'start': 53, 'end': 56},
'name': {'start': 56} # Assuming end is not needed; take till the end of the line
}
nav_info = {}
for column, column_info in column_map.items():
start = column_info['start']
end = column_info.get('end', None)
value = line[start:end].strip()
# print(f"attempting to parse {column} with value {value}")
if column == 'latitude':
lat_sign = line[column_map['lat_sign']['start']:column_map['lat_sign']['end']]
lat_sign = -1 if lat_sign == '-' else 1
value = lat_sign * float(value)
elif column == 'longitude':
lon_sign = line[column_map['lon_sign']['start']:column_map['lon_sign']['end']]
lon_sign = -1 if lon_sign == '-' else 1
value = lon_sign * float(value)
elif column == 'elevation':
value = int(value)
elif column == 'frequency':
value = int(value)
elif column == 'range':
value = int(value)
nav_info[column] = value
return nav_info
i = 0
data = []
types = []
for line in raw_file_data:
line_type = int(line[0:2])
if line_type != 3:
continue
line_data = parse_nav_info(line)
data.append(line_data)
df = pd.DataFrame(data)
columns_of_interest = ['identifier','latitude','longitude','elevation', 'frequency', 'range', 'name']
df = df[columns_of_interest]
df.head()
df.to_pickle('nav_data.pkl')
The code to read the file and import is at the beginning of the python_autopilot.py file and is fairly straightforward:
# nav_data.pkl is a pandas dataframe. yes, this should use a dict or something faster.
nav_points = pickle.load(open("nav_data.pkl", "rb"))
def get_nav_point_lat_lon(id):
nav_point = nav_points[nav_points["identifier"] == id]
return nav_point["latitude"].values[0], nav_point["longitude"].values[0]
And for the Flask side of the house, we have index.html:
<!DOCTYPE html>
<html>
<head>
<title>Autopilot Interface</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.1/socket.io.js"></script>
<script type="text/javascript" charset="utf-8">
var socket; // Declare socket globally
// Define adjustSetpoint globally
function adjustSetpoint(label, adjustment) {
socket.emit('adjust_setpoint', {label: label, adjustment: adjustment});
}
function submitDirectTo() {
const stationId = document.getElementById('target_wpt_input').value; // Grab the value from the input
if (stationId) { // Check if the stationId is not empty
adjustSetpoint('target_wpt', stationId); // Adjust the setpoint with the stationId as the value
adjustSetpoint('hdg_mode', "d_to"); // Your existing function call
} else {
alert("Please enter a station ID.");
}
}
document.addEventListener('DOMContentLoaded', () => {
socket = io.connect(location.protocol + '//' + document.domain + ':' + location.port);
socket.on('connect', () => {
console.log("Connected to WebSocket server.");
});
// Listen for update_setpoints event to initialize the UI with Redis values
socket.on('update_setpoints', function(setpoints) {
for (const [label, value] of Object.entries(setpoints)) {
const element = document.getElementById(label);
if (element) {
element.innerHTML = value;
}
}
});
// Listen for update_setpoint events from the server
socket.on('update_setpoint', data => {
// Assuming 'data' is an object like {label: new_value}
for (const [label, value] of Object.entries(data)) {
// Update the displayed value on the webpage
const element = document.getElementById(label);
if (element) {
element.innerHTML = value;
}
}
});
});
</script>
<style>
body {
font-family: Arial, sans-serif;
margin: 20px;
background-color: #f4f4f4;
color: #333;
}
h1 {
color: #005288;
}
ul {
list-style-type: none;
padding: 0;
}
ul li {
margin: 10px 0;
}
button, input[type="text"] {
padding: 10px;
margin-top: 5px;
border: 1px solid #ccc;
border-radius: 5px;
cursor: pointer;
font-size: 16px;
}
button:hover {
background-color: #ddd;
}
.button-group {
margin-bottom: 20px;
}
#target_wpt_input {
margin-right: 10px;
}
</style>
</head>
<body>
<h1>Autopilot Interface</h1>
<p>Current Setpoints:</p>
<ul>
<li>Heading: <span id="desired_hdg">0</span></li>
<li>Altitude: <span id="desired_alt">0</span></li>
<li>Speed: <span id="desired_speed">0</span></li>
<li>Heading Mode: <span id="hdg_mode">0</span></li>
<li>Target Waypoint: <span id="target_wpt">BJC</span></li>
</ul>
<p>Autopilot: <span id="autopilot_enabled">OFF</span></p>
<!-- Example buttons for adjusting setpoints -->
<div class="button-group">
<button onclick="adjustSetpoint('desired_hdg', -10)">-10 HDG</button>
<button onclick="adjustSetpoint('desired_hdg', 10)">+10 HDG</button>
</div>
<div class="button-group">
<button onclick="adjustSetpoint('desired_alt', 500)">+500 ALT</button>
<button onclick="adjustSetpoint('desired_alt', -500)">-500 ALT</button>
</div>
<div class="button-group">
<button onclick="adjustSetpoint('desired_speed', 5)">+5 KTS</button>
<button onclick="adjustSetpoint('desired_speed', -5)">-5 KTS</button>
</div>
<div class="button-group">
<button onclick="adjustSetpoint('hdg_mode', 'hdg')">Follow Heading</button>
<input type="text" id="target_wpt_input" value="BJC">
<button onclick="submitDirectTo()">Direct To</button>
</div>
<div class="button-group">
<button onclick="adjustSetpoint('autopilot_enabled', 1)">Enable Autopilot</button>
<button onclick="adjustSetpoint('autopilot_enabled', 0)">Disable Autopilot</button>
</div>
</body>
</html>
And the Flask app itself. I still think WebSockets are magic.
from flask import Flask, render_template
from flask_socketio import SocketIO, emit
import redis
app = Flask(__name__)
socketio = SocketIO(app)
r = redis.StrictRedis(host='localhost', port=6379, db=0)
setpoints_of_interest = ['desired_hdg', 'desired_alt', 'desired_speed']
# get initial setpoints from Redis, send to clients
@app.route('/')
def index():
return render_template('index.html') # You'll need to create an HTML template
def update_setpoint(label, adjustment):
# This function can be adapted to update setpoints and then emit updates via WebSocket
current_raw_value = r.get(label) if r.exists(label) else None
if current_raw_value is not None:
try:
current_value = float(current_raw_value)
except ValueError:
current_value = current_raw_value
if label == 'desired_hdg':
new_value = (current_value + adjustment) % 360
elif label == 'autopilot_enabled':
new_value = adjustment
elif label == 'hdg_mode':
new_value = adjustment
elif label == 'target_wpt':
new_value = adjustment
else:
new_value = current_value + adjustment
r.set(label, new_value)
# socketio.emit('update_setpoint', {label: new_value}) # Emit update to clients
return new_value
@socketio.on('adjust_setpoint')
def handle_adjust_setpoint(json):
label = json['label']
adjustment = json['adjustment']
# Your logic to adjust the setpoint in Redis and calculate new_value
new_value = update_setpoint(label, adjustment)
# Emit updated setpoint to all clients
emit('update_setpoint', {label: new_value}, broadcast=True)
@socketio.on('connect')
def handle_connect():
# Fetch initial setpoints from Redis
initial_setpoints = {label: float(r.get(label)) if r.exists(label) else 0.0 for label in setpoints_of_interest}
# Emit the initial setpoints to the connected client
emit('update_setpoints', initial_setpoints)
if __name__ == '__main__':
socketio.run(app)
And here’s the full code of the autopilot itself. This will be transferred to GitHub for the next post. It is a bit long and needs to be split out into a number of separate files.
With a cross track distance known, it isn’t terribly difficult to convert that distance (error) into a heading adjustment. We now have a functioning autopilot that can control our aircraft to any VOR-like point. I could extend the X-Plane nav data parsing to read all points, but I’ll leave that as an exercise for the reader. The X-Plane Python Autopilot is almost complete – all that I have left on the checklist is a “takeoff” button. Hope you enjoyed the post!
I revisited my Python X-Plane autopilot a few weeks ago because it was pretty clunky for how to adjust setpoints and such. The job I started 1.5 years ago is exclusively Python, so I wanted to redo a bit.
Quick aside: For the new PC I just built – Ryzen 9 7900x, 2x32GB 6000 MHz, etc, X-Plane 10 was the 2nd “game” I installed on it. The first was Factorio (I followed Nilaus’ megabase in a book and have got to 5k SPM). Haven’t tried the newer sims yet, but I think they’ll still be somewhat limited by my RTX 2080 Super.
Well imagine my surprise when I woke up to 6x the normal daily hits by 7am. I checked the weblogs and found that my post was trending on ycombinator.com (Hacker News). So I am going to skip pretty much all background and just post the updated code for now, and will go back and clean up this post at some point.
Without further ado: here’s what the super basic dashboard looks like
I have it separated into two main running python programs, the file that interacts with X-Plane itself, and the Flask part.
Here’s the adjusted autopilot code to check with Redis for the setpoints every loop execution:
# https://onion.io/2bt-pid-control-python/
# https://github.com/ivmech/ivPID
import sys
import os
import xpc
from datetime import datetime, timedelta
import PID
import time
import math, numpy
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
setpoints = {
"desired_roll": 0,
"desired_pitch": 2,
"desired_speed": 160,
"desired_alt": 8000.0,
"desired_hdg": 140,
"autopilot_enabled": 0
}
for key in setpoints:
# if the key exists in the redis db, use it
# otherwise, set it
if r.exists(key):
setpoints[key] = float(r.get(key))
else:
r.set(key, setpoints[key])
update_interval = 0.10 #seconds
update_frequency = 1/update_interval
P = 0.05
I = 0.01
D = 0
MAX_DEFLECTION_PER_SECOND = 2.0
roll_PID = PID.PID(P*2, I*2, D)
roll_PID.SetPoint = setpoints["desired_roll"]
pitch_PID = PID.PID(P, I, D)
pitch_PID.SetPoint = setpoints["desired_pitch"]
altitude_PID = PID.PID(P*2, P/2, D)
altitude_PID.SetPoint = setpoints["desired_alt"]
speed_PID = PID.PID(P, I, D)
speed_PID.SetPoint = setpoints["desired_speed"]
heading_error_PID = PID.PID(1,0.05,0.1)
heading_error_PID.SetPoint = 0 # need heading error to be 0
DREFs = ["sim/cockpit2/gauges/indicators/airspeed_kts_pilot",
"sim/cockpit2/gauges/indicators/heading_electric_deg_mag_pilot",
"sim/flightmodel/failures/onground_any",
"sim/flightmodel/misc/h_ind"]
def normalize(value, min=-1, max=1):
if (value > max):
return max
elif (value < min):
return min
else:
return value
def sleep_until_next_tick(update_frequency):
# Calculate the update interval from the frequency
update_interval = 1.0 / update_frequency
# Get the current time
current_time = time.time()
# Calculate the time remaining until the next tick
sleep_time = update_interval - (current_time % update_interval)
# Sleep for the remaining time
time.sleep(sleep_time)
# https://rosettacode.org/wiki/Angle_difference_between_two_bearings#Python
def get_angle_difference(b1, b2):
r = (b2 - b1) % 360.0
# Python modulus has same sign as divisor, which is positive here,
# so no need to consider negative case
if r >= 180.0:
r -= 360.0
return r
# https://gist.github.com/jeromer/2005586
def get_bearing(pointA, pointB):
"""
Calculates the bearing between two points.
The formulae used is the following:
θ = atan2(sin(Δlong).cos(lat2),
cos(lat1).sin(lat2) − sin(lat1).cos(lat2).cos(Δlong))
:Parameters:
- `pointA: The tuple representing the latitude/longitude for the
first point. Latitude and longitude must be in decimal degrees
- `pointB: The tuple representing the latitude/longitude for the
second point. Latitude and longitude must be in decimal degrees
:Returns:
The bearing in degrees
:Returns Type:
float
"""
if (type(pointA) != tuple) or (type(pointB) != tuple):
raise TypeError("Only tuples are supported as arguments")
lat1 = math.radians(pointA[0])
lat2 = math.radians(pointB[0])
diffLong = math.radians(pointB[1] - pointA[1])
x = math.sin(diffLong) * math.cos(lat2)
y = math.cos(lat1) * math.sin(lat2) - (math.sin(lat1)
* math.cos(lat2) * math.cos(diffLong))
initial_bearing = math.atan2(x, y)
# Now we have the initial bearing but math.atan2 return values
# from -180° to + 180° which is not what we want for a compass bearing
# The solution is to normalize the initial bearing as shown below
initial_bearing = math.degrees(initial_bearing)
compass_bearing = (initial_bearing + 360) % 360
return compass_bearing
# https://janakiev.com/blog/gps-points-distance-python/
def haversine(coord1, coord2):
R = 6372800 # Earth radius in meters
lat1, lon1 = coord1
lat2, lon2 = coord2
phi1, phi2 = math.radians(lat1), math.radians(lat2)
dphi = math.radians(lat2 - lat1)
dlambda = math.radians(lon2 - lon1)
a = math.sin(dphi/2)**2 + \
math.cos(phi1)*math.cos(phi2)*math.sin(dlambda/2)**2
return 2*R*math.atan2(math.sqrt(a), math.sqrt(1 - a))
KBJC_lat = 39.9088056
KBJC_lon = -105.1171944
def write_position_to_redis(position):
# position is a list of 7 floats
# position_elements = [lat, lon, alt, pitch, roll, yaw, gear_indicator]
position_elements = ["lat", "lon", "alt", "pitch", "roll", "yaw", "gear_indicator"]
position_str = ','.join([str(x) for x in position])
r.set('position', position_str)
for i in range(len(position_elements)):
r.set(f"position/{position_elements[i]}", position[i])
# position_str = ','.join([str(x) for x in position])
# r.publish('position_updates', position_str)
def get_setpoints_from_redis():
setpoints = {
"desired_roll": 0,
"desired_pitch": 2,
"desired_speed": 160,
"desired_alt": 8000.0,
"desired_hdg": 140
}
for key in setpoints:
# if the key exists in the redis db, use it
# otherwise, set it
if r.exists(key):
setpoints[key] = float(r.get(key))
else:
r.set(key, setpoints[key])
return setpoints
def get_autopilot_enabled_from_redis():
if r.exists("autopilot_enabled"):
return int(r.get("autopilot_enabled").decode('utf-8')) == 1
ele_positions = []
ail_positions = []
thr_positions = []
def update_control_position_history(ctrl):
ele_positions.append(ctrl[0])
ail_positions.append(ctrl[1])
thr_positions.append(ctrl[3])
# if the list is longer than 20, pop the first element
if len(ele_positions) > 20:
ele_positions.pop(0)
ail_positions.pop(0)
thr_positions.pop(0)
def monitor():
with xpc.XPlaneConnect() as client:
while True:
loop_start = datetime.now()
print(f"loop start - {loop_start}")
posi = client.getPOSI()
write_position_to_redis(posi)
ctrl = client.getCTRL()
bearing_to_kbjc = get_bearing((posi[0], posi[1]), (KBJC_lat, KBJC_lon))
dist_to_kbjc = haversine((posi[0], posi[1]), (KBJC_lat, KBJC_lon))
#desired_hdg = 116 #bearing_to_kbjc
multi_DREFs = client.getDREFs(DREFs) #speed=0, mag hdg=1, onground=2
current_roll = posi[4]
current_pitch = posi[3]
#current_hdg = posi[5] # this is true, need to use DREF to get mag ''
current_hdg = multi_DREFs[1][0]
current_altitude = multi_DREFs[3][0]
current_asi = multi_DREFs[0][0]
onground = multi_DREFs[2][0]
# get the setpoints from redis
setpoints = get_setpoints_from_redis()
desired_hdg = setpoints["desired_hdg"]
desired_alt = setpoints["desired_alt"]
desired_speed = setpoints["desired_speed"]
# outer loops first
altitude_PID.SetPoint = desired_alt
altitude_PID.update(current_altitude)
heading_error = get_angle_difference(desired_hdg, current_hdg)
heading_error_PID.update(heading_error)
speed_PID.SetPoint = desired_speed
new_pitch_from_altitude = normalize(altitude_PID.output, -10, 10)
new_roll_from_heading_error = normalize(heading_error_PID.output, -25, 25)
# if new_pitch_from_altitude > 15:
# new_pitch_from_altitude = 15
# elif new_pitch_from_altitude < -15:
# new_pitch_from_altitude = -15
pitch_PID.SetPoint = new_pitch_from_altitude
roll_PID.SetPoint = new_roll_from_heading_error
roll_PID.update(current_roll)
speed_PID.update(current_asi)
pitch_PID.update(current_pitch)
new_ail_ctrl = normalize(roll_PID.output, min=-1, max=1)
new_ele_ctrl = normalize(pitch_PID.output, min=-1, max=1)
new_thr_ctrl = normalize(speed_PID.output, min=0, max=1)
previous_ail_ctrl = ail_positions[-1] if len(ail_positions) > 0 else 0
previous_ele_ctrl = ele_positions[-1] if len(ele_positions) > 0 else 0
previous_thr_ctrl = thr_positions[-1] if len(thr_positions) > 0 else 0
# not currently functional - need to work on this
# new_ail_ctrl_limited = previous_ail_ctrl + new_ail_ctrl * MAX_DEFLECTION_PER_SECOND / update_frequency
# new_ele_ctrl_limited = previous_ele_ctrl + new_ele_ctrl * MAX_DEFLECTION_PER_SECOND / update_frequency
# new_thr_ctrl_limited = previous_thr_ctrl + new_thr_ctrl * MAX_DEFLECTION_PER_SECOND / update_frequency
# update the control positions
# update_control_position_history((new_ele_ctrl_limited, new_ail_ctrl_limited, 0.0, new_thr_ctrl_limited))
update_control_position_history((new_ele_ctrl, new_ail_ctrl, 0.0, new_thr_ctrl))
onground = -1
if onground == 1:
print("on ground, not sending controls")
else:
if get_autopilot_enabled_from_redis():
# ctrl = [new_ele_ctrl_limited, new_ail_ctrl_limited, 0.0, new_thr_ctrl_limited]
ctrl = [new_ele_ctrl, new_ail_ctrl, 0.0, new_thr_ctrl]
client.sendCTRL(ctrl)
loop_end = datetime.now()
loop_duration = loop_end - loop_start
output = f"current values -- roll: {current_roll: 0.3f}, pitch: {current_pitch: 0.3f}, hdg: {current_hdg:0.3f}, alt: {current_altitude:0.3f}, asi: {current_asi:0.3f}"
output = output + "\n" + f"hdg error: {heading_error: 0.3f}"
output = output + "\n" + f"new ctrl positions -- ail: {new_ail_ctrl: 0.4f}, ele: {new_ele_ctrl: 0.4f}, thr: {new_thr_ctrl:0.4f}"
output = output + "\n" + f"PID outputs -- altitude: {altitude_PID.output: 0.4f}, pitch: {pitch_PID.output: 0.4f}, ail: {roll_PID.output: 0.3f}, hdg: {heading_error_PID.output: 0.3f}"
output = output + "\n" + f"bearing to KBJC: {bearing_to_kbjc:3.1f}, dist: {dist_to_kbjc*0.000539957:0.2f} NM"
output = output + "\n" + f"loop duration (ms): {loop_duration.total_seconds()*1000:0.2f} ms"
print(output)
sleep_until_next_tick(update_frequency)
os.system('cls' if os.name == 'nt' else 'clear')
if __name__ == "__main__":
monitor()
And the flask backend/front end. WebSockets are super cool – never used them before this. I was thinking I’d have to make a bunch of endpoints for every type of autopilot change I need. But this handles it far nicer:
from flask import Flask, render_template
from flask_socketio import SocketIO, emit
import redis
app = Flask(__name__)
socketio = SocketIO(app)
r = redis.StrictRedis(host='localhost', port=6379, db=0)
setpoints_of_interest = ['desired_hdg', 'desired_alt', 'desired_speed']
# get initial setpoints from Redis, send to clients
@app.route('/')
def index():
return render_template('index.html') # You'll need to create an HTML template
def update_setpoint(label, adjustment):
# This function can be adapted to update setpoints and then emit updates via WebSocket
current_value = float(r.get(label)) if r.exists(label) else 0.0
if label == 'desired_hdg':
new_value = (current_value + adjustment) % 360
elif label == 'autopilot_enabled':
new_value = adjustment
else:
new_value = current_value + adjustment
r.set(label, new_value)
# socketio.emit('update_setpoint', {label: new_value}) # Emit update to clients
return new_value
@socketio.on('adjust_setpoint')
def handle_adjust_setpoint(json):
label = json['label']
adjustment = json['adjustment']
# Your logic to adjust the setpoint in Redis and calculate new_value
new_value = update_setpoint(label, adjustment)
# Emit updated setpoint to all clients
emit('update_setpoint', {label: new_value}, broadcast=True)
@socketio.on('connect')
def handle_connect():
# Fetch initial setpoints from Redis
initial_setpoints = {label: float(r.get(label)) if r.exists(label) else 0.0 for label in setpoints_of_interest}
# Emit the initial setpoints to the connected client
emit('update_setpoints', initial_setpoints)
if __name__ == '__main__':
socketio.run(app)
And the http template:
<!DOCTYPE html>
<html>
<head>
<title>Autopilot Interface</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.1/socket.io.js"></script>
<script type="text/javascript" charset="utf-8">
var socket; // Declare socket globally
// Define adjustSetpoint globally
function adjustSetpoint(label, adjustment) {
socket.emit('adjust_setpoint', {label: label, adjustment: adjustment});
}
document.addEventListener('DOMContentLoaded', () => {
socket = io.connect(location.protocol + '//' + document.domain + ':' + location.port);
socket.on('connect', () => {
console.log("Connected to WebSocket server.");
});
// Listen for update_setpoints event to initialize the UI with Redis values
socket.on('update_setpoints', function(setpoints) {
for (const [label, value] of Object.entries(setpoints)) {
const element = document.getElementById(label);
if (element) {
element.innerHTML = value;
}
}
});
// Listen for update_setpoint events from the server
socket.on('update_setpoint', data => {
// Assuming 'data' is an object like {label: new_value}
for (const [label, value] of Object.entries(data)) {
// Update the displayed value on the webpage
const element = document.getElementById(label);
if (element) {
element.innerHTML = value;
}
}
});
});
</script>
</head>
<body>
<h1>Autopilot Interface</h1>
<p>Current Setpoints:</p>
<ul>
<li>Heading: <span id="desired_hdg">0</span></li>
<li>Altitude: <span id="desired_alt">0</span></li>
<li>Speed: <span id="desired_speed">0</span></li>
</ul>
<p>Autopilot: <span id="autopilot_enabled">0</span></p>
<!-- Example buttons for adjusting setpoints -->
<button onclick="adjustSetpoint('desired_hdg', -10)">-10 HDG</button>
<button onclick="adjustSetpoint('desired_hdg', 10)">+10 HDG</button>
<br>
<button onclick="adjustSetpoint('desired_alt', 500)">+500 ALT</button>
<button onclick="adjustSetpoint('desired_alt', -500)">-500 ALT</button>
<br>
<button onclick="adjustSetpoint('desired_speed', 5)">+5 KTS</button>
<button onclick="adjustSetpoint('desired_speed', -5)">-5 KTS</button>
<br>
<br>
<button onclick="adjustSetpoint('autopilot_enabled', 1)">Enable Autopilot</button>
<button onclick="adjustSetpoint('autopilot_enabled', 0)">Disable Autopilot</button>
</body>
</html>
This should be enough to get you going. I’ll come back and clean it up later (both my kids just woke up – 1.5 and 3.5 years!)
Today’s blog post is driven by a desire for simplicity. If you would’ve asked me even a month ago – “Hey Austin, do you think hooking GitHub actions up to deploy a docker-compose application stack is a good way to simplify something?” I 1000% would’ve said no. But I have had to get comfortable with Docker for work over the last couple months. That, combined with some assistance from my favorite AI (GPT4), has led to something I would call “simple”. The first attempt at anything is always a bit rough (one time I did get up on an “air chair” hydrofoil on my first attempt at Seminoe Reservoir in Wyoming though, and earned $100 on a bet for it) but this is a super repeatable pattern.
WARNING ABOUT SECURITY (written by me): This solution does not automatically make your app secure. You still need to have a laser-sharp focus on security. Any vulnerability in your web app can allow an attacker to gain a foothold in your network. Your docker host should be firewalled off from the rest of your network as a first step to prevent traversal into other computers/services/systems in your network. Any steps after that should be focused on the usual attack vectors (SQL injection, keeping systems up to date, etc).
WARNING ABOUT SECURITY (written by ChatGPT): Opening a tunnel directly into your home network can expose internal systems to external threats if not properly secured. Reddit commenters (see https://www.reddit.com/r/homelab/comments/17mc2jg/not_a_fan_of_opening_ports_in_your_firewall_to/) have pointed out that bypassing traditional port-forwarding and firewall configurations can lead to unauthorized access if the tunnel is not adequately protected. The use of Cloudflared or similar tunneling services can alleviate the need for port-forwarding, but without proper security measures, such as robust authentication and encryption, the tunnel could become a vector for malicious activity. It’s crucial to ensure that any tunnel into your (home) network is securely configured to mitigate potential security risks.
Cloudflare tunnels allow for CF to route traffic to your service without port-forwarding. That’s the key for how this all works.
Table of Contents
Components of the stack
Web framework, web server, cloudflare tunnel, GitHub & Actions, docker, docker host, etc
Example website running this stack – uuid7.com (down as of 8:15am 2023-11-03 while I take my own security advice and migrate to it’s own network on my VM host. still getting DNS figured out the the tunnel can be established. back up as of 8:30am, had a block to all internal networks rule, needed to add a allow DNS to DMZ interface before it)
Docker & Docker compose
Self-hosted GitHub runner on a VM
GitHub Actions
Components of the stack
There are quite a few parts of this stack. Also I am not a fan of the word/phrase “stack” for describing these kinds of things but here we are. It really is a stack.
Flask – a basic web framework for getting stuff off the ground quickly. Substitute in any of your own frameworks here. As long as they can listen on a port, you’ll be fine. Django, ASP.NET Core, etc. would all work here.
NGINX – not strictly necessary, but it’s the web server I’m most familiar with. It logs to a standard format, and works well. Some of the frameworks mentioned above do warn against deploying directly to the internet so we stick NGINX in front.
Cloudflared (Cloudflare Tunnels) – these tunnels are super handy. They establish an outbound connection to Cloudflare from whatever they’re running on. Once the tunnel is established, anything destined for your service will go to Cloudflare first (since they are doing the DNS) and from there, it’ll get routed through the tunnel to your service.
Docker – runs containers. Hopefully I don’t need to expand on this.
Docker compose – runs multiple containers in a group. Allows for easy (well “easy” in the sense that each container can talk to each other relatively easily) networking, and ability to stand up a “stack” of related containers together.
GitHub – hosts code. Also not expanding on this
GitHub Actions – triggers actions when various criteria are hit for your repository hosted on GitHub (for example, pushing a new docker compose file when committed to main)
A host running Docker (your desktop/laptop, Linux VM, AWS EC2 instance, etc.)- place for the docker stack to be deployed
Self-hosted GitHub runner – place where the action is run when code is committed to main
Okay so now that I’ve wrote out that list, it is not exactly simple. But is is repeatable. Realistically, the only part you’ll change is Flask and the docker-compose.yml file. The rest are somewhat copy + paste.
Example website running this stack – uuid7.com
Like quite a few of you reading this, I buy domains far before I actually do anything with them. I have a side project I’m working on (a “Tinder for Restaurants”), and decided on GUIDs/UUIDs as the IDs for all my tables. UUIDv4 turns out to not work well with database indexes because it is not sequential. UUIDv7 is and works great (it has a time component as well as some randomness). I wanted to make a simple site to demonstrate UUIDv7s hence uuid7.com was born about a month ago. Screenshot below:
uuid7.com components
This is a pretty straight-forward site. There is a main.py which is the entry point for the Docker image, a styles.css, scripts.js, and an index.html template.
A typical set of NGINX log entries for a real person visiting the site with a real browser is such:
As you might expect, the site loads fairly quickly:
Docker & Compose
I’m not going to elaborate on how Docker works. The Dockerfile for the flask app is 7 lines (see below). We are getting the Python 3.11 base image, copying the code, installing flask and uuid7, exposing a port (which I don’t think is strictly necessary), defining an entry point and the file to run. Doesn’t get much easier than this.
FROM python:3.11
WORKDIR /app
COPY . .
RUN pip install flask uuid7
EXPOSE 5602
ENTRYPOINT [ "python" ]
CMD ["main.py"]
Do note that I am running Flask on port 5602:
if __name__ == '__main__':
# listen on all IPs
app.run(host='0.0.0.0', port=5602, debug=True)
Ah but you might say “Austin, I thought you also had Cloudflare Tunnels and NGINX going?”. And you would be right.
There is no NGINX “app” container, just two config files (a “default” NGINX-wide one called nginx.conf and a site-specific one called default.conf).
For the site specific config, we are just saying, listen on port 8755 for SSL requests, use the defined cert and key, and pass everything to the container named “flask” via port 5602. You are free to use whatever ports here you want. There are multiple (nginx listen port -> flask listen port). The IP/Forwarded headers are so NGINX can log the real requester and not the Cloudflare server that forwarded the request. If you do not do this step, it will look like all of your customers/clients are coming from cloudflare and you’ll never see their real IPs.
For the NGINX-wide config, there is fairly standard stuff. I copied + pasted most of this from the real default config. I did customize the log format to include the upstream request time which I like to see (this is how long it takes the upstream server, be it Flask or php-fpm or ASP.NET core takes to turn around the request). The IP addresses listed are Cloudflare servers and are where I should believe them when they say they’re forwarding from someone else. Note the last CIDR listed – 172.29.0.0/16. This is the docker network. There is actually a double forward going on and this is also necessary (real_ip_recursive is set to on).
The NGINX Dockerfile does some magic in that it generates an unsigned SSL certificate so that things “work” via HTTPS (couldn’t figure out how to do plain HTTP within the docker compose but HTTPS externally). There is an option in the Cloudflare Tunnel to ignore SSL errors, which is enabled.
The Cloudflare Tunnel container is so simple there isn’t even a Dockerfile for it, just an entry in the docker-compose.yml.
Which brings us to the docker-compose.yml file. This is the “secret sauce” that brings it all together. This file defines a “stack” of related containers that form an application. This is still somewhat magic to me.
Since it is magic, and I am still learning how to describe these various concepts, ChatGPT did a decent summary:
This docker-compose.yml file outlines a multi-container Docker application, specifically designed to run a Flask application behind an NGINX reverse proxy, with a Cloudflare tunnel for secure and fast network connections. The file is written in version 3.8 of the Docker Compose file format, ensuring compatibility with newer features and syntax.
Flask Service
The flask service is configured to build a Docker image from a Dockerfile located in the ./flask_app directory. The Dockerfile should contain instructions for setting up the Flask application environment. The FLASK_RUN_HOST environment variable is set to 0.0.0.0, allowing the Flask application to be accessible from outside the Docker container. The restart: always directive ensures that the Flask service is automatically restarted if it stops for any reason.
NGINX Service
The nginx service also builds its Docker image from a Dockerfile, which is expected to be in the ./nginx directory. This service is set up as a reverse proxy, forwarding requests to the Flask application. The ports directive maps port 8755 on the host machine to port 8755 inside the Docker container, making the NGINX server accessible from outside. The depends_on field specifies that the NGINX service depends on the Flask service, ensuring that Docker Compose starts the Flask service before starting NGINX. Like the Flask service, NGINX is configured to restart automatically if it stops.
Cloudflared Service
The cloudflared service utilizes the official Cloudflare tunnel image and runs the Cloudflare tunnel with the specified command. The CF_TOKEN environment variable, which should contain your Cloudflare tunnel token, is passed to the container through the command line. This service is also configured to restart automatically if it stops.
Networking
By default, Docker Compose sets up a single network for your application, and each container for a service joins the default network. In this case, an IPAM (IP Address Management) configuration is specified with a subnet of 172.29.0.0/16, ensuring that the containers receive IP addresses from this specified range.
The Cloudflare Tunnel
If you don’t have one already, create a cloudflare account, and add your domain and prove ownership. Then go to Cloudflare zero trust area (usually under the access link). Create a tunnel. The token is the long alphanumeric string in the code box for any of the environment types. It always starts with ‘ey’:
On the public hostname part, you need a minimum of the domain, type, and url. Here, the domain is obvious. It is going over HTTPS mostly because, again, I couldn’t figure out how to get it to do plain HTTP within the compose stack. The port here is where NGINX is listening and the hostname is the name of the NGINX container (simply ‘nginx’).
docker-compose up!
Replace ${CF_TOKEN} in your docker-compose.yml with the actual token for testing on your local machine. Then from the same folder that contains docker-compose.yml, run:
docker-compose up
The images will be built if necessary, and after they are built, they will be started and you’ll see output that looks something like this:
To kill, just do a Ctrl-C and wait for them to gracefully exit (or not).
A Place for the Stack to Live
We need a host where the docker stack can live and exist in a 24/7 uptime world. This is typically not your dev machine. I have a Dell R630 in a datacenter in the Denver Tech Center area. That will suffice. Create a VM/VPS somewhere (on a R630 in a datacenter, Azure, DigitalOcean, AWS EC2, etc), and install docker on it. Make sure you also install docker-compose. Further make sure you don’t install Docker on the Proxmox host by accident (oops). Then, install a self-hosted GitHub runner on the VM/VPS as well.
Create a secret in your repo with the Cloudflare Tunnel token, and name it the same thing as in your docker-compose.yml file (be sure to change that back!). I am using CF_TOKEN, which is represented in the docker-compose.yml file:
What your GitHub secrets will look like with a secret added:
GitHub Actions
Lastly, we need to define some actions to take place when certain events happen. In our case, we are interested in when the ‘main’ branch of the repo changes. This is my full .github\workflows\main.yml file:
name: Docker Deploy
on:
push:
branches:
- main
jobs:
deploy:
runs-on: self-hosted
steps:
- name: Checkout Code
uses: actions/checkout@v2
- name: Set up Docker Compose
run: echo "CF_TOKEN=${{ secrets.CF_TOKEN }}" >> $GITHUB_ENV
- name: Navigate to Code Directory
run: cd ${{ github.workspace }}
- name: Run Docker Compose Up with Sudo
run: sudo -E docker-compose up -d --build
Now, when you commit, it will reach out to your self-hosted runner, which will pull your code, insert the CF_TOKEN secret into the environment variables, and then run the docker-compose up command to get your stack going. Here are a couple executions. You will likely have a couple misfires when setting up the pipeline if you are doing something different than me. My record is 19 tries (read: failures) in Azure DevOps to get a build/release pipeline fully functional.
Conclusion
This post turned out to be far longer than I anticipated. I hope you find it helpful! As noted earlier, the vast majority of this is essentially copy + paste. By that, I mean once you do it once, you can easily do it again. You can run these docker compose stacks pretty much anywhere.
There is a “feature” of Cloudflare Tunnels in that they say they are randomly routed if there is more than one active tunnel. I have not tested this but this allows for some interesting possibilities. For sites that have zero persistence, like the uuid7.com example site, that means I can run this stack in more than one place for redundancy by just doing a git pull/docker-compose up.
For a little side project, I build uuid7.com almost entirely with the help of AI tools. Check it out!
I also built the stack with Docker Compose. I have resisted Docker for so long because it was such a paint for homelab type stuff. But I recently started needing to use it at work (we are migrating to AWS Kubernetes – yay! not) so I figured I’d give it a go.
With the assistance of ChatGPT, I put together a full Docker stack using Cloudflare tunnels (cloudflared), Nginx as the webserver, and Flask as the backend all in a couple hours. It works great!
That said, it is running on my main desktop at home to see if it’s a popular site so fingers crossed it holds up.
I have a 1U Datto NAS unit that I got for super cheap ($150 for 4x 3.5″ SAS3, D-1541, 4x32GB, 2400MHz, 2x 10GbaseT) that has worked quite well for me. The only downside, which is present among basically all 1U devices, is the noise.
I am not 100% sure of the default AsrockRack behavior but it seemed that if CPU temp >60C, both case and CPU fans would spike. My BlueIris instance sends videos over a couple times an hour, which would spike the fans, which would be annoying during my work from home weeks while I was in the basement, working.
The idea of using a PID loop to control fan speeds stuck with me though, and with the help of GitHub Copilot, I busted out a proof of concept in an hour or so during a particularly boring set of meetings. There is a very high probability this will work for Supermicro motherboards as well with only minor tweaks.
This is how well it works. Note the drop at the end is due to changing CPU setpoint from 57C to 55C. The temperature is very, very steady.
Without further ado, below is the main script (you’ll also need PID.py, which I borrowed a few years ago for the Coding a pitch/roll/altitude autopilot in X-Plane with Python series of posts). It can be run via SSH for debugging purposes (it is no fun to edit python via nano over ssh on FreeBSD), or with native commands if it detects it is running on the target system.
import logging
import time
import PID
import datetime
import socket
import subprocess
logging.basicConfig(
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.INFO)
# don't care about debug/info level logging from either of these packages
loggers_to_set_to_warning = ['paramiko.transport', 'invoke']
for l in loggers_to_set_to_warning:
logging.getLogger(l).setLevel(logging.WARNING)
user = "root"
password = r"password"
host = None # this is set via hostname detection below
DESIRED_CPU_TEMP = 55.0
DESIRED_MB_TEMP = 35.0
# HDD_TEMP_THRESHOLD = 44.0 # unused
MIN_FAN_PCT = 10.0
drives_to_monitor = ['da0', 'da1', 'da2', 'da3', 'nvme0','nvme1','nvme2']
# command to set fans via ipmitool
# ipmitool raw 0x3a 0x01 0x00 0x04 0x04 0x04 0x04 0x04 0x04 0x04
#cpu #fan #fan #fan #fan #fan #fan ????
BASE_RAW_IPMI = 'raw 0x3a 0x01'
INITIAL_STATE = [32,32,32,32,32,32,32,32] # all 32/64 = half speed
FAN_CURRENT_STATE = INITIAL_STATE
hostname = socket.gethostname()
if 'truenas' in hostname or hostname == 'truenas-datto.home.fluffnet.net':
host = 'localhost'
c = None
else:
from fabric import Connection # importing here because freebsd 13 (or whatever truenas core 13 is based on lacks pip to install packages)
host = "10.98.1.9"
c = Connection(host, port=22, user=user, connect_kwargs={'password': password})
current_sensor_readings = {}
cpu_temp_sensor = "CPU Temp"
cpu_fan_sensor = "CPU_FAN1"
case_fans = ["FRNT_FAN2","FRNT_FAN3","FRNT_FAN4"]
mb_temp_sensor = "MB Temp"
def limiter(input_value, min_value, max_value):
if input_value < min_value:
return min_value
elif input_value > max_value:
return max_value
else:
return input_value
def set_fans_via_ipmi(connection):
# raw_ipmi_cmd = construct_raw_ipmi_cmd() # not needed unless debug and remote
# logging.info(raw_ipmi_cmd)
if host == 'localhost':
result = subprocess.run(['ipmitool', 'raw', '0x3a', '0x01',
'0x'+FAN_CURRENT_STATE[0],
'0x'+FAN_CURRENT_STATE[1],
'0x'+FAN_CURRENT_STATE[2],
'0x'+FAN_CURRENT_STATE[3],
'0x'+FAN_CURRENT_STATE[4],
'0x'+FAN_CURRENT_STATE[5],
'0x'+FAN_CURRENT_STATE[6],
'0x'+FAN_CURRENT_STATE[7]], stdout=subprocess.PIPE)
else:
raw_ipmi_cmd = construct_raw_ipmi_cmd()
result = connection.run('ipmitool ' + raw_ipmi_cmd, hide=True)
#logging.info(result.stdout)
def scale_to_64ths(input_percent):
result = input_percent / 100.0 * 64.0
# prepend 0 to make it a hex value
result_int = int(result)
result_str = str(result_int)
if len(result_str) == 1:
result_str = '0' + result_str # turn a 0x1 into a 0x01
return result_str
def adjust_cpu_fan_setpoint(hex_value_64ths):
FAN_CURRENT_STATE[0] = hex_value_64ths
def adjust_case_fan_setpoint(hex_value_64ths):
for i in range(len(FAN_CURRENT_STATE) - 1):
FAN_CURRENT_STATE[i + 1] = hex_value_64ths
def construct_raw_ipmi_cmd():
new_state = BASE_RAW_IPMI
for i in range(len(FAN_CURRENT_STATE)):
new_state = new_state + ' 0x' + str(FAN_CURRENT_STATE[i])
return new_state
def populate_sensor_readings(sensor, value):
current_sensor_readings[sensor] = value
def query_ipmitool(connection):
if host == 'localhost':
result = subprocess.run(['ipmitool', 'sensor'], stdout=subprocess.PIPE)
result = result.stdout.decode('utf-8')
else:
result = connection.run('ipmitool sensor', hide=True).stdout
for line in result.split('\n'):
if line == '':
break
row_data = line.split('|')
sensor_name = row_data[0].strip()
sensor_value = row_data[1].strip()
populate_sensor_readings(sensor_name, sensor_value)
logging.debug(sensor_name + " = " + sensor_value)
def wait_until_top_of_second():
# calculate time until next top of second
sleep_seconds = 1 - (time.time() % 1)
time.sleep(sleep_seconds)
def get_drive_temp(connection, drive):
###########################################
# this is copilot generated, and untested #
# not sure about row_data[0] stuff #
###########################################
if host == 'localhost':
result = subprocess.run(['smartctl', '-A', '/dev/' + drive], stdout=subprocess.PIPE)
result = result.stdout.decode('utf-8')
else:
result = connection.run('smartctl -A /dev/' + drive, hide=True).stdout
for line in result.split('\n'):
if line == '':
break
row_data = line.split()
if len(row_data) < 10:
continue
if row_data[0] == '194':
drive_temp = row_data[9]
logging.info(drive + " = " + drive_temp)
def query_drive_temps(connection):
for drive in drives_to_monitor:
get_drive_temp(connection, drive)
# tune these values. the first one is the most important and basically is the multiplier for
# how much you want the fans to run in proportion to the actual-setpoint delta.
# example: if setpoint is 55 and actual is 59, the delta is 4, which is multiplied by 4 for
# 16 output, which if converted to 64ths would be 25% fan speed.
# the 2nd parameter is the integral, which is a cumulative error counter of sorts.
# the 3rd parameter is derivative, which should probably be set to 0 (if tuned correctly, it prevents over/undershoot)
cpu_pid = PID.PID(4.0, 2.5, 0.1)
cpu_pid.SetPoint = DESIRED_CPU_TEMP
mb_pid = PID.PID(2.5, 1.5, 0.1)
mb_pid.SetPoint = DESIRED_MB_TEMP
wait_until_top_of_second()
# set last_execution to now minus one minute to force first execution
last_execution = datetime.datetime.now() - datetime.timedelta(minutes=1)
while(True):
if datetime.datetime.now().minute != last_execution.minute:
# TODO: get drive temps
logging.info("getting drive temps")
query_ipmitool(c)
cpu_temp = float(current_sensor_readings[cpu_temp_sensor])
mb_temp = float(current_sensor_readings[mb_temp_sensor])
cpu_pid.update(cpu_temp)
mb_pid.update(mb_temp)
logging.info(f'CPU: {cpu_temp:5.2f} MB: {mb_temp:5.2f} CPU PID: {cpu_pid.output:5.2f} MB PID: {mb_pid.output:5.2f}')
# note negative multiplier!!
cpu_fan_setpoint = scale_to_64ths(limiter(-1*cpu_pid.output,MIN_FAN_PCT,100))
case_fan_setpoint = scale_to_64ths(limiter(-1*mb_pid.output,MIN_FAN_PCT,100))
adjust_cpu_fan_setpoint(cpu_fan_setpoint)
adjust_case_fan_setpoint(case_fan_setpoint)
set_fans_via_ipmi(c)
last_execution = datetime.datetime.now()
wait_until_top_of_second()
As you can see, it is not quite complete. I still need to add the hard drive temp detection stuff to ramp case fans a bit if the drives get hot. Those NVMe drives sure get hot (especially the Intel P4800X I have in one of the PCIe slots – see Intel Optane P1600X & P4800X as ZFS SLOG/ZIL for details).
This is what the output looks like (keep in mind the -1 multiplier in the setpoint stuff!):
And here is a summary of the script provided by the ever helpful ChatGPT with some high-level summaries. I fed it the code and said “write a blog post about this”. I took out the intro paragraph but left the rest.
The Script Overview
This script leverages the PID controller – a control loop mechanism that calculates an “error” value as the difference between a measured process variable and a desired setpoint. It attempts to minimize the error by adjusting the process control inputs.
In this script, we are implementing a fan speed control system that reacts to temperature changes dynamically. Our desired setpoint is the optimal temperature we want to maintain for both the CPU (DESIRED_CPU_TEMP) and the motherboard (DESIRED_MB_TEMP).
Exploring the Script
The Python script begins by setting up the necessary libraries and logging. The logging library is used to log useful debug information, such as the current CPU temperature and fan speed, which can help you understand what’s happening in the script.
Next, we have a section where we define some constants, such as the desired temperatures and minimum fan speed percentage. It also defines a connection to the localhost or to a remote host, depending on the hostname.
It uses ipmitool – a utility for managing and configuring devices that support the Intelligent Platform Management Interface (IPMI) to control fan speeds.
The limiter() function ensures the fan speed remains within the predefined minimum and maximum thresholds. It’s important as it prevents the fan speed from reaching potentially harmful levels.
The script also includes several functions to set and adjust fan speeds, as well as to construct the appropriate ipmitool command. One thing to note is that the fan speeds are set using hexadecimal values, so there are functions to convert the desired fan speed percentages to hexadecimal.
A very useful function is query_ipmitool(). This function runs the ipmitool command, gets the current sensor readings, and stores them in the current_sensor_readings dictionary for further processing.
The script utilizes two PID controllers, cpu_pid for the CPU and mb_pid for the motherboard, with specific setpoints set to desired temperatures.
The core logic is inside the infinite loop at the end of the script. The loop constantly reads temperature sensor data and adjusts the fan speeds accordingly. The loop runs once every second, so it can respond quickly to changes in CPU and motherboard temperatures.
Conclusion
This script demonstrates a neat way of controlling fan speed in response to CPU and motherboard temperatures. It’s an effective approach to ensure that your system runs smoothly and without overheating, while minimizing noise.