Weather Underground station network with Alopex DB & Alopex Skulk

What the service does

A Weather Underground-like network connects personal weather stations and air quality monitors run by people around the world and gathers hyperlocal weather data. Station owners get dashboards and graphs for their station, an archive of past data, and a way to share it. The data is station details and time series of temperature, humidity, wind, rainfall and other readings.

Where the data lives

Skulk time series: readings are ingested into Skulk, tagged by station and region.
Time-bucketed aggregation: dashboard graphs use time buckets for hourly and daily averages. Latest values and comparisons between regions come from PromQL.
Alopex DB tables: station registrations, owners and locations live in tables.

Deployment

Skulk is embedded in the app server that receives ingest and queries. As stations are added, the same engine grows into a Skulk server and cluster. With Skulk embedded on the station side too, readings taken while the network is down are kept locally. The stored data is Parquet, so analysis of past weather reads it directly in other tools.

Why Alopex DB and Alopex Skulk fit

A station network’s data is numbers with timestamps and tags that never stop arriving. Skulk takes on ingest, storage and aggregation of exactly this shape of data as a library inside the app.

See it in code

Station registrations live in Alopex DB (observation time series in Skulk). Just the core of counting stations per region.

import alopex

# 観測所の登録情報は Alopex DB（観測値の時系列は Skulk が取り込む）
db = alopex.Database.open("weather.alopex")

db.execute_sql("""
    CREATE TABLE IF NOT EXISTS stations (
        id INTEGER PRIMARY KEY,
        owner TEXT,
        region TEXT,
        place TEXT
    )
""")

stations = [
    (1, "tanaka", "関東", "世田谷"),
    (2, "suzuki", "関東", "横浜"),
    (3, "sato", "関西", "大阪"),
]
for s in stations:
    db.execute_sql("INSERT INTO stations (id, owner, region, place) VALUES (?, ?, ?, ?)", list(s))

# 地域ごとの観測所の数を集計する
for row in db.execute_sql("SELECT region, COUNT(*) AS n FROM stations GROUP BY region ORDER BY n DESC"):
    print(row["region"], row["n"])

Registration data is aggregated and filtered with SQL. The temperature, humidity and rainfall time series is ingested by Skulk, tagged by station and region, and shown with PromQL and time-bucket aggregation.

Alopex Skulk runs everything from ingest to PromQL queries inside the app. Give it a try for a station network.