PUDA Logger

By default, the PUDA CLI uses SQLite for local storage. This is enough for a local workstation, quick experiments, and single-user development.

For backups, persistent storage, and a more reliable record of the whole runtime environment, run the PUDA logger. It should run 24/7 on an always-on machine, in a Docker container.

The recommended template is PUDAP/puda-logger. A Python subscriber receives NATS messages, decrypts them when configured, and converts them to Influx line protocol. Telegraf receives that output through inputs.execd, then buffers, batches, and writes it to InfluxDB.

The template uses InfluxDB. Most PUDA log queries are filtered by timestamp—runs, commands, telemetry, and events over a time window.

Run the logger

Clone the logger template on an always-on machine that can reach your NATS server. This can be the same machine that runs NATS, or another reliable machine on the same network.

tip

The database server and the logger do not need to run on the same machine. A common split is InfluxDB on a NAS for durable storage, and the logger on a mini PC closer to NATS. Each host only needs network access to the services it talks to: the server must be reachable from the logger, and the logger must be able to reach NATS.

Start InfluxDB

If you already have an InfluxDB server, reuse it and skip to Start the logger. Otherwise, on the machine that will host the database—for example, a NAS—clone the template and start the server stack:

git clone https://github.com/PUDAP/puda-logger.git
cd puda-logger/influxdb/server

Create an admin token file for InfluxDB:

echo '{"token": "<token>", "name": "<name>", "description": "Preconfigured admin token for influxdb-logger"}' > admin-token.json

Copy the server environment file:

cp .env.example .env

Edit .env and set INFLUXDB_TOKEN to the same token value from admin-token.json. Change INFLUXDB_PORT only if port 8181 is already in use on this host:

.env

INFLUXDB_PORT=8181
INFLUXDB3_NODE_IDENTIFIER_PREFIX=puda
INFLUXDB_TOKEN=<token>
INFLUXDB_DATABASE=puda
INFLUXDB3_NUM_IO_THREADS=8
INFLUXDB3_WAL_FLUSH_INTERVAL=100ms
INFLUXDB3_EXEC_MEM_POOL_BYTES=70%
INFLUXDB3_CHECKPOINT_INTERVAL=1h

Start InfluxDB:

docker compose up -d

The Compose template stores InfluxDB data in a Docker volume, so logs remain available across container restarts. The defaults raise IO threads (8), flush the WAL every 100ms, and checkpoint snapshots hourly so ingest and restarts stay tractable. After the logger writes health data, influxdb-cache-init creates a last-value cache on telemetry keyed by machine_id.

For a small NAS set INFLUXDB3_NUM_IO_THREADS=4. For a 32-core ingest host use 12–16 and put the data volume on local NVMe. InfluxDB 3 Core remains a single node with a ~72 hour query window; use Enterprise when you need HA, compaction, or longer historical queries.

Start the logger

On the machine that will run the logger—for example, a mini PC on the lab network—clone the template if needed and configure the logger stack:

git clone https://github.com/PUDAP/puda-logger.git
cd puda-logger/influxdb/logger
cp .env.example .env

Edit .env and point the logger at your InfluxDB server and NATS server. Use the same INFLUXDB_TOKEN from Start InfluxDB. Set NATS_SERVERS to the NATS URL(s) the logger can reach—for example, nats://<nats-host>:<port>:

.env

INFLUXDB_URL=http://<influxdb-host>:8181
INFLUXDB_TOKEN=<token>
INFLUXDB_DATABASE=puda
INFLUXDB_ORGANIZATION=bears
NATS_SERVERS=nats://<nats-host>:<port>
LOGGER_REPLICAS=1
JS_MAX_ACK_PENDING=20000
JS_RESPONSE_MAX_AGE=24h

LOGGER_REPLICAS controls the number of logger containers sharing NATS queue groups. Start with one and monitor CPU, buffer pressure, and response acknowledgment backlog before scaling. Redelivery can produce duplicates; queue groups do not guarantee exactly-once storage. With multiple replicas, each replica aggregates its own health samples, so queries must combine those results by machine.

For plaintext NATS, leave PUDA_NATS_CA and PUDA_E2E_KEY_FILE unset or empty. For an encrypted lab, set both to local host file paths and use only tls:// server URLs:

.env — optional E2EE

NATS_SERVERS=tls://<nats-host>:<port>
PUDA_NATS_CA=/absolute/path/ca.pem
PUDA_E2E_KEY_FILE=/absolute/path/e2e.key

There is no E2EE boolean. Setting only one path, mixing URL schemes, or providing invalid files fails startup rather than falling back to plaintext. Compose mounts the CA read-only and the key writable, and supplies the container paths automatically. Give the logger its own restricted-permission copy of the lab key so rotations persist. See Security for lab setup. The logger has its own locked SDK dependency; do not replace it with the edge SDK version as part of this setup.

Start the logger with Docker Compose. Use --build on the first run, or after Dockerfile or entrypoint changes:

docker compose up -d --build

The Python subscriber initializes the response streams and durable consumers using the same TLS settings as its subscriptions. There is no separate nats-stream-init service in the current Compose file. After changing .env or stream settings, recreate the logger:

docker compose up -d --force-recreate influxdb-logger

If you change telegraf.conf.template, restart without rebuilding:

docker compose restart influxdb-logger

To run without Docker, install Python 3.11+, uv, Telegraf, and envsubst. Run uv sync --frozen, export the variables above, and configure Telegraf's inputs.execd.command with absolute paths to .venv/bin/python and subscriber.py. Render the InfluxDB placeholders in the template and start Telegraf with that configuration. See the local setup instructions; the container entrypoint uses container-specific paths.

What the logger records

The subscriber and Telegraf pipeline writes three measurements:

Measurement	NATS subjects	Description
commands	puda.*.cmd.* (requests), puda.*.cmd.response.* (responses)	Command and response log
telemetry	puda.*.tlm.health	Machine health metrics (cpu, mem, temp), stored as a 60s mean per machine
telemetry_stream	puda.*.tlm.stream.>	Individual driver samples, tagged by machine_id and stream, with the complete JSON payload in the data string field

The SDK publishes host health every 5 seconds by default. The logger aggregates cpu, mem, and temp over 60-second windows. Optional status is a tag, so a status change can produce separate points within a window. connected is not extracted or averaged; use heartbeat or health freshness for availability. Command messages and driver telemetry samples are not averaged.

Position, weight, and other driver streams share telemetry_stream without payload field-type conflicts: the JSON object is stored as a string in data, preserving nested values and types. Numeric dashboards must extract their values from that JSON. An optional RFC3339 payload timestamp supplies the point time; otherwise receipt time is used.

Command requests, health, and driver telemetry use live Core NATS subscriptions. Command responses use shared durable push queue consumers on RESPONSE_QUEUE and RESPONSE_IMMEDIATE with limits retention (24 hours by default). The subscriber preserves existing delivery subjects and acknowledgment positions when initializing them. Responses can replay within retention after downtime; Core NATS traffic cannot.

Acknowledgment happens after handing line protocol to Telegraf, not after confirmation of an InfluxDB write. Crashes or exhausted buffers can lose data, and redelivery can produce duplicates. Monitor both subscriber and Telegraf errors.

tip

For the full field and tag schema, see the influxdb-logger README.

Migrate an existing database

Stop the old logger before switching writers. Inspect the existing table schema and back up historical data before any incompatible change:

• commands: run_id, username, and status are tags. If an existing table stores one as a field, migrate the data or use a new table/database; do not assume a field can be changed into a tag in place.
• telemetry: machine_id and optional status are tags; cpu, mem, and temp are float fields. Check for conflicting types. Old connected columns can remain, but new health samples do not populate them.
• telemetry_stream: machine_id and stream are tags and data is a string field. Existing scalar columns such as x, y, or mass_g can retain historical rows; new samples populate data and leave those columns null. No deletion is needed if data is absent or already a string field. Historical rows are not automatically backfilled.

Prefer preserving compatible tables. If rebuilding is necessary, verify the backup and migration plan before removing anything. Update Grafana queries for JSON-based telemetry and any migrated table/database names.

Verify logging

First inspect the container:

docker compose logs -f influxdb-logger

Use read-only discovery to find a live stream, then watch its advertised subject:

puda machine info <machine_id>
puda machine watch <advertised-subject> --timeout 15

Confirm recent rows in InfluxDB's telemetry_stream table, and allow a full aggregation window for health rows in telemetry. Watching NATS confirms publication, not database persistence. Check command rows after normal authorized machine activity; there is no need to home a machine merely to test logging.

Next steps

• Connect Grafana to InfluxDB for dashboards and alerts on machine health and command history.
• If you run Hermes agents, install Hermes Logger to persist agent telemetry to the same InfluxDB server.