rvzd monitor

Private, in development · v0.37

Server monitoring with an assistant that asks before it acts.

A small agent on each server reports to one hub. The hub keeps the history, opens an incident when something breaks and, if you want, hands it to an assistant that reads logs, metrics and network checks, then proposes a fix. Nothing changes on a server until you approve it.

I’m interested

Write and tell me a bit about your servers. I’ll get back to you.

monitor@rvzd.dev

web-1 online degraded up 148d
Memory
HTTP, shop.example.com no response

Down since 14:02, from all three servers

  1. warningsystem

    Network check: 3 of 18 failed

    networknetcheck14:02

  2. criticalsystem

    Incident #92 opened: postgresql.service failed web-1

    incident14:02

  3. infoAI

    AI picked up incident #92

    AI action14:03

  4. infoAI

    AI analysis: reading logs AI analysis: postgres was killed by the OOM killer

    memory is back to 20%14:03

  5. warningAI

    Proposal #57: restart postgresql.service

    needs younothing runs until approved

Proposal #57 low risk web-1

Why: postgres was killed by the OOM killer at 14:01 while a report query ran. The query is gone and memory is back to 20%, so a restart should hold.

  1. 1reset_failedpostgresql.service
  2. 2restart_servicepostgresql.service

Waiting for you. The assistant can’t run this itself.

rvzd | alerts 14:04

AI proposal #57 on web-1: reset_failed, restart_service postgresql.service. Approve or reject in the dashboard.

Proposal #57 web-1 Reject Approve Approved
  1. ✓ reset_failed postgresql.service 0.2 s
  2. ✓ restart_service postgresql.service 1.6 s

active postgresql.service is running again. Checking that it holds.

Verification AI incident open resolved
sleep 30s get_service_status postgresql get_service_logs postgresql, 1m

postgres has been up for 40 s and is accepting connections, and its log has been clean since the restart. Resolved.

judgea second model grades the answer

  • resolved 0.92
  • still broken 0.05
  • needs a human 0.02
  • false alarm 0.01

Claims it ran something itself: no, 0.01

What it watches

Every minute, from every server. A CPU, memory or disk incident takes three bad reports to open and as many good ones to close, so a single spike doesn’t page you, and a fixed problem closes itself.

System
CPU, memory, every mount with inodes and read-only state, load, pressure stalls, steal.
Docker
State, health, exit codes, OOM kills and real restart counts.
systemd
Failed units, including instances like getty@tty1.
Network
Public addresses and when they change, traffic, drops, TCP retransmits, conntrack.
Firewall
UFW rules and ipsets, readable and editable from the dashboard.
Processes
Top processes with secrets stripped from their command lines.

Approval

Nothing runs until you approve it

The assistant has no tool that changes anything. It can read, and it can propose. The hub runs a proposal only after you press Approve, and only the actions on a short allow-list.

  • Actions are systemd ones: restart, stop, disable, mask, unmask, reset a failed unit. Each carries a risk level you see before approving.
  • A proposal can be a chain of up to eight steps. One approval runs it, and it stops at the first step that fails.
  • Every step is checked against the allow-list again right before it runs.
  • Proposals nobody looks at expire after 48 hours.
Proposal #61 high risk db-1

Why: backup.service has failed every night this week on a full disk. Stop it, clear the failed state and keep it from starting until the disk is sorted.

  1. 1 stop_service backup.service allowed ✓ 0.4 s
  2. 2 reset_failed backup.service allowed ✕ exit status 1
  3. 3 mask_service backup.service not run

Stopped at step 2. Step 3 never ran, and the activity log shows why.

Follow-up

It comes back to check

Every investigation becomes a topic you can open and ask about. When a problem needs time, the assistant schedules its own re-check instead of saying “keep an eye on it”.

  • Re-checks from 5 minutes to 12 hours out. When it wakes, it sees what happened while it was away.
  • If the same incident comes back, it continues the same topic with everything it already found.
  • After three checks that don’t resolve it, the topic flags you instead of quietly giving up.
#41 hel-1: disk / at 91% active
get_server_status hel-1 get_metrics_history disk, 7d list_containers hel-1

Most of it is Docker build cache (6.1 GB). The journal was rotated and gave back 1.2 GB. Growth over the week is slow, so this isn’t urgent. I’ll look again in an hour.

Re-check scheduled in 60 min

#41 hel-1: disk / at 91% active
min
  • 14:52disk / 88%
  • 15:20disk / 84%

Wakes are stored in the hub’s database. If the hub is down when one is due, it fires on restart and knows it’s late.

#41 hel-1: disk / at 91% active resolved

Woke at 15:40. 2 events since the last look.

get_metrics_history disk, 1h

Disk has held at 74% for the last hour and nothing new is writing to it. Closing this one.

If a third check still found it broken, the topic would flag you instead of giving up quietly.

Network checks

See the network from where your servers are

Ping, TCP, DNS, HTTP and traceroute, run from any server or the hub, to the internet or to each other. The same API can be quick from Frankfurt and unreachable from Singapore. Now you can see which, and where the path stops.

  • Country and network (AS) for every address, from an offline database.
  • Each run is compared with the last one and with a baseline server, so “slow” has a number.
  • HTTP checks follow redirects inside the same domain and can look for text in the page.
  • Run by hand, on a schedule, or by the assistant while it investigates.
A probe run: three servers each check six destinations. Every check passes except one: from sgp-1, api.example.com does not answer. fra-1 nyc-1 sgp-1 1.1.1.1 8.8.8.8 github.com registry.npmjs.org hub.docker.com api.example.com
17 checks pass 1 fails: api.example.com from sgp-1
from ↓ to → 1.1.1.1githubnpmapi
fra-1 DE 1.1ms 6.4ms 3.2ms 11ms
nyc-1 US 0.9ms 9.7ms 4.7ms 84ms
sgp-1 SG 2.3ms 48ms 212ms 100% loss
hub NL 1.4ms 11ms 5.1ms 14ms

14 ok 1 degraded, +164 ms vs the last run 1 failed

traceroute, folded by network

  1. sgp-1 source
  2. AS3356 Lumen · 4 hops
  3. AS174 Cogent · 2 hops, last answer
  4. no answer 5 silent hops
  5. api.example.com AS64500 · not reached

The last hop that answers is inside AS174. From fra-1 and nyc-1 the same host answers in 11 and 84 ms, so the trouble is on the transit path from Singapore, not on the server.

ok GET https://status.example.com

  1. 301→ https://www.status.example.com/
  2. 200final, 1 redirect in the same domain
dns 12 ms
connect 38 ms
tls 71 ms
first byte 143 ms
body
re:Operational found
title
Status · All systems normal
certificate
Let’s Encrypt R11
ip
203.0.113.24 · AS64500

Privacy

What leaves your servers is scrubbed

Before tool output reaches the model or Telegram, the hub strips passwords, tokens, keys and credentials in URLs. Addresses get stable pseudonyms, so the model can tell two hosts apart without seeing either one.

  • Scrubbing happens on the hub, on the way out, on every path: chat, investigations, automations, alerts.
  • The assistant only ever gets read-only tools.
  • The whole assistant is off until you turn it on.
$ list_processes web-1
redis-server --port 6379 --requirepass [REDACTED]
node /srv/worker.js BOT_TOKEN=[REDACTED]

$ get_service_logs postgresql
connection from ip-9b1e04 to ip-3fa2c1:5432
app DATABASE_URL=postgres://shop:[REDACTED]@ip-3fa2c1/shop
Out of memory: Killed process 2217 (postgres)
image ghcr.io/acme/shop:1.2.3.4 started
  • [REDACTED] passwords, tokens, keys, credentials in URLs
  • ip-3fa2c1 same address, same name, every time
  • 1.2.3.4 a version number, left alone

Automations

Automations that notify or propose

A schedule or an event starts a short pipeline: run a network plan, read logs, ask the model, check a condition, send a message or call a webhook. Each step can use the output of an earlier one.

  • Every run is recorded step by step, and you can watch it live.
  • An automation can propose an action but never runs one. That still goes through you.
  • The assistant can draft an automation from a sentence.
Network baseline run #318 running ok
  1. on schedule every 30 min · Europe/Berlin 14:30
  2. 1 net_check plan “Fleet baseline” degraded = 1
  3. 2 condition {@1.degraded} gt 0 true
  4. 3 ai_run What changed since the last run? {@1.summary} output, 212 chars
  5. 4 telegram Network: {@3.output} sent
rvzd | alerts 14:31

Network: sgp-1 can’t reach api.example.com since 14:02, the other 17 checks pass. The last answering hop is in AS174, so it’s the transit path from Singapore, not the server.

Works with what you already run

Prometheus

Point the hub at an existing Prometheus and the dashboard, charts, assistant and automations read from it, with no agents installed. Add an agent later and the host becomes one row with both.

Alertmanager

Send its webhook to the hub. Critical alerts get the same investigation as native incidents, and your pager stays your pager.

Telegram and webhooks

Incidents, proposals and digests. A persistent outbox retries after restarts, and long messages arrive as a numbered thread.

Under the hood

Two binaries, one database file, nothing else to run.

Agent
One Go binary for Linux, no dependencies. Works alone too, with its own API and dashboard.
Hub
One Go binary with SQLite. Watches its own server without a separate agent.
History
Per-minute for 30 days, hourly for 180, daily for a year.
TLS
Let’s Encrypt, even for a bare IP. Or self-signed, or off for development.
Access
Bearer tokens, 2FA codes over Telegram, a separate session per device.
Assistant
Off by default. Any model on OpenRouter, token budgets, a spend dashboard.

Want to run it on your servers?

rvzd monitor isn’t public yet. It runs my own machines while it grows up. If you’d like to try it early, get in touch.

I’m interested

Write and tell me a bit about your servers. I’ll get back to you.

monitor@rvzd.dev