UPS monitoring with NUT: clean shutdowns for a home rack

A UPS on its own buys you time; it does not spend it for you. When the battery runs flat, the load drops just as hard as it would in a power cut, only twenty minutes later. Network UPS Tools (NUT) is what turns those minutes into clean shutdowns. It reads the UPS, tells every machine on it when to stop, and as the last step tells the UPS to switch its outlets off and back on, so everything boots again when the mains returns.

This is the setup for a typical home rack: one UPS with a USB cable, a small always-on Linux box as the primary (mine is the Pi from the weekend Raspberry Pi monitoring build), and a Proxmox host and a NAS as secondaries. Commands are for Debian and its relatives.

How the pieces fit

• A driver talks to the UPS. For most USB units that is usbhid-ups.
• upsd serves the driver's data to the network on TCP port 3493.
• upsmon runs on every machine the UPS feeds and shuts that machine down when told. One of them is the primary: it sits on the machine with the cable, decides when it is time, and gets the UPS to cut power at the end. The others are secondaries and listen over the network.
• upsc is the small client you use to read values by hand.

NUT 2.8.0 replaced the old master and slave terms with primary and secondary. Older guides and some NAS menus still use the old words, and current releases accept them for compatibility, but write new configs with the new ones.

Step 1: find the UPS

sudo apt install nut
lsusb
sudo nut-scanner -U

Most UPS units with a USB port present themselves as a HID Power Device, and usbhid-ups covers them. A few budget units speak a serial protocol through a USB adapter and need a different driver such as nutdrv_qx; the hardware compatibility list on the NUT site tells you which. nut-scanner -U prints a ready-made config section for whatever it finds.

If you have not bought the UPS yet, these are common USB models with a data port that NUT normally drives with usbhid-ups. Check the exact model against the compatibility list before you order; the list below comes from specifications and published reviews, not from my bench.

# Affiliate links: we may earn a commission. As an Amazon Associate I earn from qualifying purchases.

Step 2: driver and data server on the primary

In /etc/nut/ups.conf:

[rackups]
    driver = usbhid-ups
    port = auto
    desc = "Home rack UPS"

The port line has to be there, and for USB auto is the right value. With two USB UPS units on one machine, add vendorid, productid or serial lines to tell them apart.

In /etc/nut/nut.conf, set MODE=netserver, because the secondaries will connect over the network. Then tell upsd where to listen, in /etc/nut/upsd.conf:

LISTEN 127.0.0.1 3493
LISTEN 10.0.10.11 3493

Here 10.0.10.11 is the primary's address on the management network from my home VLAN plan. Open TCP 3493 on the firewall for the secondaries' addresses and nobody else.

Accounts go in /etc/nut/upsd.users:

[upsmon_local]
    password = change-me-local
    upsmon primary

[upsmon_remote]
    password = change-me-remote
    upsmon secondary

[admin]
    password = change-me-admin
    instcmds = ALL

Use long random passwords and keep the NUT config files owned by root, group nut, mode 0640 (the packages set it up that way). The secondary account cannot set the forced-shutdown flag, so a compromised NAS cannot switch the rack off.

Step 3: upsmon on the primary

The lines that matter in /etc/nut/upsmon.conf; the packaged file has the rest, with comments:

MONITOR rackups@localhost 1 upsmon_local change-me-local primary
MINSUPPLIES 1
SHUTDOWNCMD "/sbin/shutdown -h +0"
FINALDELAY 5
HOSTSYNC 15
NOTIFYFLAG ONBATT SYSLOG+WALL
NOTIFYFLAG ONLINE SYSLOG+WALL
NOTIFYFLAG LOWBATT SYSLOG+WALL

The 1 is the power value: this machine has one power supply, fed by this UPS. FINALDELAY and HOSTSYNC are shown at their defaults. Leave POWERDOWNFLAG as your package set it; that file is what the shutdown hook looks for before it tells the UPS to cut the load.

Unit names vary a little between distributions, so run systemctl list-units 'nut*', restart what it shows, and check the system log for upsmon reporting a successful startup and the client logging in to upsd.

Step 4: read what the UPS says

$ upsc rackups@localhost
battery.charge: 100
battery.charge.low: 10
battery.runtime: 1740
battery.runtime.low: 120
input.voltage: 121.0
output.voltage: 121.0
ups.load: 23
ups.status: OL

That is trimmed; a real UPS prints a few dozen lines. The ones to know:

• battery.charge is percent. battery.runtime is seconds, the UPS's own estimate at the current load. The .low values are the thresholds at which the UPS raises its low-battery flag.
• ups.load is percent of the UPS's rated capacity.
• ups.status is a set of flags: OL on line power, OB on battery, LB low battery, CHRG and DISCHRG for the battery, RB replace battery, OVER overload, and FSD once a forced shutdown is under way. OB LB together means critical: the shutdown starts there.

If a field is missing, your UPS does not report it. NUT shows what the device sends and makes nothing up. upsc rackups@localhost ups.status reads one value, which is handy in scripts.

Step 5: the secondaries

On the Proxmox host, install nut-client, set MODE=netclient in nut.conf, and give upsmon.conf its own monitor line:

MONITOR rackups@10.0.10.11 1 upsmon_remote change-me-remote secondary
MINSUPPLIES 1
SHUTDOWNCMD "/sbin/shutdown -h +0"

Most NAS systems have a NUT client in their UPS settings: point it at the primary's address with the UPS name, user and password. Some expect a fixed UPS name and account when they act as a network client; if yours does, match it in ups.conf and upsd.users instead of fighting it. Back on the primary, upsc -c rackups@localhost lists every connected client, and both secondaries should be in that list.

Shutdown order: should the NAS go last?

Out of the box, NUT runs this sequence. The UPS reaches OB LB. The primary sets the forced-shutdown flag. Every secondary sees it, waits FINALDELAY and shuts down, all at the same moment. The primary waits up to HOSTSYNC seconds for them to disconnect, shuts itself down, and the shutdown hook tells the UPS to cut its outlets. With usbhid-ups that happens after offdelay (20 seconds by default), and once the mains is back the UPS restores power after ondelay (30 seconds).

So every secondary goes at once, at low battery. Whether that suits you depends on what leans on what.

Putting the NAS last protects anything that keeps its disks on the NAS. Virtual machines on NFS or iSCSI storage are the classic case: pull the storage from under a running VM and you get a worse mess than a plain power cut. The price is that the NAS shuts down with the least battery left, and an aging battery is exactly when the runtime estimate turns optimistic.

Putting the NAS first gives it a calm shutdown with plenty of battery in hand, but only makes sense when nothing else depends on it.

My order is hypervisor early on a timer, NAS at low battery, and the Pi as primary last of all. The router and switch never shut down: they stay up to carry the shutdown message and simply lose power at the end. Shedding the hypervisor early has a second benefit: on lead-acid batteries, halving the load usually more than doubles the remaining runtime, so the NAS gets a longer, calmer run.

The timer is upssched on the Proxmox host. Add to its upsmon.conf:

NOTIFYCMD /sbin/upssched
NOTIFYFLAG ONBATT SYSLOG+WALL+EXEC
NOTIFYFLAG ONLINE SYSLOG+WALL+EXEC

Then /etc/nut/upssched.conf:

CMDSCRIPT /usr/local/bin/ups-early-shutdown
PIPEFN /run/nut/upssched.pipe
LOCKFN /run/nut/upssched.lock
AT ONBATT * START-TIMER early 180
AT ONLINE * CANCEL-TIMER early

Three minutes on battery starts the shutdown; power coming back inside three minutes cancels it. The script receives the timer name as its first argument. It runs as the unprivileged nut user, not root, so it needs one narrow sudo rule (Proxmox does not ship sudo; install it):

#!/bin/sh
if [ "$1" = "early" ]; then
    logger -t ups "on battery for 3 minutes, early shutdown"
    sudo /sbin/shutdown -h +0
fi

nut ALL=(root) NOPASSWD: /sbin/shutdown -h +0

The first block is /usr/local/bin/ups-early-shutdown (make it executable); the second goes in a file under /etc/sudoers.d/, edited with visudo -f.

Last, check the low-battery point. battery.charge.low and battery.runtime.low in the upsc output tell you when your UPS will raise LB. If that leaves less time than your NAS needs to stop its services and park its disks, it is too late. ignorelb in ups.conf tells the driver to ignore the UPS's own flag and use thresholds you set, such as override.battery.runtime.low = 300, as long as your unit reports runtime reliably.

A test procedure you can run on a Saturday

Pick a morning, check the backups ran, and tell the household. Then, in order:

1. Read-only first. upsc rackups@localhost shows OL and sane numbers, and upsc -c rackups@localhost lists both secondaries.
2. Cut the mains for a minute. I switch off the UPS's circuit at the breaker or a switched socket rather than pulling the plug, so the UPS stays grounded. Every machine should log ONBATT, the status should read OB DISCHRG, and both should clear when power returns. If the outage is meant to fire the hypervisor's timer, leave it off past three minutes and watch that host go down alone.
3. Dry run the power-off. sudo upsdrvctl -t shutdown prints what the driver would send to the UPS without sending it.
4. The full sequence. sudo upsmon -c fsd on the primary runs the real thing: secondaries go down, the primary follows, the UPS switches its outlets off, then on again, and everything boots. For that last part, set each machine's firmware to power on after AC loss. If the UPS never cuts its outlets, you are open to the power race the NUT documentation describes: the mains returns mid-shutdown, the machines stay halted and the UPS stays on. Fix that before you trust the setup; with the systemd shutdown hook NUT ships, POWEROFF_WAIT in nut.conf makes the primary reboot itself if the power never goes.
5. Once a year, the long test. Cut the mains and let the battery run down to LB with everything running, with a stopwatch and upsc in a loop. Compare the real runtime with what battery.runtime promised at the start.

Write those five steps down where you will find them next year. A short checklist beats memory, the same reason runbooks for on-call exist.

Graphs and alerts

Four values are worth graphing: ups.load, battery.runtime, battery.charge and input.voltage. The useful one is runtime at a steady load. When the same load shows fewer minutes month after month, the battery is aging, and you will know before an outage tells you. Input voltage sags and spikes also explain a lot of "random" reboots of the things that are not on the UPS.

If you already run Prometheus, there are community NUT exporters that turn upsc fields into metrics. LibreNMS can read NUT through its application support. The field manual's list of what you can monitor puts UPS values next to the rest.

Larger UPS units take a network management card instead of, or as well as, USB. Those speak SNMP: the standard UPS-MIB from RFC 1628 plus the vendor's own MIB. Any SNMP poller can graph them, and NUT's snmp-ups driver (the nut-snmp package on Debian) can use the card for shutdowns, ideally over SNMPv3:

[bigups]
    driver = snmp-ups
    port = 10.0.10.20
    snmp_version = v3
    secLevel = authPriv
    secName = nutmon
    authPassword = change-me-auth
    privPassword = change-me-priv
    authProtocol = SHA
    privProtocol = AES

For alerts, keep it to the ones you act on: on battery for more than a minute, low battery, lost contact with the UPS, and replace battery. Everything else belongs on the graph.

Before you call it done

• upsc shows OL and the runtime you expect
• every secondary appears in upsc -c
• the low-battery point leaves time for your slowest machine
• upsmon -c fsd power-cycled the rack and everything came back on its own
• the router and switch are on the UPS, and the test steps are written down