{{.Title}} · nasmon

commit 09159a5f31914b36fb6a14de0800fefd1f3b0baf Author: a.gritsenko Date: Sat Sep 19 10:24:34 2026 +0300 Initial import: nasmon health monitor, split out of home_nas Go service (nasmon/) plus nfpm packaging (packaging/), carried over from the home_nas monorepo. Adds rpm packaging alongside the existing deb build (Fedora/RHEL/Rocky dependency names via nfpm overrides, per-format systemd unit path, dpkg+rpm-compatible maintainer scripts) and a Gitea Actions workflow that builds both formats on every push. Co-Authored-By: Claude Sonnet 5 diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..8a50094 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,15 @@ +# Files that end up on the Linux server must keep LF endings +*.sh text eol=lf +*.yml text eol=lf +*.json text eol=lf +.env* text eol=lf +*.ps1 text eol=crlf +*.go text eol=lf +*.yaml text eol=lf +*.html text eol=lf +*.css text eol=lf +*.js text eol=lf +*.md text eol=lf +*.service text eol=lf +go.mod text eol=lf +go.sum text eol=lf diff --git a/.gitea/workflows/build.yml b/.gitea/workflows/build.yml new file mode 100644 index 0000000..b4aea9e --- /dev/null +++ b/.gitea/workflows/build.yml @@ -0,0 +1,37 @@ +name: build + +on: + push: + branches: ["**"] + tags: ["v*"] + pull_request: + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-go@v5 + with: + go-version: "1.22" + + - name: vet & test + run: | + go vet ./... + go test ./... + + - name: install nfpm + run: go install github.com/goreleaser/nfpm/v2/cmd/nfpm@latest + + - name: build packages + run: | + export PATH="$PATH:$(go env GOPATH)/bin" + ./build.sh + + - uses: actions/upload-artifact@v4 + with: + name: nasmon-packages + path: | + dist/*.deb + dist/*.rpm diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ba85743 --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +/nasmon +/nasmon.exe +dist/ diff --git a/README.md b/README.md new file mode 100644 index 0000000..8e72d81 --- /dev/null +++ b/README.md @@ -0,0 +1,70 @@ +# nasmon + +Lightweight NAS health monitor written in Go. Web dashboard (CPU, memory +incl. ZFS ARC, temperatures, physical disks with SMART, ZFS pools and +datasets) with periodic SMART/ZFS checks and e-mail alerts, plus SMB/NFS +share management from the UI. + +* Web UI: `http://:8090` +* Settings: `http://:8090/settings` (also `/etc/nasmon/config.json`) +* State: `/var/lib/nasmon/nasmon.db` (SMART history, alert log) +* Service: `systemctl status nasmon` / `journalctl -u nasmon -f` +* JSON API: `/api/status`, `/api/history?hours=24`, `/api/disks//history?days=30` + +No authentication — designed for a LAN-only NAS (the `/shares` page can +create SMB/NFS users and exports, so keep it off the public internet). + +See [packaging/README.md](packaging/README.md) for the full operational +doc (ZFS dataset management, SMB/NFS shares, alert lifecycle) — it ships +inside the package as `/usr/share/doc/nasmon/README.md`. + +## Build + +Go 1.22, [nfpm](https://nfpm.goreleaser.com/) (no `rpmbuild`/`dpkg-deb` +needed — nfpm builds both formats natively). + +```powershell +# Windows +.\build.ps1 # dist\nasmon__amd64.deb + dist\nasmon-.x86_64.rpm +.\build.ps1 -Test # go vet + go test first +.\build.ps1 -Version 0.2.0 +``` + +```bash +# Linux/WSL/CI +./build.sh +TEST=1 ./build.sh +VERSION=0.2.0 ./build.sh +``` + +One-time tool setup: + +``` +go install github.com/goreleaser/nfpm/v2/cmd/nfpm@latest +``` + +## Packaging + +`packaging/nfpm.yaml` builds both formats from one spec: + +- **.deb** (Ubuntu/Debian): depends on `smartmontools`, `zfsutils-linux`, `util-linux`; + recommends `samba`, `nfs-kernel-server`, `wsdd` for the `/shares` page. +- **.rpm** (Fedora/RHEL/Rocky, `dnf`): depends on `smartmontools`, `util-linux`; + recommends `samba`, `nfs-utils`, `wsdd`. ZFS is **not** a hard dependency on + rpm — RHEL-family distros have no native ZFS package; install the + [OpenZFS](https://openzfs.github.io/openzfs-docs/Getting%20Started/RHEL-based%20distro/index.html) + repo yourself if you need the `/zfs` page's pool/dataset features. + +Both formats install the same systemd unit (`packaging/nasmon.service`, run +as root — it needs `smartctl`/`zpool`), just at their respective conventional +paths (`/lib/systemd/system` on deb, `/usr/lib/systemd/system` on rpm), plus +the bundled `wsdd.service` unit (Ubuntu/Fedora `wsdd` packages ship no unit +of their own), default config at `/etc/nasmon/config.json`, and +`postinst.sh`/`prerm.sh`/`postrm.sh` maintainer scripts (written to run +correctly under both dpkg and rpm argument conventions). + +## CI + +`.gitea/workflows/build.yml` runs `go vet`/`go test` and builds both +packages on every push, uploading `dist/*.deb` and `dist/*.rpm` as build +artifacts. diff --git a/build.ps1 b/build.ps1 new file mode 100644 index 0000000..9081323 --- /dev/null +++ b/build.ps1 @@ -0,0 +1,87 @@ +<# +.SYNOPSIS + Builds the nasmon binary (linux/amd64, static) and packs it into a .deb and .rpm with nfpm. + Output: dist\nasmon__amd64.deb and dist\nasmon-..rpm + + One-time setup: + winget install GoLang.Go + go install github.com/goreleaser/nfpm/v2/cmd/nfpm@latest + +.EXAMPLE + .\build.ps1 # version from git tag, or 0.1.0+ + .\build.ps1 -Version 0.2.0 + .\build.ps1 -Test # run go vet + go test first +#> +[CmdletBinding()] +param( + [string]$Version, + [switch]$Test +) + +$ErrorActionPreference = "Stop" +$root = $PSScriptRoot +$src = $root +$dist = Join-Path $root "dist" + +# tools: go and nfpm may live outside PATH in a fresh shell +$env:Path = "$env:Path;C:\Program Files\Go\bin;$env:USERPROFILE\go\bin" +foreach ($tool in "go", "nfpm") { + if (-not (Get-Command $tool -ErrorAction SilentlyContinue)) { + throw "$tool not found - see the setup notes at the top of build.ps1" + } +} + +if (-not $Version) { + $Version = $null + if (Get-Command git -ErrorAction SilentlyContinue) { + Push-Location $root + try { $tag = git describe --tags --always --dirty 2>$null } catch { $tag = $null } + Pop-Location + if ($tag -and $tag -match '^v?(\d+\.\d+\.\d+)(-(\d+)-g([0-9a-f]+))?(-dirty)?$') { + $Version = $Matches[1] + if ($Matches[3]) { $Version += "+git$($Matches[3]).$($Matches[4])" } + if ($Matches[5]) { $Version += ".dirty" } + } + } + if (-not $Version) { $Version = "0.1.0+" + (Get-Date -Format "yyyyMMdd.HHmm") } +} + +New-Item -ItemType Directory -Force $dist | Out-Null + +if ($Test) { + Write-Host "==> go vet / go test" -ForegroundColor Green + Push-Location $src + try { + go vet ./...; if ($LASTEXITCODE -ne 0) { throw "go vet failed" } + go test ./...; if ($LASTEXITCODE -ne 0) { throw "go test failed" } + } finally { Pop-Location } +} + +Write-Host "==> Building nasmon $Version (linux/amd64)" -ForegroundColor Green +$env:GOOS = "linux"; $env:GOARCH = "amd64"; $env:CGO_ENABLED = "0" +Push-Location $src +try { + go build -trimpath -ldflags "-s -w -X main.version=$Version" -o (Join-Path $dist "nasmon") ./cmd/nasmon + if ($LASTEXITCODE -ne 0) { throw "go build failed" } +} finally { + Pop-Location + Remove-Item Env:GOOS, Env:GOARCH, Env:CGO_ENABLED -ErrorAction SilentlyContinue +} + +Write-Host "==> Packaging .deb" -ForegroundColor Green +Get-ChildItem $dist -Filter "nasmon_*.deb" | Remove-Item -Force +$env:NASMON_VERSION = $Version +Push-Location (Join-Path $root "packaging") +try { + nfpm package -f nfpm.yaml -p deb -t $dist + if ($LASTEXITCODE -ne 0) { throw "nfpm deb failed" } + + Write-Host "==> Packaging .rpm" -ForegroundColor Green + Get-ChildItem $dist -Filter "nasmon-*.rpm" | Remove-Item -Force + nfpm package -f nfpm.yaml -p rpm -t $dist + if ($LASTEXITCODE -ne 0) { throw "nfpm rpm failed" } +} finally { Pop-Location } + +foreach ($pkg in (Get-ChildItem $dist -Filter "nasmon*" -Include "*.deb", "*.rpm")) { + Write-Host "==> $($pkg.FullName) ($([math]::Round($pkg.Length / 1MB, 1)) MB)" -ForegroundColor Green +} diff --git a/build.sh b/build.sh new file mode 100644 index 0000000..dfd9ce1 --- /dev/null +++ b/build.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# Builds the nasmon binary (linux/amd64, static) and packs it into .deb and .rpm with nfpm. +# Output: dist/nasmon__amd64.deb and dist/nasmon-..rpm +# +# One-time setup: +# go install github.com/goreleaser/nfpm/v2/cmd/nfpm@latest +# +# Usage: +# ./build.sh # version from git tag, or 0.1.0+ +# VERSION=0.2.0 ./build.sh +# TEST=1 ./build.sh # run go vet + go test first + +set -euo pipefail + +root="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +dist="$root/dist" + +for tool in go nfpm; do + command -v "$tool" >/dev/null 2>&1 || { echo "$tool not found - see the setup notes at the top of build.sh" >&2; exit 1; } +done + +version="${VERSION:-}" +if [ -z "$version" ] && command -v git >/dev/null 2>&1; then + tag="$(cd "$root" && git describe --tags --always --dirty 2>/dev/null || true)" + if [[ "$tag" =~ ^v?([0-9]+\.[0-9]+\.[0-9]+)(-([0-9]+)-g([0-9a-f]+))?(-dirty)?$ ]]; then + version="${BASH_REMATCH[1]}" + [ -n "${BASH_REMATCH[3]}" ] && version+="+git${BASH_REMATCH[3]}.${BASH_REMATCH[4]}" + [ -n "${BASH_REMATCH[5]}" ] && version+=".dirty" + fi +fi +[ -z "$version" ] && version="0.1.0+$(date +%Y%m%d.%H%M)" + +mkdir -p "$dist" + +if [ "${TEST:-}" = "1" ]; then + echo "==> go vet / go test" + (cd "$root" && go vet ./... && go test ./...) +fi + +echo "==> Building nasmon $version (linux/amd64)" +(cd "$root" && GOOS=linux GOARCH=amd64 CGO_ENABLED=0 \ + go build -trimpath -ldflags "-s -w -X main.version=$version" -o "$dist/nasmon" ./cmd/nasmon) + +echo "==> Packaging .deb" +rm -f "$dist"/nasmon_*.deb +(cd "$root/packaging" && NASMON_VERSION="$version" nfpm package -f nfpm.yaml -p deb -t "$dist") + +echo "==> Packaging .rpm" +rm -f "$dist"/nasmon-*.rpm +(cd "$root/packaging" && NASMON_VERSION="$version" nfpm package -f nfpm.yaml -p rpm -t "$dist") + +for pkg in "$dist"/nasmon*.deb "$dist"/nasmon*.rpm; do + [ -e "$pkg" ] || continue + size_mb=$(du -m "$pkg" | cut -f1) + echo "==> $pkg (${size_mb} MB)" +done diff --git a/cmd/nasmon/main.go b/cmd/nasmon/main.go new file mode 100644 index 0000000..6b2a5e0 --- /dev/null +++ b/cmd/nasmon/main.go @@ -0,0 +1,84 @@ +// nasmon — lightweight NAS health monitor: web dashboard + SMART/ZFS alerts by e-mail. +package main + +import ( + "context" + "errors" + "flag" + "fmt" + "log" + "net/http" + "os" + "os/signal" + "path/filepath" + "syscall" + "time" + + "nasmon/internal/alert" + "nasmon/internal/config" + "nasmon/internal/history" + "nasmon/internal/sched" + "nasmon/internal/shares" + "nasmon/internal/store" + "nasmon/internal/web" +) + +var version = "dev" + +func main() { + listen := flag.String("listen", ":8090", "HTTP listen address") + dataDir := flag.String("data", "/var/lib/nasmon", "directory for the SQLite database") + cfgPath := flag.String("config", "/etc/nasmon/config.json", "settings file (created on first save)") + showVer := flag.Bool("version", false, "print version and exit") + flag.Parse() + if *showVer { + fmt.Println("nasmon", version) + return + } + log.SetFlags(log.LstdFlags | log.Lmsgprefix) + + cfg, err := config.Load(*cfgPath) + if err != nil { + log.Fatalf("config: %v", err) + } + if err := os.MkdirAll(*dataDir, 0o750); err != nil { + log.Fatalf("data dir: %v", err) + } + st, err := store.Open(filepath.Join(*dataDir, "nasmon.db")) + if err != nil { + log.Fatalf("database: %v", err) + } + defer st.Close() + + sh, err := shares.Load(filepath.Join(filepath.Dir(*cfgPath), "shares.json")) + if err != nil { + log.Fatalf("shares config: %v", err) + } + + ring := history.New(24 * 60 * 6) // 24 h at 10 s + am := &alert.Manager{Store: st, Config: cfg, Send: alert.SendMail} + sc := sched.New(cfg, st, ring, am, version) + sc.LoadCachedSmart() + + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + defer stop() + go sc.Run(ctx) + + srv := &http.Server{ + Addr: *listen, + Handler: web.New(cfg, st, ring, sc, sh), + ReadHeaderTimeout: 10 * time.Second, + } + go func() { + log.Printf("nasmon %s listening on %s", version, *listen) + if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) { + log.Fatalf("http: %v", err) + } + }() + + <-ctx.Done() + log.Print("shutting down") + shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + _ = srv.Shutdown(shutdownCtx) +} diff --git a/go.mod b/go.mod new file mode 100644 index 0000000..7a2ab1d --- /dev/null +++ b/go.mod @@ -0,0 +1,21 @@ +module nasmon + +go 1.22 + +require modernc.org/sqlite v1.34.4 + +require ( + github.com/dustin/go-humanize v1.0.1 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/ncruces/go-strftime v0.1.9 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect + golang.org/x/sys v0.22.0 // indirect + modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect + modernc.org/libc v1.55.3 // indirect + modernc.org/mathutil v1.6.0 // indirect + modernc.org/memory v1.8.0 // indirect + modernc.org/strutil v1.2.0 // indirect + modernc.org/token v1.1.0 // indirect +) diff --git a/go.sum b/go.sum new file mode 100644 index 0000000..c337282 --- /dev/null +++ b/go.sum @@ -0,0 +1,49 @@ +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo= +github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= +github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= +golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic= +golang.org/x/mod v0.16.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI= +golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw= +golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc= +modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ= +modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ= +modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y= +modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s= +modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE= +modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ= +modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw= +modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI= +modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4= +modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U= +modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w= +modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4= +modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo= +modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E= +modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU= +modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4= +modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0= +modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc= +modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss= +modernc.org/sqlite v1.34.4 h1:sjdARozcL5KJBvYQvLlZEmctRgW9xqIZc2ncN7PU0P8= +modernc.org/sqlite v1.34.4/go.mod h1:3QQFCG2SEMtc2nv+Wq4cQCH7Hjcg+p/RMlS1XK+zwbk= +modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA= +modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0= +modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= +modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= diff --git a/internal/alert/alert_test.go b/internal/alert/alert_test.go new file mode 100644 index 0000000..ed4ad25 --- /dev/null +++ b/internal/alert/alert_test.go @@ -0,0 +1,305 @@ +package alert + +import ( + "path/filepath" + "strings" + "testing" + "time" + + "nasmon/internal/collect" + "nasmon/internal/config" + "nasmon/internal/store" +) + +func keys(ev []Event) []string { + out := make([]string, len(ev)) + for i, e := range ev { + out[i] = e.Key + } + return out +} + +func has(ev []Event, key string) bool { + for _, e := range ev { + if e.Key == key { + return true + } + } + return false +} + +func TestEvaluateRules(t *testing.T) { + now := time.Date(2025, 9, 13, 12, 0, 0, 0, time.Local) + old := now.Add(-60 * 24 * time.Hour) + th := config.Defaults().Thresholds + in := Input{ + Now: now, Th: th, PoolsOK: true, RootOK: true, + Smarts: []collect.Smart{ + {Device: "/dev/sda", Serial: "A", Model: "HDD", Supported: true, Healthy: true, HasTemp: true, TempC: 55, + Attrs: []collect.SmartAttr{{ID: 5, Raw: 12}, {ID: 199, Raw: 5}}}, + {Device: "/dev/sdb", Serial: "B", Model: "HDD", Supported: true, Healthy: false, HasTemp: true, TempC: 30, + LastTest: &collect.SelfTest{Type: "Short offline", Status: "Completed: read failure", Passed: false}}, + {Device: "/dev/nvme0n1", Serial: "N", Model: "NVMe", Supported: true, Healthy: true, + NVMe: &collect.NVMeHealth{PercentageUsed: 95, AvailableSpare: 100}}, + {Device: "/dev/sdc", Serial: "C", Model: "HDD", Supported: true, Healthy: true, HasTemp: true, TempC: 35}, + }, + Prev: map[string]*collect.Smart{"A": {Serial: "A", Attrs: []collect.SmartAttr{{ID: 199, Raw: 3}}}}, + Disks: []collect.Disk{{Serial: "A"}, {Serial: "B"}, {Serial: "N"}, {Serial: "C"}}, + Seen: []store.SeenDisk{{Serial: "A"}, {Serial: "GONE", Model: "Old", Device: "/dev/sdz"}}, + Pools: []collect.Pool{ + {Name: "good", Health: "ONLINE", CapPct: 40, Size: 100, Free: 60, LastScrub: &now, Errors: "No known data errors"}, + {Name: "bad", Health: "DEGRADED", CapPct: 90, Size: 100, Free: 10, LastScrub: &old, + Vdevs: []collect.Vdev{{Name: "sda", Cksum: 2}}, Errors: "No known data errors"}, + {Name: "never", Health: "ONLINE", CapPct: 10, Size: 100, Free: 90}, + }, + CPUTemp: 85, HasCPU: true, + RootFS: collect.FSUsage{Total: 100, Free: 5}, + } + ev := Evaluate(in) + want := []string{ + "smart:A:attr", "smart:A:crc", "smart:A:temp", + "smart:B:health", "smart:B:selftest", + "smart:N:nvme", + "disk:GONE:missing", + "zfs:bad:health", "zfs:bad:errors", "zfs:bad:capacity", "zfs:bad:scrub", + "zfs:never:scrub", + "sys:cpu:temp", "sys:rootfs:space", + } + for _, k := range want { + if !has(ev, k) { + t.Errorf("missing event %s; got %v", k, keys(ev)) + } + } + if len(ev) != len(want) { + t.Errorf("got %d events, want %d: %v", len(ev), len(want), keys(ev)) + } + for _, e := range ev { + if strings.HasPrefix(e.Key, "smart:C:") || strings.HasPrefix(e.Key, "zfs:good:") { + t.Errorf("unexpected event %s", e.Key) + } + if e.Key == "smart:A:attr" && e.Level != LevelCrit { + t.Errorf("reallocated sectors must be crit, got %s", e.Level) + } + } + // timeouts / reported-uncorrect alone are only a warning + ev = Evaluate(Input{Now: now, Th: th, Smarts: []collect.Smart{{Serial: "S", Supported: true, Healthy: true, + Attrs: []collect.SmartAttr{{ID: 187, Raw: 6}, {ID: 188, Raw: 3}}}}}) + if len(ev) != 1 || ev[0].Key != "smart:S:attr" || ev[0].Level != LevelWarn { + t.Errorf("187/188 only: %+v", ev) + } +} + +func TestEvaluateDisabledThresholds(t *testing.T) { + th := config.Thresholds{} // all zero = disabled + in := Input{Now: time.Now(), Th: th, PoolsOK: true, RootOK: true, HasCPU: true, CPUTemp: 99, + Smarts: []collect.Smart{{Serial: "A", Supported: true, Healthy: true, HasTemp: true, TempC: 99}}, + Pools: []collect.Pool{{Name: "p", Health: "ONLINE", CapPct: 99, Errors: "No known data errors"}}, + RootFS: collect.FSUsage{Total: 100, Free: 1}} + if ev := Evaluate(in); len(ev) != 0 { + t.Errorf("thresholds disabled but got %v", keys(ev)) + } +} + +type mailSink struct { + subjects []string + bodies []string + fail bool +} + +func (m *mailSink) send(_ config.SMTP, subject, body string) error { + if m.fail { + return errTest + } + m.subjects = append(m.subjects, subject) + m.bodies = append(m.bodies, body) + return nil +} + +type testErr struct{} + +func (testErr) Error() string { return "boom" } + +var errTest = testErr{} + +func newManager(t *testing.T) (*Manager, *mailSink) { + t.Helper() + dir := t.TempDir() + st, err := store.Open(filepath.Join(dir, "t.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { st.Close() }) + cfg, err := config.Load(filepath.Join(dir, "config.json")) + if err != nil { + t.Fatal(err) + } + c := cfg.Get() + c.SMTP.Host, c.SMTP.To = "localhost", []string{"a@b"} + c.RemindHours = 2 + if err := cfg.Save(c); err != nil { + t.Fatal(err) + } + sink := &mailSink{} + return &Manager{Store: st, Config: cfg, Send: sink.send}, sink +} + +func TestReconcileLifecycle(t *testing.T) { + m, sink := newManager(t) + t0 := time.Date(2025, 9, 13, 10, 0, 0, 0, time.UTC) + ev := []Event{{Key: "zfs:tank:health", Level: LevelCrit, Title: "Pool tank is DEGRADED", Detail: "d"}} + + out, err := m.Reconcile(Input{Now: t0}, ev) + if err != nil { + t.Fatal(err) + } + if len(out.Opened) != 1 || !out.Mailed || len(sink.subjects) != 1 { + t.Fatalf("open: %+v mails=%v", out, sink.subjects) + } + if !strings.Contains(sink.subjects[0], "Pool tank is DEGRADED") { + t.Errorf("subject = %q", sink.subjects[0]) + } + + // same event again shortly after: nothing new, no mail + out, _ = m.Reconcile(Input{Now: t0.Add(time.Hour)}, ev) + if len(out.Opened) != 0 || out.Mailed || len(sink.subjects) != 1 { + t.Fatalf("no-change cycle mailed: %+v", out) + } + + // after RemindHours: reminder mail + out, _ = m.Reconcile(Input{Now: t0.Add(3 * time.Hour)}, ev) + if len(out.Reminded) != 1 || !out.Mailed || len(sink.subjects) != 2 { + t.Fatalf("reminder: %+v mails=%v", out, sink.subjects) + } + if !strings.Contains(sink.bodies[1], "STILL OPEN") { + t.Errorf("reminder body: %q", sink.bodies[1]) + } + + // frozen prefix: alert must survive a cycle without data + out, _ = m.Reconcile(Input{Now: t0.Add(4 * time.Hour), Frozen: []string{"zfs:"}}, nil) + if len(out.Resolved) != 0 { + t.Fatalf("frozen alert was resolved") + } + + // condition gone: resolved + mail + out, _ = m.Reconcile(Input{Now: t0.Add(5 * time.Hour)}, nil) + if len(out.Resolved) != 1 || !out.Mailed || !strings.Contains(sink.bodies[len(sink.bodies)-1], "RESOLVED") { + t.Fatalf("resolve: %+v", out) + } + openNow, _ := m.Store.OpenAlerts() + if len(openNow) != 0 { + t.Errorf("still open: %+v", openNow) + } +} + +func TestReconcileMailFailureKeepsUnnotified(t *testing.T) { + m, sink := newManager(t) + sink.fail = true + t0 := time.Now() + ev := []Event{{Key: "smart:A:health", Level: LevelCrit, Title: "x", Detail: "y"}} + out, _ := m.Reconcile(Input{Now: t0}, ev) + if out.Mailed || out.MailErr == nil { + t.Fatalf("expected mail error: %+v", out) + } + open, _ := m.Store.OpenAlerts() + if len(open) != 1 || open[0].LastNotifiedAt != nil { + t.Fatalf("alert should be open and un-notified: %+v", open) + } + // within the backoff window nothing is attempted + sink.fail = false + out, _ = m.Reconcile(Input{Now: t0.Add(time.Minute)}, ev) + if out.Mailed || out.MailErr != nil { + t.Fatalf("mail attempted inside backoff: %+v", out) + } + // after the backoff the un-notified alert is retried as new + out, _ = m.Reconcile(Input{Now: t0.Add(mailRetryAfter + time.Minute)}, ev) + if !out.Mailed || len(out.Opened) != 1 { + t.Fatalf("un-notified alert not retried: %+v", out) + } +} + +func TestReconcileAcknowledged(t *testing.T) { + m, sink := newManager(t) + t0 := time.Date(2025, 9, 13, 10, 0, 0, 0, time.UTC) + ev := []Event{{Key: "smart:A:attr", Level: LevelWarn, Title: "SMART attributes degraded", Detail: "187 Reported_Uncorrect raw=6"}} + out, _ := m.Reconcile(Input{Now: t0}, ev) + if len(out.Opened) != 1 || len(sink.subjects) != 1 { + t.Fatalf("open: %+v", out) + } + id := out.Opened[0].ID + if err := m.Store.AckAlert(id, t0.Add(time.Minute)); err != nil { + t.Fatal(err) + } + + // acknowledged + unchanged: no reminder even after RemindHours + out, _ = m.Reconcile(Input{Now: t0.Add(5 * time.Hour)}, ev) + if out.Mailed || len(out.Reminded) != 0 { + t.Fatalf("acked alert reminded: %+v", out) + } + if act, _ := m.Store.ActiveAlerts(); len(act) != 0 { + t.Fatalf("acked alert still active: %+v", act) + } + + // counters grew: re-raised as new, mailed, visible again + ev[0].Detail = "187 Reported_Uncorrect raw=9" + out, _ = m.Reconcile(Input{Now: t0.Add(6 * time.Hour)}, ev) + if len(out.Opened) != 1 || !out.Mailed || len(sink.subjects) != 2 { + t.Fatalf("re-raise: %+v mails=%v", out, sink.subjects) + } + act, _ := m.Store.ActiveAlerts() + if len(act) != 1 || act[0].Detail != "187 Reported_Uncorrect raw=9" || act[0].Acked() { + t.Fatalf("re-raised alert: %+v", act) + } + + // acknowledged again, then escalated warn -> crit: re-raised + _ = m.Store.AckAlert(id, t0.Add(7*time.Hour)) + ev[0].Level = LevelCrit + out, _ = m.Reconcile(Input{Now: t0.Add(8 * time.Hour)}, ev) + if len(out.Opened) != 1 || len(sink.subjects) != 3 { + t.Fatalf("escalation not re-raised: %+v", out) + } + + // acknowledged, then the problem disappears: resolved normally + _ = m.Store.AckAlert(id, t0.Add(9*time.Hour)) + out, _ = m.Reconcile(Input{Now: t0.Add(10 * time.Hour)}, nil) + if len(out.Resolved) != 1 || len(sink.subjects) != 4 { + t.Fatalf("resolve of acked: %+v", out) + } + if open, _ := m.Store.OpenAlerts(); len(open) != 0 { + t.Fatalf("still open: %+v", open) + } +} + +func TestStoreAlertLog(t *testing.T) { + m, _ := newManager(t) + st := m.Store + t0 := time.Now().Add(-30 * 24 * time.Hour) + old, _ := st.OpenAlert("k:old", LevelWarn, "old", "d", t0) + _ = st.ResolveAlert(old.ID, t0.Add(time.Hour)) + fresh, _ := st.OpenAlert("k:fresh", LevelWarn, "fresh", "d", time.Now().Add(-2*time.Hour)) + _ = st.ResolveAlert(fresh.ID, time.Now().Add(-time.Hour)) + cur, _ := st.OpenAlert("k:cur", LevelCrit, "cur", "d", time.Now()) + + recent, _ := st.RecentAlerts(100, time.Now().Add(-7*24*time.Hour)) + if len(recent) != 2 { + t.Fatalf("recent = %+v", recent) + } + if n, _ := st.CountResolvedBefore(time.Now().Add(-7 * 24 * time.Hour)); n != 1 { + t.Errorf("hidden count = %d", n) + } + all, _ := st.RecentAlerts(100, time.Time{}) + if len(all) != 3 { + t.Fatalf("all = %+v", all) + } + // open alerts cannot be deleted, resolved ones can + _ = st.DeleteAlert(cur.ID) + _ = st.DeleteAlert(old.ID) + all, _ = st.RecentAlerts(100, time.Time{}) + if len(all) != 2 { + t.Fatalf("after delete = %+v", all) + } + _ = st.PruneAlerts(time.Now().Add(-time.Minute)) + all, _ = st.RecentAlerts(100, time.Time{}) + if len(all) != 1 || all[0].ID != cur.ID { + t.Fatalf("after prune = %+v", all) + } +} diff --git a/internal/alert/mail.go b/internal/alert/mail.go new file mode 100644 index 0000000..0a9cafa --- /dev/null +++ b/internal/alert/mail.go @@ -0,0 +1,128 @@ +package alert + +import ( + "crypto/tls" + "errors" + "fmt" + "mime" + "net" + "net/smtp" + "strings" + "time" + + "nasmon/internal/config" +) + +// SendMail delivers a plain-text message using the configured SMTP account. +func SendMail(cfg config.SMTP, subject, body string) error { + if cfg.Host == "" { + return errors.New("SMTP host is not configured") + } + if len(cfg.To) == 0 { + return errors.New("no recipients configured") + } + from := cfg.From + if from == "" { + from = cfg.User + } + if from == "" { + return errors.New("sender address (from) is not configured") + } + addr := net.JoinHostPort(cfg.Host, fmt.Sprint(cfg.Port)) + dialer := net.Dialer{Timeout: 20 * time.Second} + + var c *smtp.Client + switch cfg.Security { + case "tls": + conn, err := tls.DialWithDialer(&dialer, "tcp", addr, &tls.Config{ServerName: cfg.Host}) + if err != nil { + return fmt.Errorf("tls connect: %w", err) + } + c, err = smtp.NewClient(conn, cfg.Host) + if err != nil { + conn.Close() + return err + } + default: + conn, err := dialer.Dial("tcp", addr) + if err != nil { + return fmt.Errorf("connect: %w", err) + } + conn.SetDeadline(time.Now().Add(60 * time.Second)) + c, err = smtp.NewClient(conn, cfg.Host) + if err != nil { + conn.Close() + return err + } + if cfg.Security == "starttls" { + if ok, _ := c.Extension("STARTTLS"); !ok { + c.Close() + return errors.New("server does not support STARTTLS") + } + if err := c.StartTLS(&tls.Config{ServerName: cfg.Host}); err != nil { + c.Close() + return fmt.Errorf("starttls: %w", err) + } + } + } + defer c.Close() + + if cfg.User != "" { + ok, mechs := c.Extension("AUTH") + var auth smtp.Auth + switch { + case ok && strings.Contains(mechs, "PLAIN"): + auth = smtp.PlainAuth("", cfg.User, cfg.Pass, cfg.Host) + case ok && strings.Contains(mechs, "LOGIN"): + auth = loginAuth{cfg.User, cfg.Pass} + case ok && strings.Contains(mechs, "CRAM-MD5"): + auth = smtp.CRAMMD5Auth(cfg.User, cfg.Pass) + default: + auth = smtp.PlainAuth("", cfg.User, cfg.Pass, cfg.Host) + } + if err := c.Auth(auth); err != nil { + return fmt.Errorf("auth: %w", err) + } + } + + if err := c.Mail(from); err != nil { + return fmt.Errorf("MAIL FROM: %w", err) + } + for _, to := range cfg.To { + if err := c.Rcpt(to); err != nil { + return fmt.Errorf("RCPT TO %s: %w", to, err) + } + } + w, err := c.Data() + if err != nil { + return err + } + msg := fmt.Sprintf("From: %s\r\nTo: %s\r\nSubject: %s\r\nDate: %s\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=utf-8\r\nContent-Transfer-Encoding: 8bit\r\n\r\n%s", + from, strings.Join(cfg.To, ", "), mime.QEncoding.Encode("utf-8", subject), + time.Now().Format(time.RFC1123Z), strings.ReplaceAll(body, "\n", "\r\n")) + if _, err := w.Write([]byte(msg)); err != nil { + return err + } + if err := w.Close(); err != nil { + return err + } + return c.Quit() +} + +// loginAuth implements the legacy AUTH LOGIN mechanism (Yandex, Mail.ru, Exchange). +type loginAuth struct{ user, pass string } + +func (a loginAuth) Start(*smtp.ServerInfo) (string, []byte, error) { return "LOGIN", nil, nil } + +func (a loginAuth) Next(fromServer []byte, more bool) ([]byte, error) { + if !more { + return nil, nil + } + switch strings.ToLower(strings.TrimSpace(string(fromServer))) { + case "username:": + return []byte(a.user), nil + case "password:": + return []byte(a.pass), nil + } + return nil, fmt.Errorf("unexpected LOGIN challenge %q", fromServer) +} diff --git a/internal/alert/rules.go b/internal/alert/rules.go new file mode 100644 index 0000000..05092ce --- /dev/null +++ b/internal/alert/rules.go @@ -0,0 +1,212 @@ +// Package alert evaluates health rules, tracks open alerts and sends mail. +package alert + +import ( + "fmt" + "sort" + "strings" + "time" + + "nasmon/internal/collect" + "nasmon/internal/config" + "nasmon/internal/store" +) + +const ( + LevelWarn = "warn" + LevelCrit = "crit" +) + +// Event is one active problem. Key is stable across evaluations so the same +// condition maps to the same open alert: ":

Device	Model	Serial	Size	Bus	Temp	Health	Power-on	Self-test	Warnings
{{.Name}}	{{.Model}}	{{.Serial}}	{{bytes .Size}}	{{.Tran}}{{if not .Rota}} ssd{{end}}	{{if $sm.HasTemp}}{{$sm.TempC}}°C{{if .Standby}} zz{{end}}{{else}}—{{end}}	{{if .Error}}n/a{{else if not $sm.Supported}}n/a{{else if $sm.Healthy}}PASSED{{else}}FAILED{{end}}	{{if $sm.PowerOnHrs}}{{$sm.PowerOnHrs}} h{{else}}—{{end}}	{{with $sm.LastTest}}{{if .InProgress}}{{.Remaining}}% left{{else}}{{.Status}}{{end}}{{else}}—{{end}}	+ {{range $sm.Attrs}}{{if and (or (eq .ID 5) (eq .ID 187) (eq .ID 188) (eq .ID 197) (eq .ID 198)) (gt .Raw 0)}}{{.ID}}: {{.Raw}} {{end}}{{if .WhenFailed}}{{.ID}} {{.WhenFailed}} {{end}}{{end}} + {{with $sm.NVMe}}{{if .CriticalWarning}}warn 0x{{printf "%02x" .CriticalWarning}} {{end}}{{if .MediaErrors}}media err {{.MediaErrors}} {{end}}{{.PercentageUsed}}% used{{end}} +	—	{{if .Standby}}standby{{else if .Error}}n/a{{else}}pending{{end}}
no disks found

Pool	Health	Used	Free	Size	Capacity	Frag	Last scrub	Errors
{{.Name}}	{{.Health}}	{{bytes .Alloc}}	{{bytes .Free}}	{{bytes .Size}}	{{.CapPct}}%	{{.FragPct}}%	{{if .Scrubbing}}scrubbing{{else if .LastScrub}}{{agop .LastScrub}}{{else}}never{{end}}	{{if .Errors}}{{.Errors}}{{end}}
no pools

Name	Path	Access	Who	Connect as
{{.Name}}{{if .Comment}} {{.Comment}}{{end}}{{if not .Browseable}} hidden{{end}}	{{.Path}}	{{if .ReadOnly}}read-only{{else}}read/write{{end}}	{{if .Guest}}guest (no password){{else}}{{join .Users ", "}}{{end}}	\\{{$.Data.Host}}\{{.Name}}	+ edit · + +
No SMB shares yet.

Path	Clients	Access	Squash	Mount
{{.Path}}	{{join .Clients ", "}}	{{if .ReadOnly}}read-only{{else}}read/write{{end}}{{if .Async}} async{{end}}	{{.Squash}}	{{$.Data.Host}}:{{.Path}}	+ edit · + +
No NFS exports yet.

Alert log {{if .All}}everything{{else}}open + resolved in the last 7 days{{end}}

CPU {{.S.CPU.Cores}} cores

Memory {{bytes .S.Mem.Total}}

Temperature

System

Disks

ZFS pools details

Datasets

{{.D.Model}} {{.D.Serial}}

NVMe health

Temperature, last 30 days {{len .History}} snapshots

SMART attributes

History

Open alerts

E-mail (SMTP)

Alerts

SMART polling

Thresholds

Scheduled self-tests

SMB (Windows / macOS network shares) + {{if not .Svc.SMBInstalled}}not installed + {{else if .Svc.SMBActive}}running {{.Svc.SMBSessions}} session(s) + {{else}}stopped{{end}} +

Shares

SMB users

NFS (Linux / other NAS) + {{if not .Svc.NFSInstalled}}not installed + {{else if .Svc.NFSActive}}running + {{else}}stopped{{end}} +

{{.Name}} {{.Health}}

Datasets

{{.}}	+ +
No SMB users. Guest shares work without users.