audit funnel · semantic search · beacon HMAC

One funnel for everything

Every state-changing op across the ecosystem lands on orbit:audit. Obs server consumes, indexes, archives to MinIO. Beacons are HMAC-signed; mismatched beacons drop with audit.

Cold export

Events / sec

4,217

7d avg 3,800

Today

214M

vs 198M y-day

Crashes (24h)

38

3 new clusters

Beacons rejected

2

HMAC mismatch

Ingest · last 24h

events / min

Funnel health

healthy

Backlog

218 events (≤ 1s)

Hot retention

7 days · Redis

Cold archive

MinIO orbit-audit/ · 4.2 TB

Index lag

0.8 ms p50

Cluster

n3bul41 + n3bul42

Top event classes (1h)

edge_request

1.2M

sampled at 1% · 12,000 events

runner_step_*

4,124

94% complete · 6% failed

auth_login_*

1,082

98% ok · 2% fail

keyvault_unwrap

28,914

bulk · 482/min

semantic crash search

Semantic search

Type a symptom; the engine finds matching crash clusters via embeddings across crashes + runner_step_failed + audit_error.

exit 137OOMpublish.luawalzpunktstep 4/6~ Mon 8m

3 matches · 96% confidence

nopolarities/publish.lua · step sign · exit 137

14 occurrences · since 2026-05-22 · last 17:20

prod

docker container killed by OOM (memory.limit_in_bytes=2147483648) during step `sign` ~ minute 8

walzpunkt/deploy · step build-image · exit 137

2 occurrences · since 2026-05-26 · last 12:02

dev

buildx oom during layer export — same fingerprint as nopolarities cluster

brain-melt/publish · step bundle · exit 137

1 occurrence · 2026-05-28

dev

node heap OOM in webpack ssg pass — investigate worker pool memory cap

crash clusters · symptom-grouped

Crashes

Stack-trace + message fingerprinted; new fingerprints surface here automatically.

Crash	First	Last	Count	Surface
exit 137 · docker buildx · sign step runtime: cgroup memory limit exceeded · oom_killer invoked	2026-05-22	17:20	14	runner
e.au.in_pw · password mismatch · candidate@ user-input-only · expected after brute-force pattern	2026-05-26	17:52	8	auth
obs-3 · sse stream eof · client peer reset backend reset connection after 60s idle — fix HSTS idle timeout	2026-05-25	14:11	3	edge
tsig_key_mismatch · zone notify rejected secondary notify from 87.245.x.x · TSIG key wrong	2026-05-28	17:09	1	dns

beacons · HMAC-signed

Beacon stream

Live SSE tail. Beacons are tenant-tagged, signed with the producer's Tier-B HMAC key; verification fails route the event to the rejected bin.

Pause

time

source

payload

hmac

17:52:18.412

orbit-edge

edge_request walzpunkt.ch:/api/checkout 200 142ms (sampled)

verified

17:52:17.881

orbit-git

git_push walzpunkt main 8a4c…f2 · 1 ref · 12 objects

verified

17:52:16.221

orbit-runner

runner_step_start walzpunkt/deploy.lua step=build-image

verified

17:52:14.001

orbit-auth

token_minted scope=runner.write user=astra.kepler

verified

17:52:11.412

orbit-auth

auth_login_failed e.au.in_pw operator.candidate@orbit

verified

17:52:09.117

orbit-dns

zone_resigned orbit.zuerich 47 rrsigs in 312ms

verified

17:52:08.401

unknown

beacon dropped · hmac mismatch · from 87.245.x.x

REJECT

17:52:06.014

orbit-keyvault

tier_b_unwrap bulk count=482 (1m window)

verified

17:52:04.001

orbit-mail

outbound_signed k=orbit-2026q1 · 250 OK

verified

cold archive · MinIO orbit-audit/

Date	Tenant	Sources	Events	Size	Checksum
2026-05-27	all	12 sources	214,008,142	3.2 GB	blake3 · 8c2a…
2026-05-26	all	12 sources	198,142,401	3.0 GB	blake3 · 4f12…
2026-05-25	all	12 sources	189,884,212	2.9 GB	blake3 · 71fd…
2026-05-24	all	11 sources	142,012,884	2.1 GB	blake3 · 89a1…
2026-05-23	all	11 sources	128,442,001	1.9 GB	blake3 · 22c0…

Audit

Time	Event	Subject	Actor
17:52	beacon_rejected	hmac mismatch · from 87.245.x.x	system
14:11	archive_completed	2026-05-27 · 214M events	scheduler
12:02	crash_cluster_new	brain-melt/publish · exit 137	system
08:14	retention_trimmed	orbit:audit · 7d window	scheduler

RUM · first-party · cookieless

Page views

First-party, cookieless pageview analytics from the obs beacon. No third-party trackers, no fingerprinting — aggregate counts only, IP truncated at ingest.

Last 24h

Views · 24h

182k

+6.4% vs 7d

Unique visitors

41.2k

+3.1%

Avg time on page

1m 12s

median 42s

Bounce

38%

−2pts

Page	Property	Views	Uniques	Avg time	Bounce
/	orbit.zuerich	52,109	18,402	0m 38s	41%
/launch-2026	orbit.zuerich	31,884	12,910	2m 04s	22%
/shop	walzpunkt.ch	14,221	5,003	1m 31s	34%
/offerte	walzpunkt.ch	8,440	2,981	3m 12s	18%
/codex/eden	anno-2070	6,902	3,114	2m 47s	29%

Core Web Vitals · field data · p75

Web vitals

Real-user Core Web Vitals at the 75th percentile, from the beacon's web-vitals hook. Thresholds per Google's good / needs-improvement / poor bands.

LCP

good

1.9s

largest contentful paint · p75 · < 2.5s good

INP

needs work

214ms

interaction to next paint · p75 · 200–500ms

CLS

good

0.04

cumulative layout shift · p75 · < 0.1 good

Page	LCP p75	INP p75	CLS p75	Samples
orbit.zuerich/	1.6s	142ms	0.02	18,402
walzpunkt.ch/shop	2.4s	288ms	0.07	5,003
anno-2070/codex	3.1s	196ms	0.12	3,114

server-side · cap-token scoped · per-method

RPC latency

Per-method latency + error rate across the ecosystem's JSON-RPC / REST surfaces, sampled from the audit funnel envelopes. Slowest + erroring methods float to the top.

Method	Service	Calls 24h	p50	p95	p99	Err %
repo.push	orbit-git	4,201	142ms	880ms	2.1s	0.04%
token.mint	orbit-auth	14,201	8ms	22ms	61ms	0.00%
zone.resign	orbit-dns	312	298ms	512ms	1.1s	0.00%
run.dispatch	orbit-runner	842	44ms	180ms	420ms	1.2%
keyvault.unwrap	orbit-keyvault	28,110	3ms	9ms	28ms	0.00%

grouped by fingerprint · client + server

Errors

Uncaught exceptions + rejected promises grouped by stack fingerprint, deduped across releases. Distinct from Crashes (native/process-level).

Unresolved

Error	Type	Count 24h	Last seen	Release	Status
TypeError: cannot read 'id' of null	client	412	2m ago	station@1.9.2	unresolved
AbortError: signal aborted	client	188	14m ago	station@1.9.2	ignored
FetchError: 502 from upstream	server	61	1h ago	orbit-git@0.8	resolved
RangeError: invalid time value	client	23	3h ago	vapply@0.3	unresolved

from orbit-auth events · aggregate

Auth

Sign-in + sign-up funnels, WebAuthn step-up rates, and failure reasons — derived from auth_* events on the funnel. No PII; counts only.

Logins · 24h

3,902

+4%

Sign-ups

128

+12 vs 7d

WebAuthn step-up

71%

of admin actions

Failed logins

214

31 rate-limited

Failure reason	Count 24h	Top tenant	Trend
e.au.in_pw · bad password	142	Orbit AG	flat
e.au.rl_lo · rate-limited	31	Orbit AG	↑ spike 17:48
e.au.we_fl · webauthn timeout	28	Heinrich Graf	flat
e.au.tk_dl · token denylisted	13	Brain-Melt	↓

consent · DNT · retention · GDPR/FADP

Privacy

The beacon honours Do-Not-Track and consent state; IPs are truncated at ingest and raw events expire on a fixed window. Swiss FADP + GDPR posture at a glance.

Consent rate

82%

analytics opt-in

DNT honoured

9.1%

sessions not tracked

Raw retention

7d

then aggregate-only

Data requests

2

1 export · 1 erasure

Controls in effect

enforced

IP truncation

last octet zeroed at ingest

Cookies

none · cookieless beacon

Fingerprinting

disabled

Cross-site

first-party only

Data residency

CH · MinIO n3bul41/42

goals · funnels · per-property

Conversions

Operator-defined goals + multi-step funnels built on beacon events. Drop-off surfaces per step so a stalling checkout or sign-up is obvious.

Funnel · Walzpunkt offer request

7d

Visit /shop

5,003 · 100%

View product

2,704 · 54%

Open offer form

1,051 · 21%

Submit offer

442 · 9%

Goal	Property	Completions 24h	Rate	Trend
Offer submitted	walzpunkt.ch	63	9.2%	↑ +0.8pts
Launch signup	orbit.zuerich	128	4.1%	↑ +1.2pts
NFC card activated	orbit.zuerich	14	61%	flat

One funnel for everything

Ingest · last 24h

Funnel health

edge_request

runner_step_*

auth_login_*

keyvault_unwrap

Semantic search

Crashes

Beacon stream

Archive

Audit

Page views

Web vitals

LCP

INP

CLS

RPC latency

Errors

Auth

Privacy

Controls in effect

Conversions

Funnel · Walzpunkt offer request