Distributed Job Queue Engine
A job queue that runs every job exactly once, even when workers die mid-execution.
Workers crash halfway through jobs. Naive queues respond by either dropping that work or running it twice, and both are unacceptable if the job charges a card or sends a message. The queue needs to survive a worker disappearing without a trace.
Claims go through PostgreSQL's SELECT ... FOR UPDATE SKIP LOCKED rather than a bespoke lock service. That inherits Postgres's crash semantics for free, at the cost of binding throughput to a single database — a ceiling worth accepting for correctness I did not have to prove myself.
- 1,393 jobs/sec sustained, 6ms p50 and 36ms p99 claim latency
- Exactly-once execution via TOCTOU-safe dual-layer idempotency
- Failed work lands in an in-table dead-letter queue instead of vanishing
- Crash recovery verified across 10,000 jobs with testcontainers-go
- Golang
- PostgreSQL
- gRPC
- Docker
- Prometheus
- Grafana