HanDev QA Orchestrator

Challenge

The studio had a critical mass of Playwright test projects scattered across different repositories with different CI setups. No single place to see what failed, where, and why; flaky tests drowned in the noise; quality gates had to be computed manually.

What We Did

• Domain model: project → repository config / environments / suites / secrets / schedules / runs / tests, plus defect_rules, quality_gates, notification_channels, api_tokens, members.
• Fastify + Drizzle API with per-project permission matrix: viewer / runner / maintainer / admin. 9 permissions.
• BullMQ workers: run-test-runner (git clone → docker run → JSON report parsing → INSERT) and run-scheduler (30-s cron tick + watchdog + cleanup).
• Bind mounts — worker starts Playwright containers via docker socket; daemon resolves paths on host.
• API tokens qaop_* for CI: POST /webhooks/runs with Authorization: Bearer.
• Artefacts (HTML report, traces, video) served by nginx behind forward-auth, signed URLs.
• AI classification: defect_rules match errors to defect types; clustering by errorHash; auto-generated bug reports.
• Quality gates: rules on pass-rate / duration / defect classification; violated → run marked failed, deployment blocked.

Result

• All Playwright projects in a single dashboard. First connected: DieAlltagsfeen.
• Real run #1: 119 tests, 116 passed, 3 failed, 14 min 1 sec.
• Playwright HTML report clickable from the run card; artefacts under signed URLs.
• Stats over 30 days: pass rate by suite, duration by suite, top flaky tests.
• Project settings in 8 tabs: General / Repository / Environments / Suites / Secrets / Defect rules / Quality Gates / API tokens.