Pilot Runbook
Pre-Flight Verification & Smoke Tests
6-point automated readiness check and catalog ingestion runbook.
Before rolling out Kholo to a distributor or going live with customer traffic, execute this automated verification protocol to validate database connectivity, background queues, cryptographic signatures, ERP endpoints, and catalog master data.
1. Running the Automated 6-Point Readiness Check
From your terminal, execute the production readiness test:
cd apps/api
npx ts-node src/scripts/verify-readiness.tsExpected Output on a Ready System:
===========================================================
️ KHOLO PRODUCTION READINESS & SECURITY CHECKLIST
===========================================================
[CHECK 1/6] Database (PostgreSQL): PASS (1 Tenants, 15 Active SKUs)
[CHECK 2/6] Broker (Redis 7 & Background Task Queue): PASS (Queue Live: {"waiting":0,"active":0,"completed":0,"failed":0})
[CHECK 3/6] Meta Webhook Security: PASS (HMAC-SHA256 Timing-Safe Signature Engine Verified)
[CHECK 4/6] Voice Transcription: PASS (Voice Pipeline Operational)
[CHECK 5/6] Sage ERP & Pick Slip: PASS (SO Format: SO-2026-8454, Pick Slip: 3 lines)
[CHECK 6/6] Production Security & RBAC: PASS (Bcrypt Hashing & Pin Engine Verified)
===========================================================
READINESS SCORE: [ 6 / 6 PASS ] -> 100% READY FOR DISTRIBUTOR PILOT!
===========================================================Failure Diagnosis:
- Check 1 Fails: Check
DATABASE_URLinapps/api/.env. Ensure PostgreSQL is running (docker compose up -d). - Check 2 Fails: Check
REDIS_HOSTandREDIS_PORT. Ensure Redis is running (docker compose up -d). - Check 3 Fails: Verify
WHATSAPP_VERIFY_TOKENis at least 6 characters in.env. - Check 4 Fails: Verify
OPENAI_API_KEYis configured for Internal Speech-to-Text Model transcription. - Check 5 Fails: Check network route to the Sage Freedom Service host.
2. Ingesting Pilot Master Data (import-pilot-catalog.ts)
Once the distributor provides their catalog and customer file (CSV/Excel), run the pilot ingestion script:
cd apps/api
npx ts-node src/scripts/import-pilot-catalog.tsWhat This Script Automates:
- Tenant Provisioning: Creates or updates the distributor's
Tenantrecord with default ERP configurations. - Product Catalog Ingestion:
- Creates SKUs, full descriptions, pack sizes, base prices, safety stock, and bin locations.
- Auto-generates baseline aliases (e.g. mapping "whites 12.5" to
WSTAR-12.5KG).
- Customer Master Ingestion:
- Formats all phone numbers to strict international E.164 (
+27...). - Sets starting credit status (
ACTIVE) and pricing tiers (TIER_A,DEFAULT).
- Formats all phone numbers to strict international E.164 (
3. End-to-End Trade Simulation Test (npm run test:sim)
To verify that the AI extractor and the South African FMCG trade canonicalizer accurately parse messy orders:
npm run test:simWhat It Tests:
- Mixed-language WhatsApp messages (English, Zulu, Afrikaans terms).
- Informal brand slang:
- "10 whites 12.5" $\rightarrow$
WSTAR-12.5KG - "4 cases stoney 2L" $\rightarrow$
STONEY-2L - "2 boxes zamalek" $\rightarrow$
CBL-500ML-24S - "3 tins big ricoffy" $\rightarrow$
RICOF-750G
- "10 whites 12.5" $\rightarrow$
- Calculates total parsing confidence score and verifies that all items match with $\ge 95%$ confidence.