1. Product and workflow
This is a product-readiness checklist, not a security or regulatory certification. Apply the controls required by the system's actual risk, users, data, and operating context.
- A named user, valuable job, and clear current alternative.
- A defined boundary between what the assistant may answer, recommend, draft, or execute.
- Designed states for missing context, uncertainty, correction, escalation, and manual completion.
- A measurable outcome beyond conversation volume or model fluency.
2. Knowledge, data, and permission
- Approved sources with named owners, freshness expectations, and removal paths.
- Request-time access control that matches the current user and account.
- Sensitive data minimization, retention, deletion, and provider handling understood.
- Behavior for conflicting, stale, incomplete, or absent evidence.
- Source context or citations where users need to inspect the basis of an answer.
3. Evaluation and safe failure
- Representative questions and tasks from the real workflow, not only curated happy paths.
- Expected answers, acceptable variation, and failure categories agreed with domain owners.
- Separate checks for retrieval, answer quality, tool selection, action outcome, and permission correctness.
- Abstention, clarification, confirmation, or escalation when the system should not continue.
- Regression evaluation for prompt, model, retrieval, tool, and policy changes.
4. Tools, actions, and human handoff
See the full human-in-the-loop workflow guide when review is a material part of the operating model.
- Least-privilege credentials and server-side enforcement for every tool.
- Confirmation for consequential, external, destructive, financial, or difficult-to-reverse actions.
- Idempotency, retries, timeouts, duplicate handling, and reconciliation for integrations.
- A reviewer sees the input, evidence, proposed action, reason for review, and explicit choices.
- Every unresolved or overdue case has an owner and fallback route.
5. Observability, cost, and reliability
- Versioned model, prompt, retrieval settings, tool definitions, and evaluation set.
- Logs that connect request, retrieved context, tool calls, failures, latency, cost, and user outcome without exposing unnecessary sensitive data.
- Quality, failure, correction, escalation, latency, and cost signals with owners and thresholds.
- Rate, budget, provider outage, degraded mode, and manual continuity behavior.
- Runbooks for common failure classes and a clear incident contact.
6. Rollout and lifecycle ownership
- Shadow, internal, limited-cohort, and broader release stages chosen by risk.
- Acceptance thresholds, rollback condition, and review date for each stage.
- Named owners for data, retrieval, prompts, evaluation, integrations, incidents, user feedback, and release decisions.
- A schedule for dependency, model, policy, access, and drift review.
- A route to narrow or retire the assistant if value, safety, or operating economics change.
Choose the smallest next step
If the checklist exposes an unresolved feasibility question, test it through a Rapid POC. If the capability works but the product and operating gaps remain, use AI product development. If the main problem is orchestration across existing tools and reviewers, start with AI workflow automation.
Sources and further reading
- NIST: AI Risk Management Framework Core : Deployment-like evaluation, continuous monitoring, documented risk, and safe failure.
- OWASP: LLM06 Excessive Agency : Least privilege, downstream authorization, approval, logging, and rate limits.
- OpenAI: Evaluation best practices : Representative cases, separate tool and output checks, regression tests, and edge cases.
