Controlled trials

Use this as a release gate for real but bounded assessment sessions. A successful demo is not enough: prove candidate recovery, evidence completeness, operator response, privacy controls, and end-to-end reconciliation under failure.

1. Scope and ownership

• Define candidate count, assessment duration, policy preset/overrides, supported browser/OS matrix, geography/time zone, and trial dates.
• Name owners for candidate support, platform operations, security/privacy, reviewer decisions, and incident escalation.
• Publish candidate requirements, permission instructions, privacy notice, retention period, deletion process, and appeal/support route.
• Decide which failures block entry, pause an active test, allow a retry, or require manual review.

2. Integration acceptance

• Use a unique correlationId per customer exam attempt and verify internal sitting/retake behaviour.
• Confirm the resolved policy is recorded and matches the intended preset/overrides.
• Verify standard and strict never enable webcam snapshots and recording together.
• Ensure gaze and identityMismatch are on unless an approved policy explicitly disables them.
• Save assessment answers before awaiting endSession/client.end().
• Handle attempt blocked, preflight failed, session startup error, integrity hold, already-completed reload, clean end, and abandonment.
• For raw SDK integrations, complete every responsibility in Raw SDK integration.

3. Candidate experience

• Test first-time camera/mic/speaker/fullscreen/screen-share prompts with plain instructions.
• Confirm screen-sharing starts from a user click and wrong-surface/cancel/stopped recovery works.
• Verify microphone preparation begins before “Test microphone”; on slow asset delivery, VAD gets at most 10 seconds after the click before internal RMS fallback takes over.
• Confirm no premature “no sound” alert appears while the detector is preparing.
• Test keyboard-only navigation, visible focus, zoom, screen-reader landmarks, and mobile docs/support access.
• Provide a support channel and candidate-safe incident reference id that maps to correlationId without exposing secrets.

4. Browser and network matrix

Run the browser/network trial matrix. Include throttled uplink, high latency/jitter, offline recovery, VPN/proxy, DNS filtering, privacy extensions, device removal, sleep/wake, refresh, and tab close.

Record actual SDK version/build SHA and production candidate origin for every test. Do not expand browser support based on API detection alone.

5. Evidence delivery

• Block session start when ingest or recording-storage readiness fails.
• Monitor upload-failed, dropped, and memory storage fallback.
• Confirm snapshots/recordings arrive throughout a full-duration session, not only at startup.
• Verify clean end drains final chunks and produces a terminal status.
• Confirm interrupted sessions are labelled incomplete/abandoned and never silently presented as clean.
• Play screen/webcam evidence and inspect the timeline, photo references, analysis state, and report.

6. Security and privacy

• Approve the exact evidence set and candidate notice.
• Keep sk_ keys off the browser and restrict pk_ keys by exact origin.
• Review clipboard/writing content capture; keep it off unless explicitly justified.
• Restrict reviewer/download/delete permissions and audit destructive operations.
• Agree a real-session retention and deletion process. The general automated retention sweeper is not currently shipped.
• Test deletion against both database and object storage, plus backup/legal-hold handling.
• Establish manual review and appeal handling for gaze, face, identity, object, and screen-analysis flags.

7. Operations and incident response

• Monitor API error/latency, event rejection, storage canary failure, direct-upload failure, queue backlog, analysis failure, webhook exhaustion, and object storage growth.
• Define thresholds and an on-call response for a trial session that cannot start or end.
• Document how to inspect by correlationId, start an additive retake, retry analysis, and delete a test session.
• Reconcile terminal sessions by API polling; webhooks are notifications, not the only source of truth.
• Prepare a pause/rollback rule if evidence loss, tenant leakage, consent/retention failure, or repeated candidate lockout occurs.

8. Exit criteria

Proceed only when:

• every supported browser/network scenario has a recorded result;
• startup readiness and mid-session delivery are observable;
• required evidence is complete and reviewable for full-duration sessions;
• clean end, interruption, retake, and deletion behave as documented;
• no critical/high security, privacy, tenant-isolation, or evidence-loss issue remains open;
• support, operations, reviewer, and privacy owners have signed off;
• known limitations are communicated to candidates and reviewers.

After the trial, compare failure rate, start latency, evidence completeness, end-drain duration, false-positive review, candidate support volume, and storage/analysis cost against the agreed acceptance thresholds.