Pilot playbook

This page is the six-week pilot template. The pilot exercises every workflow end-to-end with a small AR cohort (5 to 10 ARs) before the firm commits to full network rollout. The output of the pilot is a go/no-go decision for the remaining ARs in the network.

The pilot assumes the steps in adoption path are complete through Step 7 (catalogue and rubric configuration). The pilot covers Step 8.

Cohort selection

The cohort is 5 to 10 ARs picked to exercise the workflow end-to-end. The recommended composition:

At least one AR in each risk band (low, moderate, elevated, high). Critical-band ARs are typically excluded from a pilot because the workspace’s value to them is too high to risk on a pilot timeline; they are first in the Wave 1 rollout that follows.
At least one self-employed AR (the PS22/11 case).
At least one AR with supportsImportantBusinessService=true (the SYSC 15A case).
At least one AR with a recent breach or upheld complaint (so the breach workflow runs end-to-end on a real example).
At least one AR who is comfortable with software adoption and at least one who is sceptical. The sceptic’s feedback is more useful than the enthusiast’s.

The principal-admin sends invitation emails to the cohort’s AR-users in week 1.

Week 1: install and train

Day 1. Operator-led training session for the principal-side users (60 minutes, video call). Covers the dashboard, the AR register, the breach triage queue, the file-review workspace, the firm settings, and the audit log.

Day 2. Operator-led training session for the cohort AR-users (45 minutes, video call). Covers the AR home, the MI return submission, the breach reporting flow, and the AR’s view of their own audit log.

Day 3 to 5. Cohort AR-users sign in, complete MFA enrolment (where the per-tenant flag is on), and explore the surface. The firm’s principal-admin watches the user-management surface for stalled enrolments and chases.

Day 5. Pilot kickoff retro. The firm’s compliance lead and the operator review the first-week metrics (sign-in rate, time-to-first-action, support tickets) and confirm the cohort is ready for week 2.

Outputs by end of week 1: every cohort AR-user has signed in at least once. The principal-side dashboard shows the cohort. The firm’s settings (brand kit, risk weights, rubric) are confirmed.

Week 2: first MI returns

The cohort submits the current quarter’s MI return.

Day 6 to 10. AR-users open /ar/mi, fill the return (volumes, complaints, breaches, conduct events, cancellations), and submit. The principal-side compliance team reviews each return as it lands.

Day 8. First-pass anomaly review. The MI anomaly score is computed against the AR’s 8-quarter trailing distribution; on a first submission, the trailing distribution is empty and the anomaly score defaults to 0. The anomaly signal builds over subsequent quarters.

Day 10. Compliance team marks each return accepted or queried. A queried return triggers a notification to the AR-user with the principal-side compliance officer’s note; the AR-user revises and resubmits.

Outputs by end of week 2: every cohort AR has a Q-N MI return submitted and either accepted or in query. The risk score recomputes for every AR with a submitted return.

Week 3: first file reviews

The principal-side compliance team runs file reviews against the cohort.

Day 11. The team picks 1 to 3 customer files per AR using the firm’s sampling rule (typical: 5% of customer files per quarter, weighted by risk band). The case references go into the file-review workspace.

Day 12 to 15. Reviewers work through the rubric (MCOB, ICOBS, or CONC depending on vertical) per case. Each finding is pass, advisory, fail, or n/a, with evidence and (where applicable) remediation. The reviewer saves inline; the workspace prevents complete while findings are unresolved.

Day 15. First reviews completed. Each AR’s risk score recomputes with the file-review-score input now populated. The cohort AR-users see the completed reviews in their own surface, with the Challenge affordance available for 10 working days.

Outputs by end of week 3: at least one completed file review per cohort AR. The rubric is exercised against real evidence. The reviewer’s experience is captured for the rollout decision.

Week 4: first breach end-to-end

The cohort exercises the breach workflow.

Day 16 to 17. A pilot breach scenario: the firm pre-arranges with one cohort AR to file a representative breach (a real one if available, a synthetic one otherwise). The AR opens /ar/breaches/new and submits with the SUP 15-relevant fields (category, severity, customer impact, awareAt, description).

Day 18. The breach lands in the principal-side breach triage queue with a SUP 15 timing widget showing the notification deadline. The principal-compliance-officer triages, classifies materiality, and either marks in-remediation (not notifiable) or progresses to FCA notification.

Day 19. If the breach is material or significant, the principal-admin records the FCA notification (POST /api/breaches/:id/notify-fca, step-up authentication required). The actual notification to the FCA happens out of band via standard channels; the workspace records the timestamp and the actor.

Day 20. Breach progresses to resolved and then closed. The audit chain captures every state transition with attribution.

Outputs by end of week 4: one breach has been filed by an AR, triaged by the principal, and either notified to the FCA or resolved without notification. The SUP 15 clock and the step-up authentication on the terminal action have been exercised. The breach detail page is in a state the firm can show its board.

Week 5: first annual review packet

The principal-side team runs an annual review packet for one cohort AR (the AR whose lastAnnualReviewAt is closest to a year ago, or the AR with the most pilot activity if all are recently reviewed).

Day 21 to 23. Compliance officer drafts the annual review (PATCH /api/annual-reviews/:id), aggregating the AR’s risk trajectory, breach summaries, file review summaries, MI returns, and conduct events. The packet’s references resolve to specific audit event ids so the packet is reproducible from the chain.

Day 24. Compliance officer submits to principal-admin (status in-review).

Day 25. Principal-admin (acting as named director) signs off (POST /api/annual-reviews/:id/sign-off, step-up authentication required). The sign-off carries the director’s name, the FRN, the verbatim signOffNotes, and the second-factor event id.

Day 25. PDF export. The packet exports with the firm’s brand kit applied. The firm’s compliance lead and SMF16 review the PDF for board-readiness.

Outputs by end of week 5: one signed-off annual review packet, exported as PDF. The director sign-off has been exercised with step-up authentication. The replay capability (reconstructing the packet from the audit chain) has been demonstrated.

Week 6: review and decide

Day 26. Pilot retro with the cohort AR-users (45 minutes, video call). What worked, what was friction, what is missing. The firm’s compliance lead captures the feedback.

Day 27. Pilot retro with the principal-side users. The firm’s compliance lead captures the feedback.

Day 28 to 29. The firm’s compliance lead and SMF16 review the pilot artefacts:

Sign-in rates and time-to-first-action by cohort AR.
Support tickets raised during the pilot.
Reviewer hours per file review (the pilot’s efficiency signal).
Anomalies the workspace surfaced that the firm would not otherwise have caught.
The signed-off annual review packet.
The breach end-to-end artefact.

Day 30. Go/no-go decision. The firm decides to:

Roll out to the full network on the wave plan in adoption path Step 9. Most pilots reach this decision.
Extend the pilot by 4 to 6 weeks with the addition of one or two specific scenarios (typically a multi-AR breach or an annual review for a high-risk AR).
Discontinue. This is rare; if the pilot reaches discontinue, the firm and the operator review the friction points to understand whether the workspace is the wrong fit or whether the firm’s existing processes need adjustment first.

Pilot success criteria

A pilot is “successful” if:

Every cohort AR-user has completed at least one MI return submission and one breach reporting flow without operator support beyond the week-1 training.
Every cohort AR has at least one completed file review with the rubric exercised end-to-end.
The firm has signed off at least one annual review packet using the step-up authentication and the brand-kitted PDF export.
The firm’s compliance lead and SMF16 are confident the workspace can replace the spreadsheet, email, or off-the-shelf tool currently in use.
The firm’s view of the audit chain matches the workspace’s view (no integrity flags, no challenged reviews left unresolved, no suspended sessions).

If any of these is not met by day 30, the firm extends the pilot rather than rolling out to the full network.

What the operator provides during the pilot

The operator’s pilot support:

Two training sessions in week 1 (principal-side and AR-user).
Same-day support response (within 4 working hours) for any pilot-blocking issue.
A weekly check-in call with the firm’s compliance lead.
A pilot retro and a written summary at day 30.
Configuration adjustments (rubric tweaks, risk-weight adjustments, brand-kit updates) on request without a 30-day notice window during the pilot.

After day 30, the operator’s support reverts to the standard SLA in the master service agreement.

Operational notes

The pilot runs in the production tenant, not in a separate sandbox. This is deliberate: the artefacts produced during the pilot are real supervisory records and must persist. A separate sandbox would force the firm to repeat the work after rollout.

The pilot’s audit chain is the tenant’s chain. There is no “reset” at the end of the pilot; the chain continues into the full network rollout.