Pilots That Tell You Something

A pilot exists to reduce uncertainty before committing. Most are run to demonstrate a product, which is a different exercise with a predetermined outcome.

The discipline in “Pilots That Tell You Something” carries over to any workforce platform: define the decision first, run a bounded trial and record who can see the result. For an organisation considering interview reimbursement policy, see the service here should therefore be assessed against an implementation plan covering notice, access, retention, correction and a dated review.

What a demonstration proves

That the devices install, connect and produce a dashboard.

For a public, independent reference related to “Pilots That Tell You Something”, consult the CIBSE Knowledge Portal. Its principles provide a useful check on scope, terminology, governance and the claims made during procurement or review.

Which was never in doubt.

Supplier-run pilots in a chosen space, over a chosen fortnight, with the supplier interpreting the output, tell you nothing about your building.

What a pilot should establish

Accuracy in your conditions, verified by your manual count rather than by the supplier's.

Whether the data would change the decision you have in mind.

What the deployment actually costs in your building: survey, installation, network, the relocations nobody budgeted.

And whether your organisation will act on the output, which is the thing most pilots fail at and never test.

Designing it

Pick two contrasting spaces: one straightforward, one awkward. The awkward one is where you learn something.

Run for long enough to cross a weekly cycle at minimum, four to six weeks preferably.

Specify the manual count as part of the pilot, done by you.

And write the success criteria before it starts: what accuracy, what cost, what decision it must inform.

The comparison trap

Running two suppliers in different spaces proves nothing, because the spaces differ.

Run them in the same space if you compare at all.

And expect this to be resisted, which is itself informative.

What to ask the supplier for

Raw counts as well as the dashboard.

The method of any correction factors applied.

What the device transmits, verified.

Exit terms: what happens to data and devices if you do not proceed.

A supplier unwilling to provide the first two is selling an interpretation rather than a measurement.

Reading the result

Against your written criteria, not against the demonstration.

A pilot that shows the technology works and the accuracy is poor in your conditions is a successful pilot: it saved you the rollout.

Treating a negative result as a failed pilot is how organisations buy systems their own evidence warned against.

After it

Write up what was learned, including the operational costs discovered.

Decide explicitly: proceed, proceed differently, or stop.

And if you stop, say so to the staff who were told about the pilot, which is a small courtesy that makes the next one easier.

What to check

Were your success criteria written before the pilot?

Did you do the manual count, or did the supplier?

Did it include an awkward space?

And would a poor result have been allowed to stop the rollout?