Let’s Run a Pilot

At work I hear a lot about groups looking to run what they call “pilots” which essentially means moving forward with an idea rather than going through the due diligence and thoughtful analysis one would put together for an initiative/project. The result tends to be that time, money, and resources are wasted spent on creating/modifying a process or bringing new technology in-house without 1. A plan on how to support it (if IT is even told about this at all), 2. No real idea if it solves a problem or satisfies a business need, and 3. Absolutely no clue if the benefits will outweigh the costs (more likely it’s the opposite). Now hold on there, Negative Ned, do you know how exciting it was when the “pilot” was running? It was so exciting in fact that we’ve started three more “pilots” and have vendors demonstrating solutions next week that we’ll roll out to a quarter of our sales staff to play with the week after. I’m sure we’ll figure out what problem each solution solves once we’ve seen what it can do. Sigh.

Let’s start with a definition of a pilot: Time spent near the end of a project where the solution is first rolled out to a small group in order to identify and correct defects and gaps. Once the corrections are made, the solution is then rolled out to the full user base. A pilot can help increase the project’s chance of success and users’ adoption of the new solution.

What a pilot isn’t:

  • Buying technology (or bringing it in “on demo”) from a vendor and using its feature list as the requirements for solving a problem that is perceived to exist (but hasn’t been validated)
  • Implementing a new process without first comparing a current-state baseline against the new process to verify that an improvement will be made
  • Enhancing software for a hand-picked group of employees in an attempt to prove if an idea you had will work or not

These sorts of activities are hugely disruptive, they often take people’s focus and efforts off strategy, and they rarely see a satisfactory return on investment because there are no clear objectives and no plan on how to achieve them. I’ve seen hundreds of thousands of dollars spent on these so-called “pilots” because someone has the right level of authority to approve the expenditure. Would you run your personal finances this way? Would you buy a lawnmower and figure out how you’ll use it afterwards, even though you live in a high-rise condo? Of course not. But as soon as someone gets corporate money to play with, all sense goes out the window. It’s not my money and, c’mon, the company spent $5 million last year on projects, what’s $250 thousand here or $100 thousand there? You’re fired.

I’ve previously written on why technology shouldn’t drive business change but it never fails to keep happening. But it’s shiny and new and…! Yeah yeah, pipe down. I also get excited when I hear about something AMAZING and start thinking about all the different applications of the new software or process or portable microwave emitter. But I know better than to spend money without first identifying if a problem or opportunity actually exists, checking the proposed project’s alignment to company strategies, and estimating costs vs benefits. If my analysis points to running the project, great. I’ll seek approval and run it following proper project methodology instead of barging ahead and leaving a wake of departmental churn. Prove to me that your “pilot” doesn’t do that and I’ll buy you lunch.

4 thoughts on “Let’s Run a Pilot

  • Simon

    I think pilots are useful. Sometimes you don’t even know if there is a market without throwing a bit of money at the problem and seeing if a solution emerges. I can think of a number of projects which have started out as pilots gmail, google maps, signalR all of which have turned out to be great successes. If you don’t risk anything then you’re going to be stuck doing what everybody else is doing and eventually a less risk adverse company will eat your lunch. Don’t be a RIM/Blackberry.

  • wastedgenius

    I fully support pilot projects, being that you actually do a bit of planning so you know what you want to achieve. What I have a hard time getting behind is spend without clear objectives because – in my experience – the ROI quickly diminishes.

Leave a Reply

Your email address will not be published. Required fields are marked *