In this step we will cover
What is a pilot?
When to use a pilot
How to use a pilot

What is a pilot?

A pilot is a small-scale monitored and managed live test of your solution. The pilot will allow you to identify and resolve all immediate faults or shortcomings before your project is finalised and launched. It is also a crucial opportunity to observe the immediate reactions of real intended users and the public service provider.

Building a pilot into your implementation phase is especially useful if your solution is one or more of the following:

• A physically delivered service or transaction
• A digitally delivered service or transaction
• A physical environment

When to use a pilot?

You should build a pilot into your project plan if:

• The service/interface/environment will be implemented in multiple locations
• The solution requires significant investments in equipment, hardware or software
• The solution is designed for an audience that may be more adversely affected by errors than an average user (e.g. healthcare sector)

How to use a pilot

The length of a pilot can vary from a few weeks to six months, depending on the complexity of the solution and diversity of users.

The pilot should be managed as a project in its own right. When planning, organising and preparing the pilot launch:

• Have a clearly defined plan that includes:
  • Timelines
  • Allocation of resources
  • Success indicators
  • Reviews
  • Measures for implementing amendments/improvements
• Monitor different operation hours to observe as many different user groups as possible
• Develop a simple, fixed reporting format, so that different people can monitor it
• You can incorporate testing and validation of design details that have not yet been finalised, and build this into the reporting format. Ensure that they can be changed during the pilot
• Report against pre-defined criteria as well as unforeseen user behaviour
• Introduce as many refinements/improved solutions as possible half way through the pilot phase – these can be prototypes if necessary
• Unless legally required, do not inform users that they are part of a test as they may affect their behaviour, but be prepared to explain why you are observing, if asked
• Bring all reports and observations back to the design team to discuss how each issue will be dealt with. Remember that changing a single element may have a consequential effect resulting in re-designs of other elements that are currently working effectively. In situations like this, you may have to make compromises and trade-offs.

In this step we will cover:
What do we mean by production?
Considerations before going into production

What do we mean by production?

In Shape ‘production’ means materialisation of the design. This can consist of:

• Manufacturing physical products (e.g. furniture, signs, print materials)
• Production of digital products (e.g. websites, programmes, templates, interfaces)
• Specifying, sampling and purchasing standard products, physical or digital (e.g. furniture, signs, computers/software, pre written algorithms for a booking website)
• Configuration of physical and digital products into a coherently designed solution (e.g. a combination of a physical service offer and a supporting app/website)

Considerations before going into production

A new service, interface or environment must be produced to a quality that accommodates your solution, its anticipated lifespan and its range of users and applications. It must also represent your brand, department or specialism accurately to the users.

If you have carried out a pilot, the findings should be integrated into the full-scale production format. If you have not carried out a pilot, try to ensure that the final product will accommodate small improvements and amendments over a designated start-up period following the launch.

When putting together the elements of a solution there are some factors to consider:

• Users: Most solutions will be judged by frequent and infrequent users and should address the needs of both.
• Accessibility: Aesthetic elements such as materials, forms, colours and typefaces should not compromise legibility, comprehensibility and equal access for all (inc extreme) users. For more information look up “Design for All” (Design for all foundation, 2001) or “Universal Design” (The Center for Universal Design, 2008).

You will have explored all of these elements as part of your brief during the Plan stage, but this is the time to build in new insights you’ve gathered over the rest of the process before a large share of the budget is committed to the final solution.

References

• Design for All Foundation. What is design for all? Last modified 2001. http://designforall.org/en/dfa/dfa.php
• The Center for Universal Design. About UD. Last modified 2008. http://www.ncsu.edu/ncsu/design/cud/about_ud/about_ud.htm

In this step we will look at:
Time
Budgets
Legislation

Introduction

The launch of your solution plays an important role in raising awareness and securing uptake, acceptance and appreciation in the eyes of your stakeholders and users. You might consider the launch a communication challenge and a separate design project in itself!

To make sure your launch is as successful as possible, you need to consider these elements:

Timing:
Make sure your solution is launched at the most appropriate time. Try to avoid:

• Overlapping with public service launches from other authorities
• Public/bank holidays
• Holiday seasons
• Times when the service is likely to come under particular pressure from users

Budget:
Do not underestimate the budget and time required for a proper launch. This should be planned from the beginning and can include:

• Media Adverts
• Posters
• Mailings
• Live events to publicise and support
• A PR and media (social and print) campaign

Legislation:
You may have legal requirements to inform citizens, companies or the public domain of changes that will emerge as a result of a new service. All mandatory activities related to the launch of a new solution should be mapped into the project plan as early in the project as possible.

Staff awareness and training:
When launching a live service based on a digital solution, you will need staff members to assist citizens. These interactions can also feed into the overall evaluation of your new service, which makes them a valuable resource for the Monitoring step.

Outreach/communications:
Investment in design is highly likely to result in improvements. Since this is good news, it’s an excellent opportunity to reach out to your audience of users and other relevant stakeholders.

Develop a communication strategy for your launch. This can include:

• PR/Media (inc social media) adverts – on-street/transport/magazines/in-buildings/online
• Radio/TV – talk shows, advertising, interviews
• Flyers
• Posters
• Events – this can be the most efficient budget allocation, as events are often the best way to actively engage and build a relationship with users and other stakeholders

In this step we will cover:
Methods
Objectives and value

Methods

During and immediately after the launch of a new solution it is important to allow for structured and regulated monitoring, and documentation of user experiences/feedback.

A user feedback template in the form of a questionnaire or interview is a good way of carrying out this process neutrally and consistently, regardless of the number of observation posts, and who or how many users respond.

Your template can include:

• Time: How long did users take to complete the intended action?
• Flow: Was there a steady flow of activity?
• User interface: Did users understand and use the facility instinctively?
• Use: what was the rate of completed operations that did not require assistance?
• Assistance: did users have access to assistance?
• Second time use: did the assistance enable users to complete the operation independently next time?

Objectives and value

You can add value to your monitoring by using exit interviews, asking users about:

• Their experience compared to earlier systems
• Whether they experienced any problems
• Whether they found the solution attractive

During monitoring and feedback you are also gathering data to feed into your audit during the Report stage.

Continuously monitoring your solution after the audit will help you to ensure its effectiveness and identify when it’s ready for an update. You can find out whether:

• The product is being used enough to justify its maintenance costs
• The product or service is becoming obsolete
• The product is still functional:
  • Are there better technologies that would fulfil the same task for less money/time/maintenance?
  • Has the context or operating conditions changed? (E.g. Updating a website for current operating systems/platforms)
• The product/service is due for renewal

This is your first measurement of the solution since measuring the start value in the Objectives step in the Identify stage. Based on that data you can now communicate initial results to key stakeholders.