18 Criteria for building successful e-Services

You are currently viewing 18 Criteria for building successful e-Services

UK scores high in terms of building citizen-centric services and delivering personalized digital experience. All its interfaces through which services are delivered are equally amazing in meeting satisfactory standard. The transformational improvement in user experience has been the result of adhering to digital service standard code that  helps government departments create and run good digital service. All services of a department having a digital interface are required to meet a set of 18 criteria to ensure that services are good enough for public consumption.


Each service before going live goes through discovery, alpha phase and beta phase. In the alpha phase, a service prototype is built, tested with users and demonstrated that building service is technically feasible. In the beta phase,  a working version of the service based on alpha prototype is built. The application is tested, audited and accredited. Service gets a SSL certificate, GOV.UK domain and a set of KPIs to monitor service performance is developed. The beta application may be launched to limited user base (private beta) or to one section of user at a time (public beta). Post beta stage the application goes live, and it is the time to keep improving service based on user feedback, analytics and ongoing research. In this phase, service information is secured, service performance is measured, service adheres to government service standards and accessibility requirements. The service improvements are iterated till service retires.


“The service is assessed on 18 criteria in each of the phases (Alpha, Beta and Live) for the digital experience and necessary performance and governance standards.”

The 18 assessment criteria are described as under:

  1. Understand user needs: Department must ensure that sufficient study has been done in understanding users and user needs. The services are assessed on following criteria
    1. Alpha assessment: Find the users, their needs, their stories and pain areas
    2. Beta assessment: Recruitment of users for testing service, determining the pain areas and the challenges, how pain areas have been addressed
    3. Live assessment: Changes made to make challenging task(s) easier & result from user testing, Continuous improvement plan and service evolution 
  2. Do ongoing user research: Ensure that service put in place is doing what it is expected to do. The assessments covers following:
    1. Alpha assessment: The usability tests through out the design of service and after it is built, user research plan and the frequency of usability tests.
    2. Beta and live assessment: Engaging users for usability tests, user research for people with accessibility needs, testing with users who need assisted digital support, using analytics data in user research plan and solving problem that weren’t handled in previous stages.
  3. Have a multidisciplinary team: A multi-disciplinary team helps in arriving at decision more quickly and addresses user need in a better way. In the alpha, beta and live assessments, department (and solution integrator) would be able to:
    • Describe the delivery team: Service owner, product manager, delivery manager, technical architect, assisted digital lead, designer, user researcher, developer, content designer, web operations engineer, performance analyst, front-end developer
    • Ensure that there is no overlap of roles, and each role has capacity and knowledge to make decision to improve services.
    • Plan to transfer knowledge and skills from contractors to permanent staff.
    • Have a team with full understanding of services after it has gone live.
  4. Use agile methods: Build the service using the agile, iterative and user-centered methods to meet the user needs (such as convenient for people, easy to change, improve service based on user feed back) and cost less. To meet the alpha, beta and live assessments, departments need to:
    • Explain how it is working in agile way, agile tools in use during and after service goes live
    • Show agile governance, clear and measurable goals and change/ risk management
  5. Iterate and improve frequently: The idea is to build a service that can be revisited frequently for improvements, and ensuring the capacity and technical flexibility to do so. To meet the alpha, beta and live assessments, following must be ensured:
    • Understanding of user story from user research to production
    • Ability to move user stories swiftly from user research to production in zero down time or minimal disruption to users
    • Staff deployment to analyse user research and improve services.
  6. Evaluate tools and systems: To ensure that the tools and systems used to build service don’t put department to a constraint or risk, and allow it to easily manage service after service goes live. Following stages are required to be assessed to pass this criteria:
    1. Alpha: The choice of language, framework, programming tools and standards. Set up monitoring framework (identify user and technical metrics for internal, external and capacity monitoring), and distribute monitoring data and manuals widely.
    2. Beta: Managing the limits on service, support arrangements in place in normal and out of hours and decisions outsourced to third parties.
    3. Live: Decision on tech stack and how one is getting value for money, health of service
  7. Understand security and privacy issues: Evaluate data & information that shall be shared, stored or provided in delivering service and address the security, privacy risks. Following is assessed to meet the criteria:
    1. Alpha: Threat identification, control and plan to keep up-to-date and deal with known/ unknown threats or frauds.
    2. Beta: Team’s approach to risk and security management, functioning of security designates such as Asset Owners, Data guardians, information risk owners. Policies in place such as cookie policy, privacy policy et al.
    3. Live: Measures to ensure coordination of team involved in managing risks, maintenance of up-to-date policy and understanding of threats/ frauds
  8. Make all new source code open: The advantage of making source code open are managing contract risks and ensuring re-usability. Unless convincing reason exists, source code must be published under appropriate license. The assessment criteria are:
    1. Alpha: Explain how one plans to release source, and make it reusable. Confirm the intellectual property.
    2. Beta: Show code in open internet source code repository, internal and external reuse of code in building service, and appropriate license.
    3. Live: Describe how to accept contributions and comments on source code, Handling of updates and bug fixes. Explain the parts of code made not open and why
  9. Use open standards and common platforms: Using open standard and common government platform save time and money through reuse, provide consistency in terms of experience in using government services, quick and easy to change, and ensure that services are technology neutral. The assessment criteria are:
    1. Alpha: Identify common user needs that services meet and possible reuse within or across government departments. Use open standard and common platform that can be employed in your system.
    2. Beta: Manage common data, service outputs to users, and integrate with legacy systems.
    3. Live: Meet open data responsibility, integrate with external system
  10. Test the end-to-end service: Test the end-to-end service in an environment identical to that of the live version on all common browsers and devices, and using dummy accounts and a representative sample of users. To pass alpha, beta and live assessment show that you have
    • – an effective deployment environment that can be created quickly and easily
    • – Presence of data in pre-production environment
    • – Designing and testing service with multiple devices and browsers that users use
    • – Test with users who need digital assistance.
    • – Test for non-digital parts of service if any.
  11. Make a plan for being offline: While users expect 365 day, 24X7 availability, it is imperative to have a plan if service goes offline. Following must be addressed to pass this criteria:
    1. Alpha: Impact assessment of how users are impacted if the service is unavailable for a length of time
    2. Beta and Live: Data recovery strategy and its testing, Selection of technology and platforms to meet availability requirements, business continuity plan, and holding accountability and decision making at the time of outage.
  12. Make sure users succeed first time: Create simple and intuitive services that users do not have to struggle to use it. The assessment criteria for passing alpha, beta and live assessment are given below:
    • Explain your service and what it does. Evidence of sufficient usability testing, funded assisted digital support free for users, and analytics
    • Show that majority users are able to use service in first instance only
    • User research and extent of build, measure and learn cycle for hypothesis testing
  13. Make the user experience consistent with GOV.UK: Building a consistent GOV.UK user experience across services using common design pattern and style guide helps in building trust, easy recognition, and save time and focus. The pass criteria for alpha, beta and live assessment are:
    • Team includes designers, content designers and front end developers
    • How service is using styles, components and patterns in Gov.UK design system (headers, footer, start and end page, et al.)
    • The service is responsive and works on mobile devices
  14. Encourage everyone to use the digital service: Plan for shifting to digital channels and phasing out non-digital ones to save money. To pass the criteria, following is required to be shown:
    1. Alpha: Plan for increasing digital take-up, data collection for channels where services are delivered, user research (demographics, user attitude, channel preferences et al.)
    2. Beta: Evidence of increase in digital make-up through weekly analytics and metrics, Communication improvements to shift users to digital means
    3. Live: The 5 year plan to shift users to digital means and phase out non-digital means.
  15. Collect performance data: Data collection and analysis helps to improve service and plan for next step/ evolution of service. To pass alpha, beta and live assessment, department needs to:
    • Explain why and how of performance data to capture, tools to analyze data, and addressing security risk of performance data
    • Road map of performance analysis and governance structure for following up on actionable insights from user journey and measuring data
  16. Identify performance indicators: Identify Key performance indicators (KPI) to measure the service, benchmark and improve service. To pass alpha, beta and live assessment, department should show:
    • Start with performance baseline of old system if it exist, and show improvements over it.
    • Common KPIs are:
      • Cost per transaction
      • Improvement in user satisfaction
      • Increase in completion rate
      • Increase in digital take-up
      • Level of assisted digital
      • Channel Shift
  17. Report performance data on the Performance Platform: Be open and transparent to the public about service’s performance. At different phase of service, it is assessed on following parameters:
    1. Alpha: Service is registered with performance platforms, and can support the metrics that you want to present on dashboard
    2. Beta: Metrics is populated on performance dashboard automatically
    3. Live: Show all KPIs on dashboard along with their appropriate interpretation 
  18. Test with the minister: Ministers are accountable for everything produced by department, so testing with minister brings his confidence on service. The criteria is assessed in each of the phases:
    1. Alpha: Confirm that minister tests service before it goes live
    2. Beta: How minister will test the service
    3. Live: Show evidence (Videos, Photos, Signed Letter) that Minister has tested the full service from beginning to end.

Further Readings

Leave a Reply