eGov Risk Assessment: Design-Reality Gaps"Is my e-government project likely to fail?" This page offers one technique for answering this question by assessing project risk. Follow this link for other techniques. A gap exists for all e-government projects between the design assumptions/requirements and the reality of the client public agency. The larger this gap between design and reality, the greater the risk that the project will fail. The risk assessment technique presented here asks you to rate the size of a set of design-reality gaps. Follow this link for a detailed explanation of design-reality gaps (and some related case examples). Assessing the Success/Failure FactorsAssessment consists of questions relating to a series of seven 'ITPOSMO' dimensions - information, technology, processes, objectives & values, staffing & skills, management systems and structures, and other resources - with attached rating numbers.
Presenting, Analysing and Using the ResultsOverall Rating AnalysisThe simplest and crudest thing you can do is add up the rating numbers for all seven ITPOSMO dimensions and interpret them according to the following table.
Individual Dimension Analysis and ActionThe scores for each individual dimension can be presented using a table or a diagram arranged to show the gaps in size order from largest to smallest. Both are illustrated in the worked example below. The dimensions with the largest gaps are those that should be prioritised for action if risks of failure need to be addressed. Follow this link for further details about actions to take, plus links to real-world examples based on the design-reality gap model. A slight variation is to use the weighting system described below, with the width of the arrow line related to the weight. Variations on the Basic Technique1. Who does itThese seven rating scales can be used by a single individual, such as a project consultant or project manager, to help them with their own understanding and recommendations. Alternatively, a more participative approach can be used. The seven scales can be presented to a group of key project stakeholders in a facilitated workshop. The stakeholders discuss and rate each dimension. The main problematic design-reality gaps are identified. The workshop would then move on to work out how best to close those gaps. 2. Weighted dimensionsThe basic technique makes a questionable assumption - that all dimensions/gaps are equally important to the success and failure of the project. A more complex variation would involve two rounds. In the first round, the risk assessment team would assign a weight to each of the dimensions. An 'ordinary' dimension might be given a weight of 1; a dimension that was considered 'important' in the particular e-government project could be given a weight of 2; and a dimension that was considered 'very important' in the particular project could be given a weight of 3. The weighting score would be multiplied by the rating to give an overall set of weighted ratings. For example, if 'objectives and values' were felt to be very important for this specific project, that dimension could be given a weight of 3. If the design-reality gap on that dimension to be only moderate, it could be given a rating of 5. The overall weighted rating for that dimension would be 3 x 5 = 15. From experience, the objectives and values dimension should be given a higher weighting than other dimensions because it incorporates key elements such as politics, culture, self-interest, motivation, and the aspirations that a whole variety of different stakeholder groups seek to achieve from the new e-government system. 3. More complex dimensionsThe use of just seven rating scales is very much a 'blunt instrument'. A more sophisticated - also more time-consuming - approach is to break each main dimension down into a series of sub-dimensions. Each sub-dimension is then allocated its own rating scale. For instance:
Such sub-dimensions can be pre-set, and you can link to a Risk Assessment Checklist that provides a detailed list of questions to help understand all aspects of risk on an e-government project. Alternatively, the sub-dimensions can be determined within a facilitated workshop, in which case they can be attuned to particular organisational context. 4. Creating your own dimensionsThis 'attuning' just mentioned can go further: stakeholders can use the seven suggested ITPOSMO dimensions merely as a starting point for discussion, and can then develop their own particular dimensions and sub-dimensions that are seen to be relevant to the specific context. Design-reality gaps can then be assessed for each one of those dimensions/sub-dimensions. 5. Incorporating driversDesign-reality gaps can be thought of as constraints or risks to implementation of an e-government project: they give a sense of what may make the project fail. They may not give a good sense of what may make the project succeed: the drivers. The drivers can be analysed as well, and illustrated alongside the gaps/constraints/risks using a force-field diagram with drivers on one side and constraints on the other. 6. Process as the focusIn the basic and variant approaches described above, two things can be borne in mind:
In some situations, the process may be more valuable than the outcome. It may then be appropriate to take a more iterative, learning approach to gap analysis. Here, stakeholder groups revisit gap analysis at regular intervals during the project cycle. They reflect on the dimensions selected, the ratings and the closure techniques. They also reflect on what has been learned about the project and the e-government implementation process. Pros and Cons of this TechniqueThis technique is relatively simple and quick to understand and put into practice. One key advantage is that it matches the unique situation of each individual e-government project, rather than imposing a "one size fits all" concept. On the downside, it tries to cram a lot of issues into each single dimension (particularly into 'objectives and values' and 'staffing and skills'), and it will not work well if there are competing designs or competing ideas about what counts as 'reality'. Real-World ExamplesA number of real-world cases of e-government project risk assessment and mitigation are provided on this site:
Worked ExampleA new Web-based procurement system is being implemented by the Ministry of Transportation in Gedactia. Introduction of the system is being promoted and partly-funded by an external donor, which has put in place many of the formal skills and technology required, but the project has relatively little internal support. Is this e-government project likely to succeed or fail? An assessment and answer are given below. Questions, Answers & RatingsInformation Question : What is the gap between the information assumptions/requirements of the new e-procurement system design, and the information currently in use in reality in the Ministry? Answer : The project consultants have made use of a fairly 'generic' design for the e-procurement system. In reality, this matches some core elements of information currently in use in Gedactian procurement. However, the Ministry currently makes use of slightly different information to this 'one size fits all' assumption. In reality also, there are shortcomings in availability of information that the design assumes will be present - a list of all government suppliers, accurate pricing information, and a clear set of guidelines on procurement. Thus there is a fair-sized gap between the information assumptions of the design and current realities. Gap rating : 6.5 Technology Question : What is the gap between the technology assumptions/requirements of the new e-procurement system design, and the technology currently in use in reality in the Ministry? Answer : The e-procurement system design assumes the presence of a set of robust Internet connections, Web servers, and procurement software within the Ministry; it also assumes the presence of Internet-connected systems in a broad range of suppliers. In reality, the Ministry currently makes fairly limited use of ICTs, the telecommunications infrastructure in the country is somewhat limited, and many smaller suppliers lack access to ICTs. Gap rating : 7 Processes Question : What is the gap between the work processes required for successful implementation of the new e-procurement system design, and the work processes currently in use in reality in the Ministry? Answer : The e-procurement system design requires a set of formal, rational work processes that deal efficiently with procurement. These proposed work processes under the new system design follow roughly the same lines as the current system, and that current system does function, but with a number of 'hiccups' and inefficiencies in the way that work is carried out. Gap rating : 2.5 Objectives and Values Question : What is the gap between the objectives and values that key stakeholders require for successful implementation of the new e-procurement system design, and their current, real objectives and values? Answer : The e-procurement system design assumes a procurement system that values rational functioning within public agencies, such as freedom of procurement from political interventions. The design assumes objectives of greater efficiency (whatever the impact on jobs), and of the spread of e-government. The reality is somewhat different, though it varies from stakeholder to stakeholder. The donors - who are driving the project - largely share these objectives and values; as do the project consultants and IT suppliers working for the donors. Many senior officials do not share them: they are either happy with the status quo or they have other priority objectives than e-procurement, they support a politicised rather than rational culture within the Ministry, and they are not particularly keen on the spread of ICTs within government. Many clerical staff within the Ministry similarly do not share the design objectives and values: they fear the new system and they cannot see its value. Gap rating : 7.5 Staffing and Skills Question : What is the gap between the staffing numbers and skills levels/types required for successful implementation of the new e-procurement system design, and current, real staffing and skills? Answer : The e-procurement system design assumes the presence of a whole range of competencies for both its implementation and its ongoing operation. For example, it assumes a reasonable-sized team with good experience of designing and implementing e-procurement systems; it assumes good knowledge within that team of Gedactian public sector specificities; it assumes some capacities within the Ministry to manage the implementation contract and the procurement system; it assumes a set of hands-on IT skills among clerical staff in the Ministry. In reality, some of these competencies are present and some are not. The project team has good experience, but knows little about Gedactia; the Ministry has a limited set of management experitise; and clerical staff have a few basic IT skills but not the higher-level skills that operation of the Web-based system will require. Gap rating : 6 Management Systems and Structures Question : What is the gap between the management systems and structures required for successful implementation of the new e-procurement system design, and current, real management systems and structures? Answer : The e-procurement system design assumes some limited changes to management systems compared with current reality, with the introduction of some IT management of the Web systems, and some changes to oversight mechanisms for procurement. The design assumes no significant changes to Ministry structures. Gap rating : 2.5 Other Resources Question : What is the gap between the other resources (money, time, other) required for successful implementation of the new e-procurement system design, and current, real availability of those resources? Answer : The e-procurement system design assumes two sets of financing to be available. First, a larger sum for introduction of the system; second, a smaller ongoing sum for system operation and maintenance. In reality, the donor is making available the money for the first set, and for the second for the first two years. The design also assumes a relatively gentle timescale, using an incremental approach in roll-out of the system. This seems to match fairly well with the amount of time that staff have available (and that political timescales impose). Gap rating : 2.5 Table of Results
Diagram of Gaps |
