FESTA handbook Socio-Economic Impact

Back to FESTA Handbook main page: FESTA handbook

10. Socio-Economic Impact

10.1 Introduction

All FOTs[FW] require an impact assessment analysis. Many times, especially if the FOT is supported by funding from the European Commission, the requirement is that the assessment should be performed at European level, also with details on the socio-economic evaluation

---

Naturalistic Driving Studies tend to focus on crash/explanatory factors, and Field Operational Tests generally focus on evaluation of systems or functions. Therefore, this chapter is of less relevance for NDS, depending on the hypothesis and research questions and whether impacts are involved. Part of this chapter may be relevant for NDS, for example if an overview of the findings has to be presented, or if findings are used to identify measures or in development of functions and services.

---

In the past, the impact assessment of FOTs focussed on a narrow set of impacts of interest. Few looked at the stakeholder or supplier perspectives; some measured benefits but not (social) costs; very few started out with an impact table and formally identified what the expected “main effects” of the systemss investigated would be; and some did not carry out a socio-economic impact assessment. The goal of this chapter is to provide concise advice on how to carry out a socio-economic impact assessment. Many parts of the chapter will also be relevant to FOTs conducted at a national or regional level within Europe or to FOTs conducted outside Europe. A consistent methodology for carrying out this analysis for EC-funded FOTs will maximise the comparability of the results across regions.

Another goal is to address the possible breadth of impacts that can be considered and the available resources for carrying out the assessment, like references to examples of good practice in existing (web) documents.

Although impact assessment comes at the end of the FOT chain, it should also play a role at the beginning of a project, indeed some of the choices made in setting up the FOTs are linked to the Cost Benefits Analysis (CBA). For example, choices about performance indicators and scaling up are directly linked to what can be analysed in terms of scope and impacts on the CBA. Furthermore, the expected impact is a major driver for the decisions to be made in the design of the FOT. People who are responsible for the impact assessment should be involved from the beginning of the project.

It should also be considered that policy makers expect concrete results from an FOT, such as the gain in term of reduction of accidents, and scaled up costs and benefits (on a European level). This is often promised in project proposals. However, these are results that are difficult to get and there could be various reasons why the FOT does not deliver these results in the end (or at least not with the completeness that policy makers desire). Managing expectations is therefore important, throughout the project, explaining any apparent failures early, whilst emphasising lessons learned or any secondary benefits found.

Our advice will be useful for a variety of parties: the organisations conducting the FOTs, including also the impact assessment specialist; the client commissioning the FOTs; and the consortia drawing up proposals for the FOTs. This chapter assumes that a “professional” in the area of impact assessment and of socio economic cost benefit analysis will carry out the analysis. This individual we will refer to as the “analyst”. This information on assessment is not meant as a “tutorial”.

Broadly, there are three research angles where the evidence comes from:

• Impact assessment studies typically investigate the impacts of a systems for a future time horizon. These prospective studies make use of an ex-ante impact assessment, often based on literature review, simulation work and expert estimation. They are often comprehensive in scope but they do not involve, or only to a limited extent, data from real-life conditions.

• Transport appraisal guidelines or scoping studies in this area are very much focused on the appraisal part of the impacts. They dig deep into the methodology and practise of appraisal. They also involve proposals for standardisation of appraisal. Their detriment is that they are not developed for specific use in the field of safety evaluation.

• Field Operational Test assessment studies typically assess the impacts of one or more systems functions. FOT evidence can lead to a quantum leap in the impact assessment because FOT produce measured data about effects. Therefore, the assessment can rely on ex-post measurement data.

All different research angles have their specific strengths and weaknesses. In preparing this guidance document, we found it useful to combine the strengths of the different perspectives.

10.2 Considerations

The impact assessment investigates the impacts of a technology on society. Ideally an impact assessment provides the decision maker with relevant information in a concise format. The relevant comparison is between the benefits and costs between a base case, e.g, a scenario without the ICT systems (“without case”) compared to those of the scenario with the ICT systems (“with case”). In preparing to carry out an impact assessment, the analyst is faced with making choices about the deployment scenarios, the geographical scope of the assessment and the analyses to be carried out. This section will go deeper into the issues surrounding these choices. The chapter will conclude with guidance on how to make the choices and carry out impact assessment and the socio economic cost benefit analysis.

10.2.1 Deployment scenarios

Predictions of the future, particularly over the medium or long term, cannot be precise. This argues for a scenario-based approach when developing forecasts of how future deployment of a systems might turn out. This approach permits alternative scenarios to be evaluated in the cost-benefit analysis. Very likely one scenario will emerge as more favourable with the highest benefit to cost ratio, although that scenario may not be the most probable. The scenario analysis will also enable obstacles, to the pursuit of that scenario, to be identified. This in turn can identify public policy needs and other stakeholder requirements.

For the socio-economic impact analysis deployment scenarios are required, whether they are just implicit or explicit. One approach is not to make any specific predictions but simply examine the socio-economic impacts of different levels of penetration. But even for such a simple set of scenarios, there has to be an assumed growth in penetration over a period of years. But it is more appropriate to build up some alternative scenarios with different “futures”. There are a number of potential inputs into creating those scenarios:

1. Policies of public authorities at a European, national , regional and local level. Policies and strategies for transport in general, road safety, environment, accessibility, traffic management and general ICT deployment (e.g. future mobile network capacity) are all relevant.

2. Other quasi-regulation such as EuroNCAP and standards

3. Stakeholder plans — the strategies of OEMs, telecoms operators, road operators, large fleets and so on

4. Likely developments in the various relevant markets, including costs, technical issues (such as synergies between different systemss) and competitive pressures

5. Public-private partnerships and their influence on the deployment process

6. The attitudes and willingness to pay of the general public to identify potential purchasing and usage decisions.

Deployment can be pushed or even mandated by the public authorities, which can be termed “regulatory deployment”. It can be more voluntary, with a push from major stakeholders, e.g. by formal Voluntary Agreement. Or it can be purely market-driven, i.e. totally voluntary, depending entirely on the public’s willingness to purchase systemss and use functions.

There are a number of tools for scenario development. At the most basic level there are forecasts on the growth of the vehicle market, changes in mobility and changes in the road network. Government strategies provide information about policy goals and targets. Stakeholder questionnaires and analysis provide further details on willingness to promote and invest. For the views of end-users, feedback from FOT participants on acceptance and willingness-to-pay is an important source of information and should be routinely collected in an FOT. More general information on public attitudes can be collected by means of focus groups, household surveys and stated preference studies. The last are useful as a tool to reveal trade-offs between alternative choices.

10.2.2 General issues of the socio-economic impact analysis

As at the hypotheses formulation stage, consideration needs to be given to the potential bundles of systemss to be handled in the impact assessment. Indeed, there can be a large number of permutations of market penetration of different bundle sizes and not all can be covered, and some combinations of functions may be more likely than others. Some expert judgement has to be applied here, make reference back to market surveys, etc. More information about disentangling the effects of combinations of functions can be found in (Faber et al., 2011).

An impact assessment can only be as good as the data on which it is based. Hence those carrying it out should also be involved in the performance indicator definition stage in order to ensure that the relevant indicators are being collected. These may be the obvious indicators, such as speed, route choice etc., but there may be occasions where the FOT cannot provide the desired data directly and other methods (e.g. surveys, workshops etc.) may have to be applied.

It is difficult to provide a definitive guide for conduction of an impact analysis. The process depends on the research questions and the functions. For example in terms of scaling up, the area of interest may be a particular city, country or group of countries. For a function that operates e.g. on highways it makes sense to scale up according to those networks only. There is the added problem that scaling up data from one country to another may be inappropriate for a wide variety of reasons (driver types, weather, cultural aspects etc.). More empirical data and information is needed to increase certainty on effects and conclusions.

A major step of the impact assessment is scaling up by using simulation tools. Simulation can determine indirect effects (i.e. reduction of congestion due to less accidents etc.). Simulation tools generally require modelling of driver behaviour. This modelling relies not only on specific indicators which are being collected in the FOT from the equipped vehicles. Moreover the modelling relies also on information about the interactions of those vehicles with non-equipped vehicles and other road users. The modelling can be performed in micro-simulations, which are able to provide input to the impact analysis. These interactions between non equipped and equipped vehicles are rarely addressed in the hypotheses or in measurement processes. This can only really be addressed by undertaking observational studies (video data analysis could be a possibility).

Current micro-simulation methods have their own limitations. Typically the most generally used software packages do not properly represent vehicle dynamics in terms of interaction with the road surface. They therefore are not properly capable of covering lateral dynamics of the vehicle. The networks covered are often geographically small and have often been created for purposes other than the evaluation of new vehicle-related technologies. They may not be very representative of overall national road networks and origin-destination matrices (i.e. traffic flows) are generally lacking for night-time and weekend periods.

Business analysis in a private environment serves the same purpose as socio-economic impact assessment in the public world. It provides crucial information for use in the decision-making process on further steps towards deployment. Where socio-economic analysis tries to rule out double counts on the cost side, the business analysis focuses more on the feasibility of the concept and the roles and consequence for each of the actors required in the value web. The process of balancing the transfer of value (hardware, software, money, data, permits etc.) needs to be investigated and usually there are a number of options for the business model (private, public-private and public only), each with a specific constellation of the roles of the actors involved and each with a specific risk pattern.

10.2.3 Assessment scope and process implication

At the start of the socio-economic assessment, a view will need to be taken on the scope of the analysis. Ideally the assessment would include all impacts of the systems no matter how small that impact is: safety, mobility, efficiency and productivity, environmental, user acceptance and human factors, performance and capability, legal and implementation issues, and costs. However setting an unlimited such a broad scope for a socio-economic assessment will result in excessive data collection and analysis in terms of expense and time. Given that the purpose of the assessment is to firstly ensure that the implementation of the systems is economically beneficial and secondly to aid the choice between alternatives, the scope of the assessment often can be narrowed by excluding minor or insignificant impacts as long as the exclusion of these impacts will not bias the appraisal. An impact table such as in Batelle Memorial Institute (2003, p. 45) is extremely useful at the start to clarify which impacts have been considered and which — if any — have been ruled out as negligible or impossible to assess.

10.2.4 Geographical scope of assessment

The issue related to geographical scope is the ability to translate the findings of the FOT to a “higher” geographical level. The FOT is usually carried out at one or more locations, on a regional or national scale. However, the number of equipped vehicles and, if relevant, equipped roads, as well as the number of “equipped” kilometres driven, is usually a small percentage of the total vehicle fleet and the kilometres of roads. Therefore, in order to draw conclusions about the impacts and effectiveness of the systems tested, a “scaling up” of the results is needed in order to draw conclusions and in order to ensure transferability of the results. Section 9.5 addresses the scaling up issues, which is to the national or European level. The availability of data plays a role in the decision to what level to scale up the results. Section 10.4 goes into more detail to explain how to deal with this issue.

10.3 Analysis of impacts

Exhaust emission from road traffic is a complex process to describe. Models for exhaust emissions in general include three parts: Cold start emissions, hot engine emissions and evaporative emissions. An exhaust emission model can roughly be described as:

Σ(Traffic activity) x (Emission factor)=Total emissions

Of course traffic activity data then has a high correlation to total emissions. Traffic activity data includes: mileage, engine starts and parking. In addition to traffic activity data one needs data for: the vehicle fleet; road network; meteorological conditions; fuel quality etc. If the driving pattern is influenced by the traffic situation, such data for the FOT vehicles are directly available. In order to estimate driving pattern changes for all vehicles by traffic situation, microsimulation models could be used. In order to estimate emission factors for these alternative driving patterns there is need for exhaust emission measurements or exhaust emission models on an individual level. The recorded speed traces from the FOT vehicles can also be post-processed through a fuel consumption and emissions model to produce data on environmental effects. Speed has a close relation to safety. The speed of a vehicle will influence not only the likelihood of a crash occurring, but will also be a critical factor in determining the severity of a crash outcome. This double risk factor is unique for speed. The relationship between speed and safety can be estimated by various models such as the Power Model (Nilsson, 2004; Elvik et al, 2004), that estimates the effects of changes in mean speed on traffic crashes and the severity of those crashes. The Power Model suggests that a 5 % increase in mean speed leads to approximately a 10 % increase in crashes involving injury and a 20 % increase in those involving fatalities. More examples of models for speed-safety relationships are reviewed in Aarts and van Schagen (2006). In general it is important to consider under which assumptions the models are valid. The Power Model, for example, is valid under the assumption that mean speed is the only factor that has changed in the system. Therefore these models are more suitable for FOT with systems systems mainly dealing with speed, and even then they fail to consider changes in the distribution of speed (shape of the speed distribution and changes in speed variance).

The analysis of impacts (impact assessment) represents the most sophisticated part of the assessment. It provides an overview over the most common effects (safety, mobility, environment, costs) which are considered in an FOT assessment. This assessment framework involves the distinction between direct and indirect effects (in safety mechanisms but also with respect to mobility effects, see below). It also implies the distinction between effects on internal and external costs. Mobility effects typically lead to lower internal costs of transport (i.e. time, fuel consumption) and also external costs (e.g. pollution, CO2). The reduction of external costs is flagged out separately under environmental benefits because of its importance on the political agenda. The assessment can of course also consider wider economic effects (e.g. growth and employment effects of new technologies). However, given limited time and budget, it is useful to concentrate on the main impacts.

10.3.1 Safety benefits

The most direct and easiest way to calculate safety benefits would be to compare the number of accidents (and their consequences) happening during the baseline and treatment phase in an FOT. However, usually not enough accidents happen in an FOT to make this approach feasible. Therefore other methods have to be used. Traffic safety is regarded as a multiplication of three factors, namely exposure, accident risk and injury risk (Nilsson, 2004). The assessment of safety impacts has to consider these three effects which can be combined to predict the overall safety benefit, while taking driving conditions into consideration as well. Strategic decisions are highly relevant for exposure, and driving behaviour (on tactical and operational level) is relevant for accident and injury risk. A change in exposure can be measured in the FOT directly: do people drive more or less with the system, do they drive on other road types, do they choose other routes? A change in mileage has a direct effect on exposure so on the number of accidents. Translating a change in driving behaviour into accident and injury risk is less straightforward. There are gaps in knowledge: the relation between changes in driving behaviour and (number of) accidents is often not known. Therefore there is not one method that is recommended to use. In this section a number of approaches to calculate the safety benefits of ITS applications are mentioned, with a (brief) explanation, references and information about in what situations they can be used:

• Speed has a close relation to safety. The speed of a vehicle will influence not only the likelihood of a crash occurring, but will also be a critical factor in determining the severity of a crash outcome. This double risk factor is unique for speed. The relationship between speed and safety can be estimated by various models such as the Power Model (Nilsson, 2004; Elvik et al, 2004), that estimates the effects of changes in mean speed on traffic crashes and the severity of those crashes. The Power Model suggests that a 5% increase in mean speed leads to approximately a 10% increase in crashes involving injuries and a 20% increase in those involving fatalities. More examples of models for speed-safety relationships are reviewed in Aarts and van Schagen (2006). In the ISA UK project this is elaborated, different relations (models) for different road types can for example be found in Table 13 in http://webarchive.nationalarchives.gov.uk/20110304132839/http:/cfit.independent.gov.uk/pubs/2008/isa/index.htm . The Power Model is valid under the assumption that mean speed is the only factor that has changed. Therefore these models are more suitable for FOTs[FW] with systems[FW] mainly dealing with speed, and even then they fail to consider changes in the distribution of speed (shape of the speed distribution and changes in speed variance). The model is not suitable for systems that for example influence lateral behaviour.

• Event based analysis Crashes are very rare events, thus there is a strong interest and need for the use of crash surrogates or “crash-substitute” events. The basic idea is that less severe events can be used instead of crashes to estimate safety benefits because there is a systematic and well-understood relationship with crashes. Event based analysis (EBA) uses events to estimate safety benefits. As reported in 5.3.3 the basic principle of EBA is to identify short driving segments (typically in the order of 5-10 seconds), during which the risk of crashing is judged to be higher compared to other driving in the data set, and then to analyse these events further. These events are often referred to as Crash Relevant Events (CRE), since their occurrence is thought to be indicative of actual crash risk in one way or another. EBA can be used for functions that warn for a certain event (e.g. FCW), not for functions that work continuously.

• eIMPACT method The eIMPACT method was used in the eIMPACT project for the safety assessment of in-vehicle safety systems (IVSS). The complete methodology can be found in (Wilmink et al., 2008). In short it works as follows. Effects of IVSS on traffic safety may appear in many, both intended and unintended ways. It is not possible to define in advance the group of accidents affected by the system, although system developers typically have as a starting point a target group of accidents for a system. Therefore, it is highly important that the analysis of IVSS covers all possible effects in a systematic manner. The approach was based on the system nature of transport. When one element of the system is affected, the consequences may appear in several elements and levels of the system, both immediately and in the long term, due to behavioural modification. Road safety is regarded as a multiplication of three orthogonal factors: (1) exposure, (2) risk of a collision to take place during a trip and (3) risk of a collision to result in injuries or death. In the analyses, the three main factors of traffic safety were covered by nine behavioural mechanisms as first described in (Draskóczy et al., 1998). Five mechanisms are mainly connected to the accident risk, three mechanisms deal with exposure, and there is one mechanism that deals with changes in accident consequences. Every mechanism may result in either positive or negative impacts on road safety. In summary, the analyses aims to cover not only the direct intended effects of systems but also the indirect and unintended effects, including behavioural adaptation in long term use. In addition, it was taken into consideration that the effects will vary according to road conditions and circumstances. This should ensure that all effects on safety are covered by the analyses. The starting point for the safety impact assessment were the system specifications, including detailed safety function definitions. Figure 10.2 presents an overview of the phases in the analysis. An important part of the analysis is the use of accident data. For further details the reader is referred to (Wilmink et al, 2008).

• Risk matrix approach The risk matrix approach (RMA) was developed in the euroFOT project. Details about the method can be found in (Van Noort et al., 2012). The RMA is developed for systems that function continuously (e.g. ACC) and that address an isolated accident type (e.g. rear-end collisions). The RMA associates a risk to each data point, by assuming a hypothetical accident scenario, developing from this data point. Separately there is a risk calculation from FOT data. Risks are pre-calculated once, and the application of FOT data is quite simple. This method is a variant of a method developed by NHTSA (W. Najm). The RMA does not rely on video data and is usable without in-depth accident statistics.

• This ‘method’ can be used when quantitative methods (as described above) do not work for some reason, or in addition to it. Expert judgment usually produces qualitative results. Expert judgment can be done in different ways, for example by organising a workshop, or by having experts fill in a questionnaire. Expert judgment should be based on the data that are available from the FOT (surrogate safety measures e.g. speed, speed variance, headways) but also from previous studies.

The methods mentioned above are not perfect nor applicable situations (for all functions, all type of FOTs). In the end one wants to know the relative change (e.g. how much percent of the accidents can be prevented because of a certain system, compared to driving without this system).

Other ways to gain insight in changes in safety are for example looking at the frequency of certain events (e.g. hard braking) and speed violations. More on the events can be found in Annex C. From a policy point of view events are better usable than TTC, increase of mean speed, etc.

In the end, independent of the method one chooses, it is important to clearly write down the assumptions that were being made and the consequences of these assumptions. When there is a lot of insecurity in the safety impact assessment, an option is to work with a bandwidth and not deliver one fixed number as a result, but a range. A sensitivity analysis can also help with this.

As an example and representing best practice, the FOT puts the goals of the safety analysis as follows:

1. Determine if driving conflict and crash probabilities will be reduced for drivers using the systems,

2. Determine if drivers drive more safely using the systems,

3. Determine reduction in crashes, injuries, fatalities if all fleets operating in the observed area were equipped with the systems,

4. Determine if drivers using the systems have less severe crashes than drivers without the systems.

The first step collects sensor data from each vehicle within the FOT (e.g. brake force, steering angle). Based on earlier definitions the number of driving conflicts can be determined. Thus, two numbers for the driving conflicts – reflecting the with and the without case – are available to calculate the exposure ratio. This ratio reflects the number of driving conflicts in the with case compared to the without case. To provide an example: given a systems which maintains the safe distance to a predecessor vehicle, the number of driving conflicts due to close following will be reduced from 10 conflicts per 1000 km to 5 conflicts per 1000 km. Thus, the exposure ratio equals 0.5 which indicates that driving with the systems is safer than without the systems. In general, an exposure ratio below 1 indicates a safety benefit.

The benefit of lower exposure to accident risk will likely be modified based on adaptations of individual behaviour due to psychological reasons (second step). Behavioural adaptations can comprise e.g. adapting the following distance, adapting the speed variance, adapting the lane change behaviour (risky cut-ins or changing the lane without signalling it in advance). Examples for such behavioural changes can be found in the OEM safety mechanisms (eIMPACT). In this project, nine mechanisms have been introduced which lead to positive or negative safety effects. In most cases, the motivation for behavioural adaptation is that the driver wants to avoid “public” warnings (noticeable to all passengers) and “education” by the systems.

The third step deals with scaling-up from the FOT to a wider area (EU, country, region). This process is subject to the procedure proposed in scaling up.

The last step leads to the prevention ratio. In-depth information on accidents is used to calculate the mitigation effects of using the systems. Maybe the systems cannot avoid the accident but it can mitigate the accident consequences. This issue has to be considered in determining the effects for casualties. For systemss affecting speed, the Power Model can be applied to calculate changes in severity.

Combining steps 2 to 4, it is possible to calculate the prevention ratio. For this ratio the probability of having a crash (casualty) when having a driving conflict in the with case is compared to the same probability in the without case. In the above example the number of driving conflicts in the with case was 5 and 10 in the without case. Let us assume that out of the 5 driving conflicts 1 accident occurs and out of the 10 driving conflicts 3 accidents occur. Thus, the probability of having an accident due to a driving conflict is 0.2 in the with case and 0.3 in the without case. These values reflect the prevention ratios.

10.3.2 Efficiency benefits

Efficiency benefits are typically composed of two effects. They involve:

• Direct mobility effects resulting from a smoother traffic flow, e.g. where the systems allows traffic to re-route to avoid current congestion, or improves mean speeds by encouraging safe following behaviour,

• Indirect mobility effects resulting from reduced crashes e.g. reduced delays at incidents and accidents.

Direct mobility effects can play an important role in the socio-economic impact assessment. On the appraisal level, direct mobility effects are reflected in changes of time costs, fuel consumption costs and reliability changes. Because socio-economic impact assessment identifies quite commonly reductions of time costs as a major driver of the results, direct mobility effects are generally worthwhile to explore.

The investigation of direct mobility effects typically involves microscopic traffic flow simulation. A number of models (e.g. modeller.pdf ITS Modeller, VISSIM, Paramics, DRACULA) have been applied to assess these impacts. Best practise, including on cross-validation of models, can be found in eIMPACT D4 (Wilmink et al., 2008) and Full Traffic (Technische Universiteit Delft, 2008). Typically, when traffic flow becomes more homogeneous, the standard deviation of the vehicle speed becomes lower. As a result, the average vehicle speed may increase or the infrastructure capacity improves. As a consequence, time costs and vehicle operating costs will decrease.

However, the realisation of those benefits is closely related to the likely market penetration. Mature ICT systemss typically can produce such effects, ICT systemss in the phase of market introduction typically can not. For internal efficiency it is therefore important to figure out at the beginning of the FOT assessment (when the scope is defined) whether direct mobility effects will be likely to appear or not.

Compared to the direct mobility effects, experience suggests indirect mobility effects are not restricted by conditions of market penetration. They can be realised in any case, as an add-on to the safety benefits. Indirect effects occur when the number — as well as the severity — of crashes is reduced. The benefits result from less congestion, therefore reducing journey times and fuel consumption. Typically, indirect traffic effects add up to about 10 % of the safety benefits.

Given the state of the art in traffic modelling, indirect mobility effects are assessed more frequently than direct mobility effects. Good practise on the appraisal of indirect mobility effects can be found, however, in recent European scale assessment studies (eIMPACT; COWI, 2006) and US American FOT assessments (Batelle Memorial Institute, 2003; Volvo Trucks North America Inc, 2007). Some countries have methods specifically to address these effects (e.g. INCA in the UK).

10.3.3 Environmental benefits

Environmental benefits comprise lower CO2 and air pollutants emissions. Noise also fits into this category but we would caution that noise should only be analysed where ICT systemss are expected to make a significant difference between the two scenarios (with/without the systems). CO2 and pollutants emissions are both speed dependent, with CO2 emissions directly linked to fuel consumption.

Exhaust emission from road traffic is a complex process to describe. In 5.5.3 a simplified formula is reported for calculation:

Σ(Traffic activity) x (Emission factor)= Total emissions where Traffic activity data includes: mileage, engine starts and parking. In addition to Traffic activity data one needs data for: the vehicle fleet; road network; meteorological conditions; fuel quality, etc. If the driving pattern is influenced by the traffic situation, such data for the FOT vehicles are directly available. In order to estimate driving pattern changes for all vehicles by traffic situation, microsimulation models could be used. In order to estimate emission factors for these alternative driving patterns there is need for exhaust emission measurements or exhaust emission models on an individual level. The recorded speed traces from the FOT vehicles can also be post-processed through a fuel consumption and emissions model to produce data on environmental effects.

The impact of CO2 emissions is on a global scale, and is not linked to the particular country or area type where the CO2 is emitted. The impact does, however, vary according the year in which the reduction (or increase) in emissions takes place — the impact becoming greater further into the future. Actually, mobility effects have impacts on both efficiency and environmental benefits. However, because they are transmitted through the environment, and because they are largely externalities (i.e. their incidence is mostly on individuals other than the emitter) environmental benefits fall into a special category.

10.3.4 Integration of results

FOTs should make sure that there is enough time for the integration of results. At the end of the analysis phase, there are results for traffic efficiency, behaviour, safety, environment, acceptance, etc. This has to result in overall conclusions about a system, where the different impact areas are interwoven. However, this requires sitting together with the experts, and having time to let the results ‘sink in’. Often this time is not available.

It is recommended to make a template for the reporting of results early on (for both external and internal reports), so researchers know what is expected from them. Reporting in a clear and systematic way helps with the integration of results and the overall view on the effects of the system.

10.4 Socio-economic Cost Benefit Analysis

The analyst faces choices in setting up and carrying out the analyses. The choices will be influenced by the priorities identified by those setting up the FOT. as well as budget and time constraints. The list below summarises these choices.

Methods:

• The basic choice is CBA, which summarises benefits and costs at a societal level

• Stakeholder perspectives: Makes use of the same input data as the CBA, but considers stakeholder-specific benefits, costs and financial analyses.

Identification of impacts:

• The basic choice is to use the costs incurred and the main expected benefit(s), as identified by use of the impact table.

• Other impacts — both direct and indirect — can also be included, depending on the stakeholder perspective as well as the choices made elsewhere in the project, for example in hypothesis formulation, measurement methods and equipment and modelling capability.

• Willingness-to-Pay evidence, if also collected during the FOT. can be used to supplement the analysis methods above.

Scope of geographical assessment:

• The basic choice is the country level. In this case, the generic data needs (see section 1.4.2) are limited to the country in question.

• EU-level analyses are preferred. These require substantially more general data from individual countries. Extra challenges in execution can be encountered due to differences in definitions or classifications.

10.4.1 System costs

System cost estimation is an element within FOTs which is quite often neglected. System promoters may not see costs as an impact. However from a socio-economic point of view, they are a (negative) part of the impact of systems. Cost estimation should take care of the following aspects:

• Cost elements to include: The system costs comprise the costs of in-vehicle, roadside infrastructure equipment and nomadic devices. Besides that, operating and maintenance costs have also to be considered. Examples of good practise for system costs can be found in US American FOT assessments (Freightliner FOT, Mack FOT, Volvo FOT).

• Relevant size of costs: CBA applies a resource based view. This means looking at potential savings of productive resources and on the other hand at the resources necessary to achieve this effect. The implication for cost estimation is that only the input of productive resources is relevant and not potential market prices. The convention proposed e.g. by eIMPACT is to use the cost price (the price of the ICT system paid by the manufacturer to its supplier) plus a mark-up which is allowed for in-vehicle implementation. However, the contrary, market prices are relevant for user-centred analyses. Generally, in the face of limited evidence it is useful to apply the “Factor 3” rule of thumb, which means that in the automotive industry market prices for ICT systems differ from the cost prices by a factor of 3.

• Process of cost estimation: Typically, cost estimation will be carried out by an expert group comprising of FOT internal staff and external industry experts. To avoid conflicts with confidentiality and the like, it appears sometimes helpful to introduce rough estimations to the group instead of working from blank sheets. Guidance to rough estimations for investment and OEM costs can be applied from an US-American database on ITS costs and benefits (www.itscosts.its.dot.gov).

10.4.2 Classification of assessment methods

Figure 10.2 gives a classification of socio-economic assessment methods, based on which of the elements are included, and in particular: • Whether a full set of impacts is addressed — for example, if a significant CO2 reduction can be anticipated, has it been included; • Whether the assessment is from the social perspective only, or whether financial and stakeholder analyses are also provided. The recommendation is that the FOTs should be designed to be as complete as possible, both in terms of impacts and stakeholder views. The assessments in the FOTs reviewed are examples of good practice. However, they differ in the types of analyses carried out, as well as in the scope of the effects examined, with the exception of safety impacts.

Figure 10.2 highlights another dimension in which assessment methods can be classified, namely whether or not they make use of case-specific Willingness-to-Pay (WTP) evidence. In the design of future FOTs, we recommend that clients and analysts consider WTP studies as a way of getting better evidence on the users' likely demand for the products. WTP can provide uniquely useful evidence on the value of the ICT system to consumers and producers. In absence of this, FOTs can refer to evidence in the literature (market-based). WTP studies will, however, add to the cost and skill set required for FOTs, so the advantages and disadvantages will need to be weighed in each case.

We note that past FOTs have generally relied on market-based values (e.g. the U. S. CAS and Mack FOT), although the U. S. ICC FOT did make use of specific WTP evidence, and as such is a useful reference. Also, we note that most previous assessment guidelines, including eIMPACT, assume that literature-based values will be used. Here, we leave the option open and recommend that clients and analysts decide at the inception phase of the FOT whether or not to go down the WTP route.

10.4.3 How to carry out the assessment

10.4.3.1 Carrying out a CBA

The socio-economic impact assessment of a system within an FOT should be based on a CBA, since it is the most widespread, commonly accepted and practised method for analysing socio-economic impacts. It is clear that CBA accounts for all benefits and all costs on a society level, including benefits and costs to all groups. CBA follow a four-step-process involving framework and preparatory work, measuring impacts, appraising impacts in a common monetary value and confronting the discounted society benefits with the costs of the policy measure. However, this process leaves also some room for shaping the individual steps of the process. We recommend considering the following issues.

1. CBA framework:

Definition of the cases to be compared: Looked at is the with-case (ICT-system equipped) against the without-case (without the system).

Base year and time horizon of the assessment: CBA can be performed for the whole life cycle of the considered system or only for selected target years. This decision depends on information needs.

Geographical scope: Because of data availability the geographical scope should be congruent to existing statistical reporting ICT systems. Reference only to the local area where the FOT takes place is insufficient for this reason and because the results of different FOTs need to be compared. This implies however that the socio-economic impact assessment has to undergo a scaling up procedure before the CBA in order to project the impacts from the FOT itself on to a larger area. The most practical appears to be assessment at the national level (assuming “nationwide deployment”). However, it is even more useful to provide results on a European level. The European perspective is important when the effects of FOTs in different member states should be compared or when policy measures are planned or considered to ensure a European scale deployment (e.g. eCall).

Discount rate: The discount rate ensures that benefits and costs are expressed for a common base year. A discount rate of 3 % (real) is recommended as a default (see 'Other economic parameters').

Deployment scenario: It has to be estimated which share of new vehicles or which share of the total vehicle fleet will be equipped with the system in the target years and over the assessment period as a whole (depends on answer to 'Base year and time horizon' issue above). For life cycle assessment it is also necessary to estimate the development of the equipment (technical capabilities, costs).

Impact table: The impact table serves as an instrument to expedite identification of impacts. It is aimed to ensure that the FOT team and the group responsible for the socio-economic impact assessment are fully aware of the complete impacts of the system. For efficiency reasons and likely budget constraints (competing FOTs and competing assessment issues within an FOT) it is necessary to concentrate the analysis on the significant impacts — impacts expected to be negligible, or impossible to analyse within the resources available, should be flagged as such in the impact table. Concerning the system, safety is the relevant impact by definition. Direct and indirect mobility impacts and environmental impacts are typically also addressed. System costs will always be relevant.

2. Inputs for impact assessment (including cost estimation)

Impact measurements represent an essential input to the cost-benefit assessment. We would normally expect most of these to feed through from the FOT experiment to the scaling-up procedure (chapter 9) to the CBA inputs. In particular, accident prevention and system costs at the national / EU levels should be delivered this way. Impacts on mobility and environment will typically require additional analysis at the CBA stage (although in a well-designed FOT experiment, it may be possible to gather data specifically on any expected sources of benefit, e.g. reduced variability of traffic speeds or reduced fuel consumption; see the TAC Safe Car FOT). The analysis of different FOT assessment has revealed some evidence on best practise for impact measurement. The requirements for CBA can be provided as a sort of output specification. This makes sure that the socio-economic impact assessment will be provided with the appropriate input data for carrying out the assessment. In terms of an output specification the following elements have to be put in place:

• Accident and traffic performance database: See section 10.4.4.

• Effectiveness of the system: These values represent key output of the FOT which have to be provided to the socio-economic impact assessment.

• Procedure for scaling up the effects to nationwide/European level

• Cost estimations: See section 10.4.1 on system costs.

3. Impact valuation

Methodological base for impact valuation: The general objective of this step is to provide unit values for the physical impacts. Several methods compete in the field of impact appraisal. They can be subdivided in objective approaches (e.g. damage costs, avoidance costs) and subjective approaches (e.g. willingness-to-pay). In European member states, different practises and preferences exist for impact appraisal. A lot of surveying and standardisation efforts have been made by projects like HEATCO (Bickel et al., 2006) to come to common European base. As a general recommendation, it can be stated that unit values for CBA should be based on objective approaches. However, willingness-to-pay information can largely contribute to a higher quality of the assessment when analyses for the users are carried out.

Good practise on unit values: See eIMPACT (Assing et al., 2006), HEATCO and the handbook on external costs of transport.

National or European unit values: This decision corresponds with the geographical scope. Assessment on national level will typically make use of national cost unit rates. For European scale assessment we recommend using the harmonized values contained in HEATCO — note that these are still differentiated by country, but are on a harmonised theoretical basis.

4. Results

Cost-benefit analyses can produce different summary measures of performance. It represents good practise to calculate the Net Present Value (NPV) by summing up all discounted values of benefits (plus sign) and costs (minus sign). Moreover, Benefit-Cost Ratios (BCR) are a very common expression of system profitability which can be calculated by dividing the total benefits by the total costs. It is also practical (see "Base year and time horizon") to calculate “snapshot” BCR for target years. In this case, the costs will be transformed to annual values (using the discount rate) and will be compared to the target year benefits. For FOTs, we recommend the calculation of both figures, NPV and BCR.

For the social CBA, we recommend reporting:

• safety benefit (€M);

• other benefits to road users (€M) — mainly time savings, operating cost savings and reliability gains;

• environmental benefits (€M) — including climate change, regional and local air quality effects; noise; and other impacts;

• revenue to operators (€M) — there may be multiple operators, including infrastructure and service operators – each will want to know the impact on themselves (financial), although for the social CBA these revenues may be aggregated;

• costs to operators (€M) — including capital, maintenance and operating costs;

• revenue to automotive OEMs (€M);

• costs to automotive OEMs (€M);

• revenue to government (€M) — including tax revenue changes;

• costs to government (€M) — including investments in R&D and in implementation of ICT systems.

Tabulation of the social CBA is shown in Table 10-2. All entries are at Present Values. A common base year (for prices and discounting) aids comparison across different technology options. RAILPAG (EIB, 2005) has a more detailed breakdown by stakeholders (an ‘SE Matrix’), which some analysts may find helpful in presenting the social CBA.

In cases where the public sector expects to contribute to the development or implementation of the system, we recommend also presenting a Benefit:Cost Ratio with respect to public sector support, which HEATCO (Bickel et al., 2006: 41-2) identifies in use by the EC, UK and Switzerland:

The calculation of the Benefit-Cost ratio (BCR) is delicate issue in CBA. On one hand the BCR is a very powerful measure, because it applies to the common situation where investment budgets are limited and maximum value for money is required (making best use of a scarce resource). On the other hand the definition of ‘costs’ (the denominator) can be problematic. As a general rule, the BCR is useful when the denominator is defined in the same way for all options being compared — for example, NPV per unit of central government budget (which would be a BCR of interest to central government). Our recommendation of a BCR with respect to Public Sector Support broadens this to the budget for public expenditure as a whole. This avoids creating an incentive to manipulate the BCR by shifting costs to local and regional government.

In the example shown in Table 10-2, the BCR with respect to public sector support will be 4240/(379-34) = 10.3, which indicates a high social return from each € of public funds contributed.

Table 10-2: Social CBA tabulation

 

Note:Sign - all negative impacts on the Group affectedare shown with a negative sign, thus Costs appear with a negative sign; 2008 base - indicatesappraisal at constant general prices using 2008 CPI, and with 2008 as the base year for discounting in the Present Value column.

10.4.3.2 Carrying out a stakeholder analysis

In contrast to CBA, only particular benefits and costs are relevant for particular stakeholders. The reduction of exhaust and CO2 emissions are not benefits to users, unless they are charged for it (through vehicle-taxes or tolls). The costs of in-vehicle equipment do not represent costs to the government, unless the government agrees to pay for a share of this. The consequence is that ICT systems which are profitable on society level (NPV, BCR) will not be deployed when a relevant stakeholder group is economically impaired. Hence, it is necessary to include stakeholder perspectives in the FOT socio-economic assessment.

Practically, stakeholder analyses make also use of accounting costs and benefits, but on the level of the individual stakeholder group. This implies the following for users, but also in general:

• Cost and benefits must be investigated according to their stakeholder relevance. Safety benefits (reduced accident and casualty risks) for instance are relevant to users (and to insurance companies as well).

• The appraisal of the impacts can be different. Users face market prices when considering the investment in a system (see factor 3 rule of thumb). For benefit evaluation the implication is to use market values if available (e.g. fuel consumption: station prices (incl. taxes) instead of net prices). Otherwise, willingness-to-pay approaches have their justification here because they are better suited to reflect individual preferences.

Further adaptations to the CBA approach involve the use of a different discount rate (reflecting private sector interest rates) and the use of a different result measure (fair market price for a pre-defined annual vehicle mileage or the critical (break-even) mileage for a given market price).

The stakeholder analysis reporting will vary with the analytical methods used. For example, in the TAC Safe Car Project, Monash University used subjective questionnaire methods to investigate users’ acceptance of several ICT systems including ISA (Regan et al., 2006).

Another useful form of stakeholder analysis from the User perspective is Willingness-to-Pay evidence, as shown in eIMPACT D3 (Assing et al., 2006 p. 119). For the vehicle OEMs and both infrastructure and service operators:

• Where they are commercial bodies, a financial analysis will provide the most important stakeholder information;

• Where they are public sector agencies, a financial analysis may need to be combined with an assessment against their public service objectives – however, in some cases the overall social CBA will serve this purpose, depending on the approach taken by the agencies involved.

10.4.3.3 Carrying out a financial analysis

The internal rate of return (IRR) of a project is the interest rate that will generate an NPV of zero. In an equation, this is:

where IRR is internal rate of return.

The stakeholder for whom the IRR is calculated compares the IRR with a target rate. This target rate depends for each stakeholder. For public authorities as a stakeholder the target rate will be less than for private investors as stakeholders.

In any case, a calculated BCR or IRR should be accompanied by an NPV. We recommend that financial IRRs are reported for all FOTs.

The IRR concept can be modified for comparison reasons. For his approach, the cash flow streams are subtracted. With the new cash flows the modified IRR is calculated. If the IRR is above the trigger rate, the project with the larger cash flow is the better project.

Of key interest will be the IRR from the point of view of specific stakeholders (or stakeholder types). The IRR for vehicle OEMs will influence their decision about investing in the technology. Similarly, the IRR for infrastructure operators and service operators will influence their decisions – particularly where these are commercial operations.

Hence the key information will be in the form:

IRROEMs = … %

IRRRoadAuthorities = … %

Further IRRs should be reported where there are other stakeholders with a commercial interest, for whom significant impacts are expected. Tables such as those used by WebTAG (DfT, 2005) also provide a useful series of snapshots of the financial impact. In this case, in order to be meaningful the tables should relate to specific stakeholders or stakeholder types, e.g. vehicle OEMs or road authorities.

The financial results can be taken a stage further by reporting the breakeven point in terms of sales or market penetration, or the target price, down to which the system must be engineered in order to achieve financial viability. Graphical presentations may be useful in these cases.

10.4.4 Data needs

The data needed to carry out a socio-economic assessment for an FOT are extensive, and fall into two broad categories:

• FOT-specific data which will be gathered during the FOT itself
• Generic data, which play a role in:
  • Scaling up the results from the experimental situation of the FOT to the national or EU level
  • Reaching a socio-economic assessment, based on the FOT data scaled-up to National or EU level.

The following section outlines the FOT-specific and generic data likely to be needed. Thereafter recommendations on ensuring data quality and validity are given. Management of the data for socio-economic assessment is after this.

10.4.4.1 FOT-Specific Data

The key items of FOT-specific data likely to be needed are:

Accident rates (or risks) with and without the ICT system in place for the FOT sample. These will need to be differentiated by all the key drivers of accident rates (risks) in the FOT sample (e.g. road type; driver type; traffic conditions) so that accurate extrapolations can be made to the whole network. Accident rates (or risks) will be needed with and without the ICT system in place for the FOT sample. These may need to be derived from data on unsafe behaviours if the sample is too small to contain a significant number of actual accidents, although this is likely to be done as part of the performance indicators in any case. See section 10.3 on Safety Benefits.

One approach to estimating the impact on accident rates uses the effectiveness rate ( % of relevant crash type avoided) as in the Collision Avoidance Systems (CAS) Benefits Study (NHTSA Benefits Working Group, 1996).

A more sophisticated approach can produce data on accident severity as well as accident rates. Since accident severity is determined by the severity of the most serious casualty only, a complementary item of data would be any expected change in the number of casualties per accident. Regan et al. (2006) measured time spent buckled-up and time before buckling-up to produce injury severity estimates.

Examples of how data is produced for accident severity and accident rates can be found in Regan et al. (2006),UMTRI et al.(2006), Volvo Trucks North America et al. (2007) and USDoT (1999).

Whichever approach is used to estimate accident rates and accident severity, the analysis will need to take account of any options in the implementation path. For example, in the Freightliner FOT study (Batelle Memorial Institute, 2003) there were four possible deployment groups (Hazardous Materials tankers; all tankers; tractor trailers; all large trucks) — input data will be required for each of these options.

Multiple scenarios may also be needed to enable sensitivity testing. That is, where there is uncertainty over accident rates/severity or other key variables, this can be handled through ‘what if’ scenarios based on combinations of the possible outcomes (Batelle Memorial Institute, 2003).

There may also be some value in having spatially differentiated data, and being able to link behaviour to traffic conditions, e.g. urban / nonurban and traffic congestions (Technische Universiteit Delft; 2008, Volvo Trucks North America et al., 2007).

Market penetration forecasts: In the literature, SEiSS (VDI/VDE-IT, 2005 and Baum et al., 2006) gives particular attention to market penetration.

Usage, reliability and compliance: Although the CAS Benefits Study (NHTSA, 1996) made assumptions about usage, reliability and compliance rather than gathering data, it did draw attention to these important factors in the out-turn effectiveness of ICT systems. Usage refers to the percentage of drivers (or of driving time) for which ICT systems installed on the vehicle will be switched-on and active. Reliability refers to the likelihood that ICT systems will operate without failure, technically. Compliance refers to the percentage of occasions on which the driver’s behaviour complies with warning or indication provided by the system.

Attitudinal and acceptance data: Many FOTs gathered attitudinal and acceptance data (Regan et al., 2006; UMTRI et al., 2006; NHTSA, 2006; and Volvo Trucks North America et al., 2007).

Costs of the ICT systems: See section 10.4.1. In some FOTs, data has been gathered which inputs directly into the maintenance and operating cost calculations (Volvo Truck FOT). Where the assessment period is longer than the expected service life of the equipment, replacement costs should be included (e.g. in the Freightliner FOT one round of replacement was included since the service life was 10 years and the assessment period 20 years; Batelle Memorial Institute, 2003).

10.4.4.2 Generic Data

The key items of generic data likely to be needed are:

National and EU-level network, fleet and traffic data, which are used in scaling-up the findings from the FOT to the level of political interest: The International Road Traffic and Accident Database, IRTAD (ITF, 2008) contains traffic data for the EU27. This includes vehicle kilometres on the total road network, vehicle kilometres on motorways, and vehicle kilometres on urban roads. Vehicle kilometres on rural roads can be derived; some data are missing.

ProgTrans European Transport Report (latest version: 2007/08) can be used as a valid source for forecasts. It contains vehicle stock and vehicle kilometres for 1) cars, 2) buses and coaches and 3) goods vehicles. Generally, the report covers past, present (incl. short-term forecasts for the next years) and future (longer term for selected target years). In the 2007/08 report the following years are covered: 1995, 2000, 2005, 2006, 2007, 2008, 2015 and 2020. Geographically, they cover EU-27 by member state plus some more (Norway, Switzerland, Croatia, Turkey, Belarus, Russia, Ukraine plus China, Japan and the USA).  

Accident data (accidents, fatalities, severe and slight injuries) for base scenario: National databases are available. At the EU-level, the collection and compilation fo accident data as a basis for the safety impact assessment is a challenge, especially when specific target accidents are going to be explored. Several EU-projects are dedicated to harmonising accident databases, See TRACE ([2]) or SafetyNet (http://erso.swov.nl/safetynet/content/safetynet.htm) for more information. Forecasts of road safety are needed. An example can be found in eIMPACT “Impact Assessment of Intelligent Vehicle Safety Ssytems”, in which road safety predictions for 2010 and 2020 for the EU-25 are presented.

More detailed network specifications (e.g. infrastructure equipment) may be required for some systems: the presence/absence of beacons, signalisation. Basic figures (e.g. share of Trans-European Road Network equipped with dynamic traffic management) are available from the eSafety Forum Implementation Roadmap Working Group.

Speed-flow relationships or network models, which allow journey times and costs to be derived from changes in flows: Although these are strictly much more than just ‘data’, it is worth highlighting the key role they play in socio-economic assessment of transport ICT systems. Many of the effects of new ICT systems will be mediated through changes in traffic flow on the network – for example, advanced warning ICT systems allow drivers to change route to avoid hazards, but the net effect on travel times and costs is dependent not only on the behaviour of the individual, but on the behaviour of large numbers of individuals and the interaction with the limited capacity of the network. Hence, at the very least, knowledge of speed-flow relationships is needed to understand the consequences of shifting traffic across the network.

HCM (2000) and FGSV (2001) are sources of speed flow relationships. Network models or strategic transport models incorporate this data and have much wider functionality. The fact that these models are very expensive to develop and maintain means that they tend not to be developed for one socio-economic assessment in isolation. Instead, part of the socio-economic assessment process is usually to identify models already existing which can provide the necessary functionality.

Evidence on accident costs, used to measure the benefits of accident reduction and changes in accident severity: The HEATCO project (Bickel et al., 2006) was designed specifically to provide harmonised cost estimates for socio-economic assessment in Europe. We recommend that the HEATCO accident cost values are used in the FOTs, and we provide one additional piece of evidence to fill a gap in HEATCO which is a generic dataset on the costs of ‘damage only’ accidents.

Two of the main issues in this field are:

• An apparent inconsistency between ‘willingness-to-pay’ (WTP) methods and ‘cost of damage’ or ‘human capital’ methods as a basis for values — empirically, WTP methods can produce significantly higher values for fatalities in particular (see Assing et al., 2006: Table 16);
• Double counting of casualties’ lost future consumption, which is included in both lost future output and WTP to reduce accident risk.

HEATCO addresses these issues by specifying a common framework in which the different elements of accident costs measured by each method can be reconciled. For example, ‘human capital’ methods do not capture people’s full valuation of safety risk, whilst WTP-based values do not capture the external resource costs of accidents (e.g. healthcare costs borne by the state) but often do double-count lost future consumption, as already noted. The HEATCO framework includes:

• property damage;
• medical costs;
• administration costs;
• lost output;
• welfare losses due to casualty reduction.

As a result, the HEATCO values for fatalities are neither as high as the US NHTSA’s willingness-to-pay values cited by Assing et al., nor are they as low as the NHTSA’s cost-of-damage values. They are broadly in line with ‘best practice’ European values used in cost-benefit analysis, and the differences can generally be understood by examining the differences in the underlying measurement methods.

Other important functions of the HEATCO values are to provide:

• A common unit of account in the face of taxes and subsidies – HEATCO values are provided at the factor cost unit of account (Bickel et al., 2006: 52);
• A common price base year;
• A common currency, €, for European-level assessments.

In HEATCO accident values are listed in a table, expressed as values per casualty saved. These values do include the full set of accident costs, per casualty.

To apply these values, analysts will require further data:

1. Forecasts of accidents with and without the technology in place – based on the FOT findings and the results of the scaling up process.
2. If these forecasts do not address unreported accidents, then factors for the number of unreported accidents given the number of reported accidents can be found in HEATCO (Bickel et al., 2006, Table 5.1).
3. Growth in the values over time – an elasticity of 1.0 with respect to GDP per capita, thus a 2.0 % annual increase in GDP per capita would imply a 2.0 % annual increase in the values of accident reduction.
4. Damage only accident values. As Baum et al. (2007) shows, savings in damage only accidents can make up a large proportion of the benefits from ICT safety systems. Damage only accident costs may be approximated at 17 % of the cost of a Slight Casualty (Nellthorp et al., 1998).

National level assessments may wish to take advantage of the most recent safety valuation evidence at the national level. For multi-national assessments it will be important to ensure that any national evidence is checked for consistency across boundaries, and conversions made if necessary (e.g. in terms of base year, unit of account, cost elements included, measurement methodology, etc).

For EU-level assessments, consistency across countries and comparability between assessments will be important, which makes the use of a harmonised set of values (as above) more attractive. If the harmonised values are found not to provide the detail which the analyst wants — e.g. if differentiated accident costs by road type or user type are expected to be a key requirement for a particular assessment — then it may be appropriate to vary the values above, based on more detailed information (for example, the accident cost data included in national level assessment guidelines).

Evidence on values of time savings and vehicle operating cost savings, used to measure the benefits of changes in traffic flow: Values of travel time savings will be needed to assess the benefits of improved traffic flow due to the ICT system. HEATCO Tables 4.6-4.8 provide suitable values for working and non-working passenger trips, and for freight transport (Bickel et al., 2006: 73-75). These values increase with GDP per capita, at an inter-temporal elasticity of 0.7.

Sometimes there will be an impact on reliability, not only expected (mean) travel times, and in these cases we recommend using the reliability ratios set out in HEATCO Table 4.3 to value changes in the standard deviation of journey time.

Vehicle operating cost savings are also likely to arise from changes in traffic flows. The traffic models used to predict traffic flow responses to ICT systems will typically be capable of predicting changes in Vehicle Operating Costs, and the fine network detail in these models usually makes it more logical to calculate these cost savings within the model, rather than attempting to do so based on model outputs. As a result, standard values are not offered for these impacts by HEATCO (Bickel et al., 2006: 135-140).

Emissions factors and values for the damage caused by emissions of greenhouse gases, air pollutants and noise: HEATCO provides values for both sources of emissions (Bickel et al., 2006, Tables 6.2, 6.4). Values for particulate (smoke) emissions are differentiated between urban and non-urban locations, due to their much localised impact pathway. Other air pollutants are valued uniformly at country level. [ HEATCO] provides a shadow price of CO2 by year of emissions (Bickel et al.., 2006, Table 6.12) which should be applied to all forecast changes. The impact of noise changes may be quantified using the HEATCO values for road, rail and aircraft noise in each member state (Bickel et al., 2006, Table 6.9).

Other economic parameters such as the social discount rate: Discount rates are required for socio-economic assessment. In line with HEATCO, we recommend using a risk-free social time preference rated for the countries to which the assessment would apply. If a default discount rate at the EU level is required, we would recommend using 3 % per annum (real). GDP growth data for the members of the EU27 (required for updating values of accidents, etc, over time) is available from Eurostat.

10.4.4.3 Data quality and validity

The EC ROSEBUD project provided the following guidance as part of a “professional code for analysts” (BASt et al., 2005, p. 46):

“Data has to be attributed correctly to its sources, especially when different data sources like national or international accident databases or in-depth databases are used. Where and how estimations were made to fill data gaps needs to be documented. Regression models should be used to generate future time series; trend extrapolations can replace them where available data are insufficient for regressions”.

In addition, we would recommend that:

• The principles of statistics apply — statistical tests should be used wherever possible to determine if hypotheses about ICT system impacts are supported by the FOT evidence, and sample sizes should be chosen to obtain statistically significant results;
• When scaling-up from the FOT to the national or EU27 level, a methodical approach based on the key drivers of safety/other significant outcomes identified in the FOT should be used (cross reference);
• Confidence intervals as well as mean data should be recorded for key variables – note that confidence intervals are given in HEATCO for the various economic parameters recommended;
• We have noted the need to recognise the uncertainty in the data using sensitivity analysis – if analysts wish to take a more advanced approach and use Monte Carlo simulation or related techniques (for example to derive a probability distribution on NPV or BCR) that would be welcome as it simplifies the outputs seen by the decision-makers, although it does place an additional burden on the analysts;
• Known problems with the data should be acknowledged and acted upon, e.g. UMTRI et al. (2006) excluded a proportion of drivers whose trials were invalidated (in that case 9 out of 87 drivers), and some trips by the remaining drivers. Well-known problems with the omission of unreported accidents from data have prompted Bickel et al. (2006, Table 5.1) to provide adjustment factors for different accident severities and types.

Record keeping and data storage are important. This includes qualitative/subjective data, and evidence gathered during deliberative studies (e.g. UMTRI et al. (2006) ensured that focus group evidence was captured on video and by a court stenographer).

Finally, the US NHTSA observes that “the validity of any experimental test results depends on the experimental condition effects that were placed on the drivers” (NHTSA, 1996: p36). Care is needed, therefore, when extrapolating data from short-term experiments to long-term term adjustments in behaviour and demand for ICT systems — e.g. the CAS Benefits Study proposed that “a better estimation of the safety benefits … can be achieved as more relevant test data are gathered especially from long-term, large-fleet field operational tests” (p.C-8).