Difference between revisions of "FESTA handbook From Functions to Hypotheses"

From FOT-Net WIKI
Jump to navigationJump to search
(Created page with 'Under construction')
 
Line 1: Line 1:
Under construction
+
'''From Functions to Hypotheses'''
 +
 
 +
The final objective of an [http://wiki.fot-net.eu/index.php?title=FOT FOT] is to evaluate in-vehicle [http://wiki.fot-net.eu/index.php?title=Function functions] based on Information Communication Technology ([ ICT]) in order to address specific [http://wiki.fot-net.eu/index.php?title=Research_question research questions]. These [http://wiki.fot-net.eu/index.php?title=Research_question research questions] can be related to safety, environment, mobility, traffic efficiency, usage, and acceptance. By addressing the [http://wiki.fot-net.eu/index.php?title=Research_question research questions], [http://wiki.fot-net.eu/index.php?title=FOT FOT]s promise to furnish the major stakeholders (customers, public authorities, [ OEM]s, suppliers, and the scientific community) with valuable information able to improve their policy-making and market strategies. Individuating the most relevant [http://wiki.fot-net.eu/index.php?title=Function functions] and connected [http://wiki.fot-net.eu/index.php?title=Hypothesis hypotheses] to successfully address the above-mentioned [http://wiki.fot-net.eu/index.php?title=Research_question research questions] is one of the major challenges in an [http://wiki.fot-net.eu/index.php?title=FOT FOT] . In this Chapter, the process of individuating the [http://wiki.fot-net.eu/index.php?title=Function functions] to be tested in an [http://wiki.fot-net.eu/index.php?title=FOT FOT] and the relevant connected [http://wiki.fot-net.eu/index.php?title=Hypothesis hypotheses]  will be elucidated. Specifically, the reader will be guided in the process of
 +
 
 +
1) selecting the [http://wiki.fot-net.eu/index.php?title=Function functions] to be tested,
 +
 
 +
2) defining the connected [http://wiki.fot-net.eu/index.php?title=Use_case use cases]  to test these [http://wiki.fot-net.eu/index.php?title=Function functions],
 +
 
 +
3) identifying the [http://wiki.fot-net.eu/index.php?title=Research_question research questions] related to these use cases,
 +
 
 +
4) formulating the [http://wiki.fot-net.eu/index.php?title=Hypothesis hypotheses]  associated to these [http://wiki.fot-net.eu/index.php?title=Research_question research questions] , and
 +
 
 +
5) linking these [http://wiki.fot-net.eu/index.php?title=Hypothesis hypotheses]  to the correspondent [http://wiki.fot-net.eu/index.php?title=Performance_indicator performance indicators].
 +
 
 +
The [http://wiki.fot-net.eu/index.php?title=FOT FOT] chain shows specifically the steps reported above.
 +
 
 +
The steps may be influenced by other elements of the [ FOT] chain. The selection of [http://wiki.fot-net.eu/index.php?title=Function functions] may be driven by the socio-economic impact that is expected, or by the [http://wiki.fot-net.eu/index.php?title=Research_question research questions]. When details are filled in later on in the process, it may be necessary to re-visit earlier steps, for example limits in resources or technical capabilities may lead to a decision to limit the amount of [http://wiki.fot-net.eu/index.php?title=Function functions], [http://wiki.fot-net.eu/index.php?title=Use_case use cases]  and [http://wiki.fot-net.eu/index.php?title=Hypothesis hypotheses].
 +
 
 +
:# '''Systems and functions'''
 +
 
 +
In the last few years, the number of [ ICT] [http://wiki.fot-net.eu/index.php?title=Function functions] available for use on standard vehicles and more generally while travelling has been rapidly increasing. [ ICT] [http://wiki.fot-net.eu/index.php?title=Function functions] are intrinsically designed to provide the driver or traveller with new, additional information. However, the extent to which this increased amount of information from these [ ICT] [http://wiki.fot-net.eu/index.php?title=Function functions] results in clear and positive effects on safety, environment, mobility, usage, and acceptance in real traffic [http://wiki.fot-net.eu/index.php?title=Situation situation] is unknown. [http://wiki.fot-net.eu/index.php?title=FOT FOT]s warrant to evaluate, for the first time, these [ ICT] [http://wiki.fot-net.eu/index.php?title=Function functions] in a real traffic [http://wiki.fot-net.eu/index.php?title=Situation situation]  during naturalistic driving. In this handbook we refer to 1) in-vehicle, 2) cooperative, and 3) nomadic systems intended as a combination of hardware and software enabling one or more [ ICT] [http://wiki.fot-net.eu/index.php?title=Function functions]. Depending on the different [http://wiki.fot-net.eu/index.php?title=System systems] implementing a specific function, different challenges may have to be faced during the [http://wiki.fot-net.eu/index.php?title=FOT FOT] design.
 +
 
 +
It is important to note that [ NDS] in the future will increasingly include new [http://wiki.fot-net.eu/index.php?title=System systems] as they increase in market penetration. For example, future [ NDS] will be able to be used for with-and-without analyses of [http://wiki.fot-net.eu/index.php?title=System systems] that are new today, but commonplace tomorrow (e.g. Forward collision warning may be as commonplace as ABS tomorrow).
 +
 
 +
 
 +
 
 +
::# '''''Vehicle systems'''''
 +
 
 +
Vehicle Systems are a combination of hardware and software enabling one or more [http://wiki.fot-net.eu/index.php?title=Function functions] aimed at increasing driver’s safety and comfort. Vehicle Systems promise
 +
 
 +
1) to increase driver safety by increasing driver’s attention in potentially hazardous [http://wiki.fot-net.eu/index.php?title=Scenario scenarios] (such as the Forward Collision Warning),
 +
 
 +
2) to improve the driver’s comfort by automating some of the operational driving tasks (such as the Adaptive Cruise Control function),
 +
 
 +
3) to increase driver mobility by furnishing timely traffic information (such as the Dynamic Navigation function), and
 +
 
 +
4) to increase safety in critical situation by automating the vehicle response (such as the Collision Mitigation function).
 +
 
 +
Vehicle [http://wiki.fot-net.eu/index.php?title=System systems]  are becoming more and more standard equipment, even in middle class vehicles and commercial vehicles. However, their impact on the driver, the traffic system, the society, and the environment in the short-, but especially in the long-term is not fully understood. [http://wiki.fot-net.eu/index.php?title=FOT FOT]s can help quantifying the impact of vehicle [http://wiki.fot-net.eu/index.php?title=System systems]  on driver’s workload and to understand how different [http://wiki.fot-net.eu/index.php?title=Function functions] interact with each other in a real complex traffic [http://wiki.fot-net.eu/index.php?title=Situation situation]. Further, [http://wiki.fot-net.eu/index.php?title=FOT FOT]s will expose these [http://wiki.fot-net.eu/index.php?title=Function functions] to many improbable [http://wiki.fot-net.eu/index.php?title=Scenario scenarios] which are not possible to be tested during the [http://wiki.fot-net.eu/index.php?title=Function functions] evaluation phase.
 +
 
 +
::# '''''Cooperative systems'''''
 +
 
 +
[ Cooperative Systems] are vehicle [http://wiki.fot-net.eu/index.php?title=System systems]  based on vehicle-to-vehicle ([ V2V]), vehicle-to-infrastructure ([ V2I]) and infrastructure-to-vehicle ([ I2V]) communication technology. Communication technology has enabled a new class of in-vehicle information and warnings which are more precise in terms of time and location. It is foreseen, that the integration of [ cooperative systems] with autonomous [http://wiki.fot-net.eu/index.php?title=Function functions] will provide a new level of [ ADAS]. Infrastructure based information tells the driver for example what is the appropriate speed to keep on a specific part of road or warns the driver in case of ice on the road or fog. At European level, cooperative [http://wiki.fot-net.eu/index.php?title=Function functions] and [http://wiki.fot-net.eu/index.php?title=System systems] have been developed in several projects under the 6th and 7th Framework Programme, namely: [http://www.cvisproject.org/ CVIS], [http://www.pre-drive-c2x.eu/ PRE-DRIVE C2X], [http://www.prevent-ip.org/ PReVENT], [http://www.safespot-eu.org/ SAFESPOT], and [http://www.watchover-eu.org/ WATCHOVER].
 +
 
 +
One of the initial objectives of [ V2V], [ V2I], and [ I2V] is to increase road safety. The development of safety-critical [ V2V] [http://wiki.fot-net.eu/index.php?title=System systems] in Europe has been mainly promoted by the Car-to-Car Communication Consortium ([ C2C-CC]). More recently, the EC has placed emphasis on the application of [ cooperative systems] to achieve environmental and efficiency impacts.
 +
 
 +
 
 +
 
 +
The lower level communication protocols are based on [ IEEE] 802.11p working in the 5.9 GHz. frequency range.
 +
 
 +
A key point in [ cooperative systems] is standardisation. In October 2007 Technical Committee for Intelligent Transport Systems ([ ITS]) was launched by [ ETSI] with the target to turn research results into standards and in October 2009 European Commission published a [http://ec.europa.eu/enterprise/sectors/ict/files/standardisation_mandate_en.pdf Mandate] (N. 453) which is setting deadlines for the standardisation activities. The mandate has been accepted by [ ETSI] and [ CEN] and the activity is currently (2011) ongoing. One of the objectives of the mandate is  harmonisation with U.S. and Japanese standards as well as taking into account the ISO activity which includes the [ CALM] (Continuous Air interface for Long and Medium range communication) architecture.
 +
 
 +
[ Cooperative systems] differ in several areas to conventional areas, which directly influences the [http://wiki.fot-net.eu/index.php?title=FOT FOT] planning and operation.
 +
 
 +
Firstly, there are currently no [ cooperative systems] available in the vehicle market – this implies that all participating vehicles in the [http://wiki.fot-net.eu/index.php?title=FOT FOT]  have to be equipped manually with prototype [http://wiki.fot-net.eu/index.php?title=System systems]. This is costly and makes it difficult to find suitable test vehicle fleets. Also, drivers are not aware of such  and have no knowledge of their functionality. Before starting the [http://wiki.fot-net.eu/index.php?title=FOT FOT] , drivers have to be educated on the usage of [ cooperative systems].
 +
 
 +
Another difficult issue lies in the essence of cooperation: all [ V2V] [http://wiki.fot-net.eu/index.php?title=Function functions] rely on more than one vehicle being in communication range, and [ V2I] relies on roadside-stations. Before starting the [http://wiki.fot-net.eu/index.php?title=FOT FOT] , it is hard to estimate the number of vehicles in one area and how often vehicles will pass a Road-Side Unit ([ RSU]). This ''penetration rate'' is a crucial factor in designing and evaluating an [http://wiki.fot-net.eu/index.php?title=FOT FOT] .
 +
 
 +
Current estimates from recent [ V2X] [http://wiki.fot-net.eu/index.php?title=FOT FOT]s predict a minimal penetration rate of 10% for [ V2X] [http://wiki.fot-net.eu/index.php?title=Function functions] to show a noticeable effect on traffic safety and efficiency. Given the size of the [http://wiki.fot-net.eu/index.php?title=FOT FOT] area and the average distribution of vehicles, the number of participating vehicles should usually be considerably higher.
 +
 
 +
The penetration rate also influences the ''Frequency of ''[http://wiki.fot-net.eu/index.php?title=Event events]'' (FoE)''. In [ cooperative systems] [http://wiki.fot-net.eu/index.php?title=FOT FOT]s, the FoE is roughly related to the number of necessary vehicles for a function. While some [http://wiki.fot-net.eu/index.php?title=Function functions] work with only two vehicles (e.g. a slow vehicle warning), other [http://wiki.fot-net.eu/index.php?title=Function functions] require several more (e.g. traffic-jam ahead warning). For certain [http://wiki.fot-net.eu/index.php?title=Function functions] the combined frequency of [http://wiki.fot-net.eu/index.php?title=Event events] might make a naturalistic [http://wiki.fot-net.eu/index.php?title=FOT FOT] unfeasible.
 +
 
 +
A proper assessment of the penetration rate impact can be derived from dedicated simulations. A simulation environment should consider traffic effects, communication models, applications and their respective influence on each other. Such dedicated simulation environments are able to predict the frequency of [http://wiki.fot-net.eu/index.php?title=Event events] for the developed [http://wiki.fot-net.eu/index.php?title=Function functions] and thus support the [http://wiki.fot-net.eu/index.php?title=FOT FOT] design and setup process greatly in cooperative systems.
 +
 
 +
There are several implications derived from this effect to the test setup and execution of [ cooperative systems] [http://wiki.fot-net.eu/index.php?title=FOT FOT]s:
 +
 
 +
* The number of equipped vehicles is considerably higher or the [http://wiki.fot-net.eu/index.php?title=FOT FOT] needs to run considerably longer to collect enough data to be representative.
 +
* It might be very difficult or even not possible to run the [http://wiki.fot-net.eu/index.php?title=FOT FOT] uncontrolled for some [http://wiki.fot-net.eu/index.php?title=Function functions](e.g. Emergency electronic brake light,..)[  [http://wiki.fot-net.eu/index.php?title=FOT FOT]]. Controlled testing implies the usage of dedicated tools to specify the test cases, the test [http://wiki.fot-net.eu/index.php?title=Scenario scenarios] and to run the test. They also need to be linked to the vehicles and drivers to control and monitor the running test. (See 6.5 for more details)
 +
 
 +
It is most likely that the initial market drive will be for [ V2I]-based applications until there is a critical mass of equipped cars on the road. [http://wiki.fot-net.eu/index.php?title=FOT FOT]s can assess the technological and business feasibility of [ cooperative systems] and may be necessary to complete the validation of such [http://wiki.fot-net.eu/index.php?title=System systems]. In fact testing to validate [ cooperative systems] may require very complicated set-up which may not be possible unless in a real-traffic set-up.
 +
 
 +
One important issue is the installation of infrastructure devices that could be needed. Permission for the testing and team involvement should be planned in advance considering the impact on normal traffic during the installation phase. In the testing phase proper logistic support is needed in order to allow technical people to operate with debugging equipment in order to tune the [http://wiki.fot-net.eu/index.php?title=System systems]. Assessment of [ RSU]s and antennas positioning could be rather time-consuming. Protection and maintenance of road-side equipment should be ensured.
 +
 
 +
In comparison to [ ADAS] [http://wiki.fot-net.eu/index.php?title=Function functions] or [ nomadic devices], [ cooperative systems] are more complex to analyze due to the distributed nature of involved components. In former cases it can be sufficient to log the vehicle state and the [http://wiki.fot-net.eu/index.php?title=Function function]state. In contrast, to completely evaluate the functionality of a [ cooperative system] it is also necessary to handle all incoming and outgoing [ V2X] traffic and the Local Dynamic Map state on the vehicle side. As other vehicles and Roadside stations directly influence the [http://wiki.fot-net.eu/index.php?title=System system]  functionality the scope of logging has to be adapted as well. While a focus on single vehicles can be sufficient in other [http://wiki.fot-net.eu/index.php?title=FOT FOT]s, it has to be widened to complete [ ITS] areas in [ cooperative systems]. The logging [http://wiki.fot-net.eu/index.php?title=System system]  has to be able to cope with the broader scale both in [ ITS] station and [ ITS] area scope.
 +
 
 +
Each [ ITS] Vehicle Station needs to be equipped with a logging device. This device or software component has to be able to write a common log destination from all important sources on the Application Unit and Communication Unit:
 +
 
 +
* Vehicles Data ([ CAN] Bus)
 +
* Position and Time (if not present on [ CAN])
 +
* Networking: Incoming and Outgoing Messages (CAM, DENM)
 +
* Local Dynamic Map Status (List of all neighbours, List of all [http://wiki.fot-net.eu/index.php?title=Event events])
 +
* [http://wiki.fot-net.eu/index.php?title=Function function] Status
 +
* Other Facilities (connection to backend-services, etc.)
 +
 
 +
It is obvious, that all these various sources provide a huge amount of data. This is a major consideration for the logging [http://wiki.fot-net.eu/index.php?title=System system]  on each station and should influence it in two ways:
 +
 
 +
* Qualitative: Each log entry should be reduced to the minimum (entry type ID, timestamp, value). It is possible to further reduce the required space, e.g. by only writing the time and not the date into each entry. It can also be useful to compress the written files. It is strongly advised to use a “strict” logging scheme, which only allows pre-defined log entry types, as this simplifies post-processing and also allows to save disk space.
 +
* Quantitative: Logging does not have to record all values all the time.''''' '''''Although data storage cost is lowering more and more, if necessary,  it is possible to reduce the used space drastically by using pre-defined logging profiles. These are generated with a mapping from those [http://wiki.fot-net.eu/index.php?title=Performance_indicator performance indicators] needed for evaluation to the available measurements. For each test a logging profile can be pre-selected, which contains only those measurements needed.
 +
 
 +
In more complex settings the logging [http://wiki.fot-net.eu/index.php?title=System system]  can also automatically switch profiles depending on the measures values (a critical [http://wiki.fot-net.eu/index.php?title=Situation situation] can be detected from measurements and a wider logging profile can be selected for a given time). It is also possible to select the log profile directly in controlled testing to guarantee that all necessary measures are logged. The quantity of data to be collected is discussed in section 1.2 of the Chapter on Data Analysis and Modelling where two approaches (“space mission” and “minimum data set collection”) and related issues, are presented.
 +
 
 +
The [ ITS] Roadside Station provides similar measures (except vehicle data) and it is advised to use a similar architecture for the logging system. In contrast, the measures available to the [ ITS] Central Station can differ widely and are specific to each [http://wiki.fot-net.eu/index.php?title=FOT FOT]. Roadside sensors may be needed to provide information related to interaction with non-connected vehicles and other situational variables.
 +
 
 +
For controlled testing [http://wiki.fot-net.eu/index.php?title=Scenario scenarios] it is advisable to include monitoring features to the logging component. Monitoring is a real-time transfer of selected information available to the logger. In a central counterpart to monitoring this information is displayed to visualize the current test progress. Typically the monitoring value set is a subset of logging and should at least include vehicle ID, position, heading and speed. In this minimal set the test operator can follow vehicle progress through the test. More advisable monitoring sets include direct information from developed [http://wiki.fot-net.eu/index.php?title=Function functions] to monitor in real-time, if the test intent (to trigger the developed [http://wiki.fot-net.eu/index.php?title=Function functions] in a specified way) has been met. Test operators can thus determine if a test was successful (and the detailed logs are valuable for evaluation) or if tests have to be repeated right away. Sophisticated test control and monitoring tools support operators in this task.
 +
 
 +
Log files in [ cooperative systems] [http://wiki.fot-net.eu/index.php?title=FOT FOT]s can quickly take up gigabytes of data. A rough estimate for a typical [ cooperative system] is 500Mb per day per vehicle. It is important to design the test and logging [http://wiki.fot-net.eu/index.php?title=System system]  to cope with this data amount. Log files have to be saved locally on vehicle and roadside station, have to be transferred to the storage and have to be integrated into a central database. Throughout this process it has to be assured, that no data can be lost due to a single point of failure.
 +
 
 +
The transfer of data from vehicles to the central storage might exceed possible bandwidth from 3G connections. Other possible ways include transfer over WiFi and manual transfer by [ USB] Stick or Drive. Log data should be compressed to speed up transfer times. Depending on the vehicle availability (average run-time over day, time in between access to vehicles) one solution has to be chosen and the [http://wiki.fot-net.eu/index.php?title=System system]  has to be specified in a way, that the in-vehicle storage can fit enough data to hold all log-files until extraction. When large international [http://wiki.fot-net.eu/index.php?title=FOT FOT] are planned, [ cooperative systems] may require interoperability tests. The costs, the organizational and legal aspects (prototypes in foreign countries) should not be underestimated when planning these testing sessions.

Revision as of 10:46, 25 October 2011

From Functions to Hypotheses

The final objective of an FOT is to evaluate in-vehicle functions based on Information Communication Technology ([ ICT]) in order to address specific research questions. These research questions can be related to safety, environment, mobility, traffic efficiency, usage, and acceptance. By addressing the research questions, FOTs promise to furnish the major stakeholders (customers, public authorities, [ OEM]s, suppliers, and the scientific community) with valuable information able to improve their policy-making and market strategies. Individuating the most relevant functions and connected hypotheses to successfully address the above-mentioned research questions is one of the major challenges in an FOT . In this Chapter, the process of individuating the functions to be tested in an FOT and the relevant connected hypotheses will be elucidated. Specifically, the reader will be guided in the process of

1) selecting the functions to be tested,

2) defining the connected use cases to test these functions,

3) identifying the research questions related to these use cases,

4) formulating the hypotheses associated to these research questions , and

5) linking these hypotheses to the correspondent performance indicators.

The FOT chain shows specifically the steps reported above.

The steps may be influenced by other elements of the [ FOT] chain. The selection of functions may be driven by the socio-economic impact that is expected, or by the research questions. When details are filled in later on in the process, it may be necessary to re-visit earlier steps, for example limits in resources or technical capabilities may lead to a decision to limit the amount of functions, use cases and hypotheses.

  1. Systems and functions

In the last few years, the number of [ ICT] functions available for use on standard vehicles and more generally while travelling has been rapidly increasing. [ ICT] functions are intrinsically designed to provide the driver or traveller with new, additional information. However, the extent to which this increased amount of information from these [ ICT] functions results in clear and positive effects on safety, environment, mobility, usage, and acceptance in real traffic situation is unknown. FOTs warrant to evaluate, for the first time, these [ ICT] functions in a real traffic situation during naturalistic driving. In this handbook we refer to 1) in-vehicle, 2) cooperative, and 3) nomadic systems intended as a combination of hardware and software enabling one or more [ ICT] functions. Depending on the different systems implementing a specific function, different challenges may have to be faced during the FOT design.

It is important to note that [ NDS] in the future will increasingly include new systems as they increase in market penetration. For example, future [ NDS] will be able to be used for with-and-without analyses of systems that are new today, but commonplace tomorrow (e.g. Forward collision warning may be as commonplace as ABS tomorrow).


  1. Vehicle systems

Vehicle Systems are a combination of hardware and software enabling one or more functions aimed at increasing driver’s safety and comfort. Vehicle Systems promise

1) to increase driver safety by increasing driver’s attention in potentially hazardous scenarios (such as the Forward Collision Warning),

2) to improve the driver’s comfort by automating some of the operational driving tasks (such as the Adaptive Cruise Control function),

3) to increase driver mobility by furnishing timely traffic information (such as the Dynamic Navigation function), and

4) to increase safety in critical situation by automating the vehicle response (such as the Collision Mitigation function).

Vehicle systems are becoming more and more standard equipment, even in middle class vehicles and commercial vehicles. However, their impact on the driver, the traffic system, the society, and the environment in the short-, but especially in the long-term is not fully understood. FOTs can help quantifying the impact of vehicle systems on driver’s workload and to understand how different functions interact with each other in a real complex traffic situation. Further, FOTs will expose these functions to many improbable scenarios which are not possible to be tested during the functions evaluation phase.

  1. Cooperative systems

[ Cooperative Systems] are vehicle systems based on vehicle-to-vehicle ([ V2V]), vehicle-to-infrastructure ([ V2I]) and infrastructure-to-vehicle ([ I2V]) communication technology. Communication technology has enabled a new class of in-vehicle information and warnings which are more precise in terms of time and location. It is foreseen, that the integration of [ cooperative systems] with autonomous functions will provide a new level of [ ADAS]. Infrastructure based information tells the driver for example what is the appropriate speed to keep on a specific part of road or warns the driver in case of ice on the road or fog. At European level, cooperative functions and systems have been developed in several projects under the 6th and 7th Framework Programme, namely: CVIS, PRE-DRIVE C2X, PReVENT, SAFESPOT, and WATCHOVER.

One of the initial objectives of [ V2V], [ V2I], and [ I2V] is to increase road safety. The development of safety-critical [ V2V] systems in Europe has been mainly promoted by the Car-to-Car Communication Consortium ([ C2C-CC]). More recently, the EC has placed emphasis on the application of [ cooperative systems] to achieve environmental and efficiency impacts.


The lower level communication protocols are based on [ IEEE] 802.11p working in the 5.9 GHz. frequency range.

A key point in [ cooperative systems] is standardisation. In October 2007 Technical Committee for Intelligent Transport Systems ([ ITS]) was launched by [ ETSI] with the target to turn research results into standards and in October 2009 European Commission published a Mandate (N. 453) which is setting deadlines for the standardisation activities. The mandate has been accepted by [ ETSI] and [ CEN] and the activity is currently (2011) ongoing. One of the objectives of the mandate is harmonisation with U.S. and Japanese standards as well as taking into account the ISO activity which includes the [ CALM] (Continuous Air interface for Long and Medium range communication) architecture.

[ Cooperative systems] differ in several areas to conventional areas, which directly influences the FOT planning and operation.

Firstly, there are currently no [ cooperative systems] available in the vehicle market – this implies that all participating vehicles in the FOT have to be equipped manually with prototype systems. This is costly and makes it difficult to find suitable test vehicle fleets. Also, drivers are not aware of such and have no knowledge of their functionality. Before starting the FOT , drivers have to be educated on the usage of [ cooperative systems].

Another difficult issue lies in the essence of cooperation: all [ V2V] functions rely on more than one vehicle being in communication range, and [ V2I] relies on roadside-stations. Before starting the FOT , it is hard to estimate the number of vehicles in one area and how often vehicles will pass a Road-Side Unit ([ RSU]). This penetration rate is a crucial factor in designing and evaluating an FOT .

Current estimates from recent [ V2X] FOTs predict a minimal penetration rate of 10% for [ V2X] functions to show a noticeable effect on traffic safety and efficiency. Given the size of the FOT area and the average distribution of vehicles, the number of participating vehicles should usually be considerably higher.

The penetration rate also influences the Frequency of events (FoE). In [ cooperative systems] FOTs, the FoE is roughly related to the number of necessary vehicles for a function. While some functions work with only two vehicles (e.g. a slow vehicle warning), other functions require several more (e.g. traffic-jam ahead warning). For certain functions the combined frequency of events might make a naturalistic FOT unfeasible.

A proper assessment of the penetration rate impact can be derived from dedicated simulations. A simulation environment should consider traffic effects, communication models, applications and their respective influence on each other. Such dedicated simulation environments are able to predict the frequency of events for the developed functions and thus support the FOT design and setup process greatly in cooperative systems.

There are several implications derived from this effect to the test setup and execution of [ cooperative systems] FOTs:

  • The number of equipped vehicles is considerably higher or the FOT needs to run considerably longer to collect enough data to be representative.
  • It might be very difficult or even not possible to run the FOT uncontrolled for some functions(e.g. Emergency electronic brake light,..)[ FOT]. Controlled testing implies the usage of dedicated tools to specify the test cases, the test scenarios and to run the test. They also need to be linked to the vehicles and drivers to control and monitor the running test. (See 6.5 for more details)

It is most likely that the initial market drive will be for [ V2I]-based applications until there is a critical mass of equipped cars on the road. FOTs can assess the technological and business feasibility of [ cooperative systems] and may be necessary to complete the validation of such systems. In fact testing to validate [ cooperative systems] may require very complicated set-up which may not be possible unless in a real-traffic set-up.

One important issue is the installation of infrastructure devices that could be needed. Permission for the testing and team involvement should be planned in advance considering the impact on normal traffic during the installation phase. In the testing phase proper logistic support is needed in order to allow technical people to operate with debugging equipment in order to tune the systems. Assessment of [ RSU]s and antennas positioning could be rather time-consuming. Protection and maintenance of road-side equipment should be ensured.

In comparison to [ ADAS] functions or [ nomadic devices], [ cooperative systems] are more complex to analyze due to the distributed nature of involved components. In former cases it can be sufficient to log the vehicle state and the functionstate. In contrast, to completely evaluate the functionality of a [ cooperative system] it is also necessary to handle all incoming and outgoing [ V2X] traffic and the Local Dynamic Map state on the vehicle side. As other vehicles and Roadside stations directly influence the system functionality the scope of logging has to be adapted as well. While a focus on single vehicles can be sufficient in other FOTs, it has to be widened to complete [ ITS] areas in [ cooperative systems]. The logging system has to be able to cope with the broader scale both in [ ITS] station and [ ITS] area scope.

Each [ ITS] Vehicle Station needs to be equipped with a logging device. This device or software component has to be able to write a common log destination from all important sources on the Application Unit and Communication Unit:

  • Vehicles Data ([ CAN] Bus)
  • Position and Time (if not present on [ CAN])
  • Networking: Incoming and Outgoing Messages (CAM, DENM)
  • Local Dynamic Map Status (List of all neighbours, List of all events)
  • function Status
  • Other Facilities (connection to backend-services, etc.)

It is obvious, that all these various sources provide a huge amount of data. This is a major consideration for the logging system on each station and should influence it in two ways:

  • Qualitative: Each log entry should be reduced to the minimum (entry type ID, timestamp, value). It is possible to further reduce the required space, e.g. by only writing the time and not the date into each entry. It can also be useful to compress the written files. It is strongly advised to use a “strict” logging scheme, which only allows pre-defined log entry types, as this simplifies post-processing and also allows to save disk space.
  • Quantitative: Logging does not have to record all values all the time. Although data storage cost is lowering more and more, if necessary, it is possible to reduce the used space drastically by using pre-defined logging profiles. These are generated with a mapping from those performance indicators needed for evaluation to the available measurements. For each test a logging profile can be pre-selected, which contains only those measurements needed.

In more complex settings the logging system can also automatically switch profiles depending on the measures values (a critical situation can be detected from measurements and a wider logging profile can be selected for a given time). It is also possible to select the log profile directly in controlled testing to guarantee that all necessary measures are logged. The quantity of data to be collected is discussed in section 1.2 of the Chapter on Data Analysis and Modelling where two approaches (“space mission” and “minimum data set collection”) and related issues, are presented.

The [ ITS] Roadside Station provides similar measures (except vehicle data) and it is advised to use a similar architecture for the logging system. In contrast, the measures available to the [ ITS] Central Station can differ widely and are specific to each FOT. Roadside sensors may be needed to provide information related to interaction with non-connected vehicles and other situational variables.

For controlled testing scenarios it is advisable to include monitoring features to the logging component. Monitoring is a real-time transfer of selected information available to the logger. In a central counterpart to monitoring this information is displayed to visualize the current test progress. Typically the monitoring value set is a subset of logging and should at least include vehicle ID, position, heading and speed. In this minimal set the test operator can follow vehicle progress through the test. More advisable monitoring sets include direct information from developed functions to monitor in real-time, if the test intent (to trigger the developed functions in a specified way) has been met. Test operators can thus determine if a test was successful (and the detailed logs are valuable for evaluation) or if tests have to be repeated right away. Sophisticated test control and monitoring tools support operators in this task.

Log files in [ cooperative systems] FOTs can quickly take up gigabytes of data. A rough estimate for a typical [ cooperative system] is 500Mb per day per vehicle. It is important to design the test and logging system to cope with this data amount. Log files have to be saved locally on vehicle and roadside station, have to be transferred to the storage and have to be integrated into a central database. Throughout this process it has to be assured, that no data can be lost due to a single point of failure.

The transfer of data from vehicles to the central storage might exceed possible bandwidth from 3G connections. Other possible ways include transfer over WiFi and manual transfer by [ USB] Stick or Drive. Log data should be compressed to speed up transfer times. Depending on the vehicle availability (average run-time over day, time in between access to vehicles) one solution has to be chosen and the system has to be specified in a way, that the in-vehicle storage can fit enough data to hold all log-files until extraction. When large international FOT are planned, [ cooperative systems] may require interoperability tests. The costs, the organizational and legal aspects (prototypes in foreign countries) should not be underestimated when planning these testing sessions.