SCOREF
| SCOREF | |
|---|---|
| General information | |
| Type: Field operational test | |
| Tested system/service: Cooperative Systems | |
| Countries: France | ? test users |
| 20 partners | 42 vehicles |
| Active from 09/2010 to 09/2013 | |
| Contact | |
| http://blog.inria.fr/scoref/ | |
| Gérard SEGARRA | |
| gerard.segarra@renault.com | |
| RENAULT France | |
| Catalogue entries | |
| Data catalogue | Tools catalogue |
| Data sets used in this FOT: No data set is |
The following tools were used in this FOT: No tool is linked |
A French Field Operationnal Test for Road Cooperative Systems, in collaboration with DRIVE C2X and CO-DRIVE projects. The purpose of the project is to test a first set of thirteen use cases, in order to prepare a large-scale FOT before deployment. A hundred of naive drivers have been recruited and three test sites were opened in order to cover a diversity of driving situations. Most use cases were related to road-safety.
Abstract Cooperative road-based systems (C-ITS) are devices allowing the exchange of various kinds of information (CAN data, server-based contents) between nearby vehicles, or between vehicles and Road-Side Units (RSUs). This technology, termed C2X, opens up several prospects for applications in the automotive field: in addition to infotainment (e.g.notifying the user of points of interest for tourism) it is expected to act as a relay for driving assistance systems (e.g. collision alarm in an intersection) and as a system to support the implementation of eco-friendly driving practices (e.g. suggesting alternate modes of transportation). A set of studies regarding the acceptability of C-ITSs was carried out in France between 2012 and 2013, as part of project Score@f, in order to compare the expected benefits of these systems to their effective use. Hundred participants, aged 25 to 61, were recruited and requested to test a cooperative system in one of three following conditions : 1) as part of a dynamic driving simulation (Simulator Studies), 2) as part of their everyday home-to-work journey (Naturalistic Tests) or 3) finally within a run (Controlled Tests). In all conditions, drivers’ behaviours were recorded and questionnaires and interviews were run. Drivers agree on the relevance of applications related to road safety. However, between 43 and 66% of participants rejected four of the six the safety measures that had been provided. Four arguments were presented: the information is thought to be (1) too trivial, (2) redundant with information provided by another medium, (3) too late, and (4) presented too frequently. Whatever the case, the disruption introduced by consulting the Human- Computer Interface and the stress triggered by the alarm are therefore not compensated. Applications related to mobility and comfort lead to even greater divergence: none of these solutions were supported by a clear majority of participants. Three factors of acceptability emerge: (1) the a priori disposition of drivers related to using a new technology, (2) trust in the technical system, and finally (3) the definition each user gives of "good information" and the ability of the system to adjust to it. Our results suggest recommending the introduction, within the design process, of filters related to the triggering conditions of each use case.
Details of Field Operational Test
Start date and duration of FOT execution
Four small-scale FOTs.
- FOT#1 (Piloting): Controlled Tests, A10 (Test Site 1). Start: Late August, 2012 End: Early September, 2012 (34 drivers)
- FOT#2: Naturalistic Tests, Yvelines (Test Site 2). Piloting. Start: March, 28, 2013. Duration: 1 month (6 drivers) Test. Start: June, 10, 2013. Duration: 1 month (9 drivers)
- FOT#3: Controlled Tests, Yvelines (Test Site 2) Start : May, 2013 End: June, 2013 43 half-and-hour runs (24 drivers)
- FOT#4: Controlled Tests, Isère (Test Site 3). Start : June, 2013 End: September, 2013 53 fifty-minutes runs (53 drivers)
Geographical Coverage
Three different areas: Yvelines (78), Isère (38), A10 (North of Orleans)
SCORE@F is contributing to DRIVE C2X. Unfortunately the technical implementation was not compatible with DRIVE C2X which implemented former ETSI TC ITS standards (CAM & DENM version 1) while SCORE@F has been implementing the latest version. So the liaison was mainly achieved for non-technical matters (driver behaviour, user acceptability...etc.).
- SISCOGA. http://wiki.fot-net.eu/index.php?title=SISCOGA
- simTD. http://wiki.fot-net.eu/index.php?title=SIMTD
- DRIVEC2X. http://wiki.fot-net.eu/index.php?title=DRIVE_C2X
Objectives
Score@ f is the first French FOT dedicated to road cooperative systems. The purpose of the project is to test a first set of use cases, in particular those related to road safety, in view of preparing a large-scale FOT before deployment. For this purpose, naive drivers were recruited and several test sites were opened in order to cover a diversity of driving situations.
Results
GLOBAL RESULTS:
- Qualification of the performance of G5 technology (partial results focusing on the use of CCH: 5.895 to 5.905 GHz. Coverage was variable with a minimum of about 300 meters for V2V communication and a maximum of 1.3 km for V2I on motorway (A10). Qualification of the standards enabling the interoperability between Renault vehicles, PSA vehicles and NEAVIA Road Side Equipement.
- Identification by all stakeholders of a viable deployment. This led to a two steps deployment:
First step: The building of a deployment pilot concentring 2000 vehicles and 100 road side equipment in a limited geographical area (south and south east of Ile de France (outskirt of Paris). This deployment pilot is called SCOOP@F (Système Coopératif Pilote en france)and is built with road authorities (public and private) and OEMs.Second step: General deployment.
DETAILS FOR ACCEPTANCE:
* HMI issues Human-Machine Interfaces substantially influence the perception of each use case usefulness. This is particularly true in the case of safety-related use cases, for which the HMI consultation may conflict with the supervision of the road. Acceptance evaluations were sometimes penalized by an unfavorable location of the HMI. Beyond this limit, drivers do ask, concerning advisory and warning messages, for augmented reality solutions: windshield projection or mirror display. The sound dimension is identified as a crucial support to drive the attention and inform the driver of the type of message (warning/ advisory/ informative) before visual consultation.
* Safety-related use cases (V2V advisory/warning messages) Messages related to road safety are in theory very positively received: drivers do consider highly valuable to better anticipate any road hazard. They are theoretically perceived as likely to avoid an accident, to reduce the risk for other or or at least to provide comfort (less stress). However, in practice, several use cases have been rejected, either systematically either under specific conditions. Four reasons have been provided:
- The warning/ advisory is trivial: it informs the driver of something that he is already awarded about [1];
- Rising stress that comes with the warning/ advisory is not motived enough: the driver considers that the discomfort is not justified in view of the actual risk, whether this risk is too law (e.g. : vehicle parked on the roadside) or it is considered very unlikely (e.g. Pedestrian warning) [2]
- The warning/ advisory is relevant but it is too late: time is now to focus on the road [3];
- The warning/advisory may be too frequent: the driver fears not to bear forward alerts, which will lose anyway their ability to attract his/er attention.
Finally, all drivers are highly interested in being provided information (1) issued early enough and related to (2) an actual hazard (3) that is placed on the road. Around this typical case, opinions differ according to drivers and to their own definitions of danger.
* Acceptance of the comfort and mobility use cases (I2V informative messages)
- VMS messages are more or less popular according to their content: road hazard signaling, traffic information, road facilities or pedagogical message. For some drivers, any message that is no longer related to road safety is not relevant. In this case, only few messages, that providing first priority information, are accepted. For other drivers, all or part of the other information are also welcomed. It is often the case for traffic-related information, to the extent that the driver believes s/he is not already sufficiently informed by his/her Smartphone or GPS.
In all cases, drivers ask for a merging of the information carried by the cooperative system with the information provided by the other on-board systems. They wish to avoid the consultation of several media and the confusion that could ensue if the various media do not provide the same information.
- Points of Interest Notifications, in turn, generate more consensual judgments. Drivers seem to agree to keep touristic POI insofar they meet their interests. Car Pooling suggestions arouse the curiosity of a majority of drivers, who declare to be a priori in favor of introducing a friendly dimension to car use.
Lessons learned
The naturalistic FOT is an essential and powerful tool. It is the preferred system to create a private experience that is connected to everyday life. It makes it possible to obtain data of the highest quality as well as data that had not been foreseen by designers. Finally, it is a crucial tool to allow an encounter between the various stakeholders in the chain of communication: implementing this method allows these stakeholders to live a concrete experience of collaborative work. However, a naturalistic FOT is a tool that is complex and very costly to set up, particularly in the case of C-ITSs. At the current time, its cost-benefit ratio should be improved. Indeed, it is inoperative for any function that cannot be tested for safety reasons (e.g. motorcycle warning). Furthermore, it has turned out to be of limited effectiveness for all functions concerning car-to-car interaction – i.e. functions relying on two fitted vehicles to cross paths. In everyday practice, many participants adjusted their travel times and route depending on traffic: they did not systematically choose a specific route; and to be sure to cross paths with them, it was necessary to use the confederate vehicles for extended spans of time.
The controlled FOT is interesting to be able to compare drivers and to accelerate encounters with system functions (several successive use cases during a route). However, this version of a FOT can quickly lead to loss of quality in data collection. It is difficult to establish a private, personal experience of the system, particularly when participants are driving along in a cohort because of mutual distraction effects. It is also difficult to establish the credibility of messages, including when one introduces the elements corresponding to alarms in the real scene. Indeed, the drivers know they have been convened for the study and quickly express the view that the succession of incidents is unrealistic. The risk here is losing the key benefit of an open-road study, i.e. its ecological validity. This depends largely on the efforts of experimenters (choosing the route more or less judiciously) on the one hand, and on the drivers on the other hand – each driver being more or less prompt to project him- or herself into real-world use.
Studies in dynamic simulators have led to very positive results. First, they were the experiments that could be most flexibly implemented. More interestingly, dynamic driving simulator appears to be able to create an experience: immersion, which at one time appeared to be the weakness of this kind of device, seemed to be good for most of our participants. In contrast, the simulator is a limited tool for crude analysis of user behavior. It allows broad comparisons between drivers. but the precise values of dynamic parameters can undergo heavy distortions with respect to the real world. Let us also note, finally, that this tool opens up the possibility of studying critical use cases, but that it presents, in itself, some limitations. It is not possible, including when fully “in context”, to expose volunteers to deep fears. It is also often very complicated and costly to alter a 3D environment to tailor it to an ideal scenario for each message.
To conclude, we recommend:
- To combine several different methods of data collection. No one single method is sufficient. Particularly, FOTs should be prepared and completed with simulator-based trials;
- To favor an iterative approach, by including, in addition to the FOT, lighter stages of data collection so as to provide continuous input for the design process;
- To improve the tools for FOT supervision, particularly for processing video data and collecting user impressions (diaries, interviews, and acceptability inquiries);
- To develop tools to assess customer value associated with the FOT, in order to provide the economic elements that are crucial to system deployment.
Main events
- September 20, 2011 : Forum des Acteurs No.1 at Inria Rocquencourt - Domaine de Voluceau, 78150 Rocquencourt, FRANCE. http://blog.inria.fr/scoref/forum-des-acteurs-2011/
- September, 25, 2012: Forum des Acteurs No.2. at IFSTTAR Satory - 25 allée des marronniers, 78000 Versailles - https://project.inria.fr/scoref/forum-des-acteurs-2012/
- September, 24, 2013 : Forum des Acteurs No.3 / Final Event at INRIA Rocquencourt - Domaine de Voluceau, 78150 Rocquencourt, FRANCE.https://project.inria.fr/scoref/forum-des-acteurs-2013/
Financing
Summary, type of funding and budget
Overall
5.6 M€
Public
2.7 M€
Private
2.9 M€
Cooperation partners and contact persons
Partners
- Large corporations: Renault (Project leader), PSA Peugeot Citroën, Orange, Cofiroute, Hitachi, Viveris, Egis Mobility
- SMEs: LAB, UTAC, Intempora, Deveryware, Marben, Neavia, Fareco
- Academics: Eurecom, IFSTTAR, INRIA, Telecom Business School
- Public bodies: Centre d'études Techniques de l'Équipement du Sud-Ouest et de l'Ile de France
- Public authorities : Conseil Général des Yvelines, Conseil Général de l'Isère
Contact persons
- Public Authorities/ Public bodies:
- Conseil Général de l’Isère : Jean-Daniel Demond. jeandaniel.demond@cg38.fr - Conseil Général des Yvelines: Didier Meheut. DMeheut@yvelines.fr - CETE Ile-de-France : Ludovic Simon. - CETE Sud-Ouest: Louadhi Khoudour. louahdi.khoudour@developpement-durable.gouv.fr
- Industry:
- Vehicle Manufacturer: Renault: Gérard Ségarra. gerard.segarra@renault.com; PSA-Peugoet-Citröen: Alain Servel. alain.servel@mpsa.com
- Supplier: Hitachi: ; Neavia: Guillaume Grolleau. guillaume.grolleau@neavia.com ; Intempora: Nicolas du Lac. nicolas.dulac@intempora.com
- Road Operators; Cofiroute : Franck Petit. franck.petit@cofiroute.fr; Conseil Général de l’Isère : Jean-Daniel Demond. jeandaniel.demond@cg38.fr; Conseil Général des Yvelines: Didier Meheut. DMeheut@yvelines.fr
- Telecom
- Others
- Users: For Renault recruitment : Cécile Barbier. ccilebarbier@gmail.com / Marlène Bel. bel.marlene@renault.com ; For PSA-Peugoet-Citröen recruitment: Luciano Ojeda. luciano.ojeda@mpsa.com
- Universities:
- Telecom Business School : Olivier Segard. olivier.segard@telecom-em.eu
- Research Institutes: INRIA: Anne-Charlotte Nicoud. anne-charlotte.nicoud@mines-paristech.fr; LAB : Cyril Chauvel. cyril.chauvel@lab-france.com; IFSTTAR: Hasnaâ Aniss. hasnaa.aniss@ifsttar.fr; EURECOM:
- Others (specify):
Applications and equipment
Applications tested
Road Safety
- Road Works Warning
- Car Breakdown Warning
- Post-crash Warning
- Weather Warning (slippery road, heavy rain, fog)
- Emergency Electronic Brake Lights (*)
- Approaching Vehicle (*)
- Obstacle Warning (the driver receives a warning)
- Obstacle Notification (the driver sends a warning)
- Third party on collision course warning (*)
- Pedestrian Warning
- Animal/other Warning
We use DRIVEC2X terminology. Nevertheless, please note that we make a distinction between an increase in awareness and a warning, depending on the time remaining before to potential crash at the time of information delivery (> 5s, < 5s). Consequently, most of the earlier DRIVEC2X use cases exist in two Score@f versions - e.g. : DRIVEC2X Obstacle Warning covers both the Score@f Obstacle Warning and the Score@f Obstacle Awarness Increase.
(*) tested on driving simulator only - no technical development
Traffic flow and management
- Speed Limit
Others (comfort & mobility)
- In-Vehicle notification (VMS: Variable Messages Sign)
Two more apps were not tested by our users but shown at the Forum des Acteurs, 2013.
- Point of Interest notification
- Dynamic Car pooling
Vehicle
The test fleet included 28 vehicles: 16 private cars equipped with a C-ITS kit for the Yvelines Naturalistic tests (Renault cars), 11 pool vehicles for the Yvelines Controlled FOT (PSA and Citroen cars) and 1 pool vehicle for the Isère Controlled FOT (Renault car).
Beside the test fleet, fifteen more vehicles, provided by Renault, PSA-Peugeot-Citroen and IFSTTAR were also equipped for technical testing. These vehicles were also used to play the role of comparse vehicles in the controlled and naturalistic tests (e.g.: act as a stopped car for CBW study)
Equipment carried by test users
Infrastructure
- Test Site#1: A10.
Four road-side units (RSU) placed on a 3-ways motorway axis of 13 kilometers, situated at the North of Orléans, between two exits.
- Test Site #2: Yvelines.
Ten road-side units (RSU) placed on a perimeter that included: a small portion motorway (N12), a tunnel (quadriplex) and a rural-road of about ten kilometers (RD91, "la Minière"). The rural road is one of access to Technocentre Renault (spontaneous path of the volunteers for Naturalistic Tests) and has the advantage of providing a fairly rough topography (tight corners).
- Test Site #3: Isère.
Four road-side units (RSU) located in the area of St Egrève to Noyarey, passing through the town of Voreppe, to cover a variety of road types: motorway, departmental, communal. The environment is rich, both for its environmental diversity (curves, roadwork) and for its type of traffic (from fluid to extremely dense). The site is still open.
Test equipment
Methodology
Pre-simulation / Piloting of the FOT
Three forms of piloting can be considered within Score@f project.
- The first simulator study (Study #1) had a clear piloting function. It aimed to pre-test a panel of use cases in order 1. to remove those which were poorly perceived by the drivers and 2. to refine the HMIs.
- The first study on the open road (Study #2) can also be seen as a form of piloting. The aim was still to refine HMI and use cases triggering conditions.
- The standard piloting was finally held on the first half of the Study #5. One month FOT has been devoted to methodological development (March-April) before the actual trials month (June-July).
Method for the baseline
Techniques for measurement and data collection
Protocol design
As C-ITS were tested by drivers for the first time, a constructive approach was preferred. Six small-scale experiments were planned and realized between 2012 and 2013, including various facilities : open roads and driving simulator.
A range of use cases was pretested by 12 participants on a driving simulator on Study#1 (March, 2012). Study#2 was dedicated to a trial on open-road (pre-piloting): 34 participants made an half-hour run on highway context (A10, North of Orleans, August-September, 2012). A larger study on driving simulator, aiming to retest use cases dedicated to highway context in a protected environment, took then place (Study #3, November, 2013). Finally, the three last studies were FOTs: Study#4 was a the Naturalistic FOT (Yvelines, 9 participants, June 11-July 10 2013) preceded by its piloting (Yvelines, 6 participants, March-April 2013); Study#5 consisted with Controlled Tests - 1 hour-run- aiming to complete the data collection with a systematic approach (Yvelines, 23 participants, June 2013); and Study#6 replicated Controlled Tests in another environment (50 minutes-runs in Isère, 51 participants, from June to September 2013).
Testing environments
Open roads studies are seen as the main testing environment; nevertheless these studies have been completed by simulator studies which were useful: 1. to prepare FOTs and 2. to make experienced use cases that cannot be tested on open-road (hazard-free environment).
Open road studies did mix a naturalistic approach and a controlled approach. The first approach was considered the best, due to its realistic character, but the second approach is viewed as necessary to facilitate the collection of comparable data (meaningful baseline and treatment conditions). The controlled approach was also important to gather more data on use cases that refer to infrequent events in real life (e.g. post crash warning)
Recruitment goals and methods
Six samples of participants have been recruited, for a total of about a hundred drivers: 12 drivers for Study#1, 34 drivers for Study#2, 36 drivers for Study#3; 17 drivers for Study#4, 23 drivers for Study#5 and 51 drivers for Study#6.
The table below sums up the main recruitment criteria. For the Naturalistic test, note that we select drivers whose daily home-to-workplace trips were similar and performed at same hours, this in order 1. to enhance the probability that they will cross one another and 2. to facilitate the organization of fake event thanks to comparse vehicle.
Methods for the liaison with the drivers during the FOT execution
Methods for data analysis, evaluation, synthesis and conclusions
Sources of information
http://www.pole-moveo.org/pdf-projets-das/Scoref-A.pdf
http://www.fot-net.eu/download/Wiki/scoref_presentation_en_022011.pdf Presentation of the project
