Back to FESTA Handbook main page: FESTA handbook

7. Guidelines for Data Acquisition

This section aims to provide guidelines and recommendations for how to handle data in an FOT study. Data acquisition, data storage, and data analysis tools will be covered here.

Figure 7.1: Data structuring

Please refer to Figure 7.1 for an overview of a data handling structure for an FOT, and for the naming conventions used in this document. The example data structure above includes data from an electronic data acquisition system, as well as collected subjective data. The Data Acquisition Unit (on the right) comprises sensor systems requiring raw data decoding. The raw data may then be pre-processed, in this case by low-level data processing such as simple filtering or calculation of directly derived results. Both raw data and pre-processed data (derived from raw data) are stored in the same format locally and can be kept locally, until moved from the Data Acquisition System (DAS) to the main storage, and used for analysis.

Acquisition of subjective data may also be performed. Subjective data is also considered as acquired from a sensor. This data can similarly be subject to manual or automatic decoding, database storage (pre-processed or not), and subsequent use in performance indicator calculations.

It is important to create documentations of developed tools and processes in order to enable their usega after the project end (e.g. usage in follow-up project) and also to provide detailed test protocol with all relevant test scenarios and potential risks.

7.1 The measures and sensors tables

FESTA recommends use of the FESTA matrix, a spread sheet in Excel format containing three tables: “performance indicators”, “Measures”, and “Sensors”, that may in a later stage be utilised to create a relational database. The sensor table should be used in preparation of, during, and after an FOT project, in conjunction with the other two related tables (see Appendix 1 in Deliverable D2.2). Properly handled and thoroughly implemented the tables are valuable tools for data structuring and for data requirements specifications, and for identification of connections between sensors, measures and performance indicators.

The FESTA matrix is intended as an aid and gives examples of commonly used sensors, together with specifications. It is not exhaustive and future users are encouraged to modify, expand, or limit the matrix to suit the particular FOT.

7.2 Data acquisition

Methods of data acquisition in FOTs include methods to collect background data, digitally acquire data from sensors, and subjective data (such as data acquired from questionnaires). In addition, data in the form of manually or automatically transcribed data and reductions of collected data is also considered sensor acquired data (but with a manual sensor – the analyst). In FESTA all the data sources mentioned above are considered sensors. Subsequently can all data be acquired, stored, and processed in a generalised way.

All of these different data types are used to support the hypotheses defined for the specific ''FOT . The data to be collected should be defined and based on research questions and hypotheses.

7.2.1 Background data acquisition

The background data about the driver is crucial and needs to be collected integrated in the driver interaction procedure. Due to privacy issues different parts of the background data may or may not be suitable for storage in a database, or be used in statistical and other forms of analyses.

Data could be gathered by interviews and/or questionnaires, by different tests, or by specific instruments. The driver background information should be considered as acquired from a sensor, and preferably be added into the database and to the sensor matrix though subject to privacy issues.

7.2.2 In-vehicle data acquisition

An in-vehicle Data Acquisition System (DAS) is needed in FOTs where the focus is to study in-vehicle systems by collecting data from the systems in the vehicle and also from sensors mounted on the vehicle such as video cameras. A suitable DAS can differ from study to study and a specific solution cannot be recommended for all types of Deliverable D2.2 for a list of different DAS solutions.

The guidelines and requirements in this document are based on experiences from FOTs using some kind of in-vehicle data acquisition.

7.2.3 Nomadic devices

A Nomadic Device (ND) or an aftermarket device could be either part of the function/system under test, or it could be part of the data acquisition system, acquiring specific FOT data.

Nomadic Devices can also be used as data storage tools as they are easy to install and use on different kind of vehicles. If the vehicle has a dedicated gateway for ND, this option can be used for capture of further vehicle related data.

Using the local wireless connections, the storage capacity of ND could be extended with large capacity hard disks. A possible drawback of a ND, when used as a DAS in itself, is that test subjects must remember to bring the ND to the vehicle every time he/she uses the vehicle.

7.2.4 Subjective data acquisition

As explained before, subjective data are also considered as “sensor” data in the scope of the FOT methodology. All subjective data should therefore be stored and handled logically as if it were collected from a “real” sensor. Subjective data may include data acquired from the test participants in different ways. Results from interviews, questionnaires and focus groups are typical types of subjective data. In evaluations of cooperative systems, consideration should be given to collecting subjective data from other actors such as control room operations.

The result from the subjective data acquisition should preferably be stored in an electronic format. Electronic compilation of the questionnaire may be considered to reduce the overall manual work and cost, maybe by using web based tools.

For subjective data to be stored, the following related information is required:

Date and time (hh:mm) of test start
Date and time (hh:mm) of test end
Subject ID code
If present, reference to objective data (file name, location)

It is important to think about processes and tools for gathering questionnaires at the beginning of the project.

e.g. provide smart phone apps to fill out questionnaires
e.g. keep drivers in the analysis loop

7.2.5 Real time observation

In this context, real time observation data is data collected by an observer that directly or indirectly (in real-time or afterwards – for example on video) is observing the drivers and systems to be evaluated. The data acquisition process is usually relatively manual but the results should be transferred to digital format and uploaded to the FOT database for further analysis.

Real time observation data help provide a more detailed picture of a driver’s behaviour, as well as verifying the information gathered by other instruments. As the overall purpose of an FOT is to collect information on as natural driving as possible, the real time observation is almost exclusively done through video cameras hidden in the vehicle compartment, as an observer physically in the car will impose the risk of the driver not acting the way he or she would otherwise. Direct real time observations must therefore be carried out with great care and as unobtrusively as possible, or avoided completely.

7.2.6 Additional data sources in Cooperative systems

The Cooperative Systems architecture implies the possibility of additional data sources. Specifically RSUs connected to proper sensors may provide traffic information and environment data. RSUs connected to Traffic light controller are able to provide traffic light phases and Intelligent traffic control centre dispatch traffic information and alternative routes. It is important that all data records contain a time stamp synchronized to GNSS clock.

7.2.7 Acquisition of infrastructure data and other services

7.2.7.1 General aspects

The infrastructure can be equipped with sensors to detect e.g. traffic or weather conditions. Data from such systems can be collected in raw format or in an aggregated form. If data is collected both on the vehicles and on the infrastructure separately, it is necessary to synchronise the two sets of data. It is recommended that GPS time is used as the synchronisation source.

7.2.7.2 Infrastructure

It is in many countries required to contact local road authorities before the installation of equipment close to a road. Working close to or on roads may (depending on country) require special training or licence. In some countries it is even required to use a special company or local road authorities for any installation work close to or on roads.

7.2.7.3 Services

When using such sources it is recommended for traceability (during and after the project ends) to record information about for example version of service, update rates and resolution/precision of the information they have during the duration of the study. It is also recommended to invite the service providers for discussions and possibly partnership in the FOT .

7.3 Specific sensors

7.3.1 In-vehicle video

Most state-of-the-art FOT experts consider video as a very important data source for the identification of driver behaviour and reactions, as well as for the process of analysis to understand the underlying context with regards to the surrounding. When a certain situation or event has been identified for evaluation of a particular system, the video provides the analyst with information about the context of both driver behavioural aspects and the interaction with the environment (if external video is used).

Video can be used in mainly two ways:

Driver monitoring: Firstly the driver eye/head movements in relation to the vehicle/environment/context, and secondly the driver interaction with the vehicle and other driver actions (pedal, gear shift, steering wheel handling, mobile phone, eating, etc.)
Environment/contextual monitoring: Helps understanding the driving contexts by collecting information about the surrounding traffic.

For a specific FOT the number of cameras needed, the needed resolution, the views captured by the cameras, etc., should be defined by what is needed to address the hypotheses.

To find the correct requirements thorough investigation is needed. Evaluate the results using calculations of predicted storage needs, as well as evaluation of video image quality and set requirements accordingly. In this evaluation experts from both study design/analysis planning as well as the analysis team should be included.

Generally the following parameters affect video quality, and need to be considered: picture resolution, frame rate, colour settings, “regions of interest”, bit rate strategy, bit rate limitations, “quality” strategy settings, and other video compression method related settings.

For more exhaustive methods for defining the video requirements, for information on potential camera related quality problems involving interlacing, and further information on video compression, see section 3.2.1 in Deliverable Deliverable D2.2.

7.3.1.1 Recommendations

Make a thorough analysis of video acquisition requirements. Set requirements necessary for each individual view, to possibly achieve a first limitation of video data. Choose a well-tested hardware video frame grabber/compression solution, and choose compression suitable for the FOT . The [MPEG-4] part 2 and part 10 (H.264) compression feature sets have been used successfully in other FOTs, as well as MJPEG compression.

Video data analysis is normally complex. For this reason, it is recommended to limit video analysis to minimum and try as far as possible to use automated processes.

7.3.2 Internal vehicle bus data

Most modern vehicle manufacturers features one or several internal vehicle networks such as CAN, LIN, MOST or FlexRay. An internal network may carry large amounts of useful information for the FOT. However, there are several concerns with accessing and ascertaining quality on data from an internal vehicle bus.

7.3.2.1 Accessing the vehicle bus

Accessing information from an internal vehicle bus can be highly complex and even void warranty if it is done without the OEM’s permission and supervision.

7.3.2.2 OEM cooperation

A description of the entire vehicle network will often contain proprietary information, and may reveal detailed information on specific functions and the vehicle system architecture. Thus, non-disclosure agreements are typically required and can be hard to attain.

An option to using the entire vehicle bus description is that the OEM only provides the description of a selection of signals which enable access to the most important data. Still, however limited the access to the vehicle bus is, there may still be proprietary issues.

By using a bus gateway, the OEM will be able to extract data from the bus without providing any information about how the bus actually works. The gateway is programmed by the OEM to read certain information, decode it and then pass it on to the FOT logger equipment. An illustration of the process is shown in Figure 7.2.

Figure 7.2: Illustration of process for securing proprietary vehicle-bus data using a gateway.

When physically connecting to the vehicle bus, it is important to follow applicable standards of the bus in order to prevent interference that may reduce network functionality (and thus warranty). It should also be stressed that every bus implemented by an OEM might not necessarily fulfil the standards in every detail.

7.3.2.3 Sensor specifications and details

When acquiring sensor data from a vehicle bus, the information is passed through several stages before it can be read from the bus (see section 3.2.2 in Deliverable D2.2). These stages are likely to affect the signal value both in terms of amplitude and frequency and thus need to be closely observed.

To ascertain that qualitative measures are attained, plenty of contribution is required from the OEM. A thorough description of a vehicle bus and its ECUs will in many cases require involvement by subcontractors as well. A list of required details for successful data acquisition from the frequently used CAN bus is provided in section 3.2.2 in Deliverable D2.2.

7.3.3 Automatic in-vehicle driver monitoring

7.3.3.1 Head/eye tracker

Two of the main issues for many systems are visual distraction and the effects the system has on the driver attending to traffic versus to the system. By using eye or head trackers in the vehicles, continuous data of some driver state/attention measures can be obtained. The problem with head and eye trackers is mainly the risk of significant data dropouts due to limitations in driver head and gaze tracking.

State-of-the-art head/eye-tracker technology today is relatively expensive, but the benefit of using such a system should not be underestimated. It is recommended that head or eye tracking systems are employed in FOTs where driver state is an issue.

Using an unobtrusive system is a requirement for head/eye-tracking systems for FOTs. The driver should not have to initiate the tracking system or wear any device; the system should be as inconspicuous as possible to the driver. Also, the system should not require any manual calibration in the field.

FOTs using eye-tracking devices have continued to find this a challenge.

7.3.3.2 Other

It is strongly recommended that adding sensors that the driver has to put on and wear should be avoided to assure as natural driver behaviour as possible. (For further examples see section 3.2.3 in Deliverable D2.2.)

7.3.4 Extra analogue/digital data sources

Access to certain information that is not available on vehicle bus systems is often needed. An analogue/digital I/O device for data acquisition is thus required. Also, anti-aliasing issues need to be addressed. The requirements for the different kinds of extra analogue and digital sensing need to be defined in the study design and will not be covered here.

7.3.5 Radar and other non-video environment sensing

7.3.5.1 Sensors already integrated (by OEM/Road Administration)

If a required environment sensor is integrated into the vehicle or the infrastructure, great effort should be spent on trying to add the sensor data to the used data acquisition system. In this integration several issues have to be considered (for details see section 3.2.5 in Deliverable D2.2):

OEM allowing/disallowing access to data.
system interfaces: Some low level sensing information (e.g. object tracks from radar) may require special interfaces to be implemented both in hardware and software;
system comparability: If the studied vehicles have different OEM integrated systems, they will provide different quality of data as well as different resolution, range, field of view etc.

7.3.5.2 Add-on environment sensing

If additional sensing needs arise, please consider the recommended process for adding sensing:

Use the sensor matrix for sensing needs and identification;
Requirements on the extra sensing need must come from hypotheses and performance indicator requirements;
Additional considerations: Does the extra sensing require additional interfaces? Does it require information from vehicle buses? How can the sensor be integrated without significant effort? Does it require repeated calibration?

General guidelines for specifications are difficult to define. However, for example radar and other object tracking sensing systems, required field of view, radial and angular resolution/precision is important to define based on the hypothesis. For further suggestions, see section 3.2.5 in Deliverable D2.2.

7.3.6 GPS

A GPS device can provide a GPS Time reference time stamp and a difference between the GPS time and UTC. It is highly recommended that this is used for synchronisation within a data acquisition unit as well as between systems (in-vehicle, ND, infrastructure and services). Information about the present local time zone can be useful for subsequent analyses and synchronisation with non-UTC devices.

Multipath propagation of the GPS signal is a dominant source of error in GPS positioning, since they depend on local reflection geometry near each receiver antenna. In most cases these can be corrected to a large extent with DPGS solutions. The errors depend on time of day, and satellite positioning (in zenith or low orbits) as well as other atmospheric disturbances.

7.3.7 Audio and driver annotation

Continuous audio recording is potentially a significant privacy issue and is not recommended. In state-of-the-art FOTs drivers have had access to a comment button on the dashboard, which, when pressed, would start recording any verbal comments during a pre-set number of seconds (usually around 20-60 seconds). Use of the button could be encouraged if the driver feels that it is warranted or at agreed (critical on non-critical) events. Be sure to inform the drivers consistently about the annotation possibility and provide some simple guidelines on how to offer the comment.

7.3.8 System function/status

A system under evaluation, such as an LDW, ACC or FCW, needs to be continuously monitored to ensure that it is operating properly. The system status signal will thus form a measure to be recorded in the data acquisition system. The status signal will depend on the specific system and needs to be provided by the system manufacturer or vehicle manufacturer, thus requiring strong collaboration with the actual provider.

7.3.9 Vehicle metadata

Information about the studied vehicle is important for analysis and study design. The recommendation is that for each type of study, systems, functions and specifications that may act as confounding parameters in a specific analysis should be stored. During analyses, these parameters may have confounding effects if an inhomogeneous test fleet is used. Examples:

FOTs with focus on safety: Information should be stored about integrated systems that may contribute to driver distraction,
FOTs with focus on environmental issues: A more powerful engine, automatic gear shift or four-wheel-drive, are most likely to be confounding parameters in an environmental analysis.
Experts have also stressed through the seminars, that wider metadata - from weather conditions to prevailing economic conditions is also essential for interpreting data post-study.

7.3.10 Coding/classification/transcription

As part of the data reduction and analysis process, described elsewhere in this handbook, sections of time will often be labelled with classifications according to a coding scheme or syntax. Depending on the study, sections of time can be given categories such as “crash”, “near-crash”, “incident”, “Curve speed warning”, “lane change”, “crash avoidance by steering”, etc. When classifications are made they are often saved and thus become a new data source which is added to the database. For example, an index indicating all instances of lane changes in the dataset can be created and saved. Regardless if the classification of data is performed by a human analyst transcribing video or by an algorithm applying kinematic trigger values to the data, this process of classification should be seen as a type of sensor providing a new data source. Thus, it is comparable to other types of off-line performance indicator calculations (see Figure 7.1). It is recommended to plan that these new data sources or measures will be created during the data reduction phase after the data has been uploaded to the FOT database.

7.3.11 Geographical Information System (GIS)

One of the lessons learned in state-of-the-art FOTs in the US is that geographical information, such as road curvature, roadside embankments and other on-and off-road information has been underestimated as a valuable source of data in the analysis. Contextual indicators of events and situations identified in the analysis provided added insight into both behavioural aspects and how the infrastructure influences system performance. (See section 3.2.12 in Deliverable D2.2 for more information.)

GIS derived information about the current road (road type, speed limit, rural/urban, banking, curve radii, etc.) could be used directly in the vehicles as separate measures. By doing this on-line, the necessary post-processing is reduced, but requires additional software and possibly hardware. One advantage of performing this on-line can be that the absolute position (e.g. GPS) does not have to be stored, thus reducing the some privacy concerns. An advantage of using commercial navigation software as part of the on-line map-matching and information extraction can be that there is no need for potential in-project development of map-matching algorithms. This technique has been used in some state-of-the-art FOTs. It is important to validate that map-matching and other GIS data extraction is done in a proper way.

7.3.12 Communication unit

A communication unit is a device, which provides vehicle to vehicle and vehicle to infrastructure connections in Cooperative Systems FOTs. It also has to be considered as a source of measures for all communication-based aspects. Typical measures include received and sent messages, network traffic congestion, number of connected nodes etc.

7.3.13 Application Unit

The Application Unit is a device to run developed functions. Typically this is an x86-based computer in ruggedized configuration. If an application unit is deployed in an FOT it also provides a variety of measures. All developed functions should be providing measures of their state and of key variables used. The underlying Facility layer (e.g. vehicle access, position access, local dynamic map) is also a valuable source of measures.

It has to be decided, if it is feasible to also run the data acquisition system directly on the Application Unit, which simplifies the access to all key variables. On a downside the DAS might consume too much processing power or drive capacity, thus interfering with the functions.

7.4 Mechanical requirements

The following mechanical guidelines and requirements are primarily applicable for FOTs with in-vehicle Data Acquisition systems.

7.4.1 Size and weight

The system should preferably have negligible effect on the driver’s use of the vehicle – including limitations in trunk space.

7.4.2 Connectors and interfaces

State-of-the-art FOTs state that from experience as much as 80 % of the DAS hardware problems can be deduced to physical connector issues (to the DAS and to peripherals). It is recommended that connectors with some locking between connector genders are used. Cable pull-relief should be used when possible.

7.4.3 DAS mechanical cover and ease of access

It is recommended that a layman without tools is able to find and have visual access to any indicator LEDs on the DAS. Also, having the possibility to connect interface devices without having to remove covers is preferable.

7.4.4 Crashworthiness and vibration resistance

For all FOTs the minimum requirements for ruggedness is that the entire system should operate under the normal driving conditions for the specific FOT . including the harsher situations of normal driving.

7.4.5 DAS environmental requirements

Environmental requirements for the DAS mainly concerns temperature. If the DAS is placed in a shielded location, the need for water resistance may be negligible, although the DAS internal parts should be able to withstand reasonable levels of condensation. If applicable, it is recommended that a simple dust/particle filter is placed by the main air intake of the DAS. Important considerations include sufficient cooling of the system due to internal heat generation and for ambient temperature. The cooling needed depends on the processor used and the workload. If the FOT is using video compression done in software, the need for processing power may be drastically higher than if only simple signals are recorded. High processor load generates significant levels of heat. The need for forced ventilation is depending on the DAS mounting position.

Too high and too low temperatures (both static and transient) do affect the DAS. Components with moving parts need special attention. Hard drives are some of the most sensitive components. Automotive grade hard drives are available, although somewhat more expensive than normal consumer hard drives, and with limited storage capacity. Flash memory cards or solid state drives (SSD) are available in increasing capacities and are clear alternatives for operating system hard drives and for storage in lower data volume FOTs.

7.5 Electrical requirements

7.5.1 Power management

The main requirement for the DAS setup is that the installation should never affect or impair normal vehicle function. This requirement should be enforced in all environmental conditions and is one of the most critical issues for drivers to accept equipment to be installed in the vehicles. If FOT installations are rumoured to impede on the trustfulness on vehicle operation, few people will volunteer in any subsequent study, and the study at hand as well as subsequent studies may be compromised.

The entire DAS installation may not draw power so that the vehicle battery charge falls below the level of being able to start the vehicle. Care should be taken so that the system does not draw power (or only minimum power) if the ignition has been turned on but the engine did not start, or if the engine stops without the ignition key being removed. State machine evaluation should be applied.

7.5.2 Interference with in-vehicle equipment

In all in-vehicle installations the aim should be to minimise or manage without the use of AC powered devices. The DC/AC and DC/DC converters are sources of electromagnetic noise and may affect both standard in-vehicle equipment (such as the FM radio), and the FOT installation itself.

Attaching any equipment to the in-vehicle bus systems has to be done very carefully. Transmitting data on vehicle buses should in most cases not be needed or not done at all in an FOT implementation. Failure to adhere to this might be dangerous and result in vehicle operational malfunction that may result in significant cost, injury or death, or produce other very unwanted results. Even adding only listening/eavesdropping devices should be approved by the vehicle manufacturer before being implemented.

7.5.3 Laws and regulations

Depending on the FOT study at hand different regulations will apply. In some cases CE certification of the FOT equipment may be necessary, and each project must verify what is applicable to the specific study. If the vehicles are to be driven in non-EU countries the specific regulations for each region should be verified.

For wireless communication and for some sensing systems there are regulatory restrictions on transmitting electromagnetic radiation. A few example of sensing systems that have direct regulatory restrictions are LIDAR and RADAR. The restrictions may be based on electronic interference or harm. This has to be taken into account for each individual FOT sensor setup. Still, for each instrument and jurisdiction, care should be taken to investigate the applicable regulations.

Several countries have regulations for equipment employing radar or laser technology on public roads, as these can affect effectiveness of for example authority speed surveillance instruments.

7.6 DAS data storage

7.6.1 Storage capacity estimation

The main aim of the storage capacity estimation is to guarantee the availability of free space for recording the vehicle data. Storage capacity depends on the following factors: number of recorded signals, sample rates of the recorded signals, sample size of the recorded signals, data collection method, driving hours, data size reduction (filtering/compression), and data deletion procedure.

Ideally, the sample rate for each signal should be the lowest possible able to guarantee no information, relevant for answering the FOT hypotheses, is lost in the sampling process. Some sample values are reported in the FESTA Deliverable D2.1 for CAN and sensor data.

Using a dynamic sample rate and a high sample-rate buffer would make possible adapting the resolution of the recorded data depending on the event. The drawbacks of using dynamic sample rate are that the recording system complexity (and probability of faults) increases, more post-processing on the data will be necessary to handle the different sample rates. Database design and search becomes more complicated.

The data collection method can be continuous, or limited to specific events of interest, for example time intervals in which the lateral acceleration is above a certain threshold or in which the vehicles enters a curve with speed above a certain threshold (see section 3.5.1 in Deliverable D2.2). In FOTs where only triggered storage is used and where the driver subjectively triggers in some way, it is important that past data also is recorded by using for example a ring buffer (buffering one to five minutes in the past). The level of pre-trig time will differ between projects, and should be defined based on the hypotheses for each study.

Note that events such as occurrences of safety systems warnings can be considered. The probability of such events should be part of the knowledge of the FOT designer and should be used for the estimation of the storage capacity.

Driving hours depend on the nature of the driver and the vehicle. See section 3.5.1 in Deliverable D2.2 for examples.

Compression algorithms can help to reduce the data size. A lossless compression (such as zip) can be used for CAN and sensor data whereas a lossy compression (such as MPEG-4 and MP3) is normally acceptable for voice and video, respectively. The drawback of compression is that the complexity of the system increases and new possible sources of error and malfunctioning are introduced.

A safe data deletion procedure implies that no data is deleted in the vehicle until a copy of the data has been backed-up, verified, and stored in a safe place.

Figure 7.3: Factors influencing data size

Please refer to section 3.5.1 in Deliverable D2.2 for storage capacity estimation equations, as well as data size estimation examples.

Storage capacity may be depleted and the intervals for retrieval and uploading may present some variability. These factors should be taken into account by guaranteeing enough tolerance on the final storage size. Since no space to record data would result in data loss a 20 % to 50 % on storage size tolerance is recommended.

In some studies where the levels of allowed data loss are very small it is recommended to use direct on vehicle data backups. This can be done for example by using several storage media with a data mirroring solution (e.g. RAID).

When triggered data acquisition has been chosen as the data collection method, great care has to be taken to define the trigger definitions. Even if triggered logging is used for the evaluation of effects, most studies will require baseline data. It should be possible configure and acquire baseline data, using the same DAS, also for these cases.

FOT activities that use any type of triggered data acquisition and have high data-rate data sources will need significant amounts of main memory to handle the necessary ring-buffer for pre-triggering storage. Moreover, estimations of needed storage space and data retrieval/upload frequency are affected (See section 3.5.2 in Deliverable D2.2.)

7.6.2 Data retrieval/uploading procedure

Data retrieval/uploading procedures are needed to make sure that all collected data is backed-up and stored in a safe place in order to minimise data loss. The aim is to prevent data loss, verify data completeness, and to prevent data storage waste (caused by double storage).

Figure 7.4: Data retrieval /uploading steps

7.6.2.1 Data transfer

Data transfer is aimed to assure that a copy of the data collected in the vehicle is stored in a safe location. Data back-up is aimed to prevent data loss by having a multiple set of the data stored in different safe places. Data verification is aimed to assure that no data was lost during data transfer and data back-up. Vehicle data deletion is aimed to ensure that storage space is newly available in the vehicle once the data has been safely transferred and backed-up.

Depending on the support used to record the data in the vehicle and the data size, different data transfer modes can be implemented. For a list of potential modes, transfer rates, and ranges please see section 3.5.4 in Deliverable D2.2.

Generally data transfer poses two main problems: It may be time-consuming and data can be lost during the transfer. The following table presents an assessment of different transfer modes.

Table 7-1: Pros and cons for different data retrieval/uploading modes

7.6.2.2 Data back-up

Data should be backed-up and stored in a safe place as soon as it is available. Ideally, the backed-up data and the main copy of the data should be in two different safe places. In some studies where the levels of allowed data loss are very small, it is recommended to use direct on vehicle data backups. This can for example be implemented by using several storage medias with a data mirroring solution (e.g. RAID).

7.6.2.3 Data verification

Due to the potentially huge amounts of data handled, data verification is important since the probability of errors during the copying process is high.

7.6.2.4 Vehicle data deletion

Data from the vehicle should be deleted only once the data has been backed-up and verified.

7.6.2.5 Data loss

Experience from previous FOTs tells that data loss at the retrieval/upload stage is common even if it could be almost totally avoided with a robust and well-tested procedure. To prevent data loss during the data upload/retrieval procedure it is important to verify that data is consistent before deleting it from the vehicle. In case data is picked up and the data is not consistent, the vehicle data logger should be checked as soon as possible. Monitor the state of the data logger so that any issues can be recognised and fixed as soon as possible.

7.7 System configuration

Although DAS system configuration needs can be different, one basic requirement that apply to all DASs is that it should be possible to find and configure a specific DAS after the study has been finished.

7.7.1 DAS inventory management

A system for basic inventory management is recommended for FOTs with more than a few vehicles in use. For such a system to be efficient, sensors, DAS units, vehicles and all other equipment need to be included, as well as relevant supporting procedures developed. For any one point in time it should be possible to deduce the exact hardware and software configuration of a particular installation.

7.7.2 Configuration tools and traceability

In addition to an inventory management system, it is appropriate to employ a system (and supporting management procedures) for configuration management. Software versions and additional information such as (but not limited to) software configuration files, start-up scripts, device calibration files, and other file based data needed for proper operation of the DAS should be stored together with information about the present DAS hardware configuration.

Similarly to above, it is important that traceability of software configuration changes is maintained. In this way, the exact software configuration of a particular installation at any particular time can be reproduced and reviewed.

7.7.3 Switching between configurations

For some FOTs switching between configurations in each vehicle may be necessary. In addition to having to change DriverID when a new subject is introduced to a vehicle there are a few other situations where it may be important to know who was driving or what was done with the vehicle. If this is not separated from the subjects, these trips may be classified erroneously. Examples of situations to consider are: the vehicle driven back for overhaul/maintenance, validation testing just prior to vehicle delivery, and other engineering testing.

If possible, the use of remote desktop tools over wireless (e.g. WLAN or 3G) or wired networking (Ethernet) is recommended for remote administration.

7.8 Acquisition of data

When controlling the power supply to the DAS the start-up and shutdown speeds must be optimised to reduce loss of data. Loss of data can occur both during hardware initiation when no software is started and during hardware termination when no software is able to trigger on a vehicle restart.

7.8.1 Start-up

Normally the data acquisition will start as the vehicle ignition is turned on. In order to minimise the data lost during the start-up procedure, the hardware and software must load and initiate as quick as possible. The start-up time (or the duration where data is lost) should be well monitored and documented, preferably as a property associated with each recorded trip (since it might differ with temperature etc.). Start-up of the DAS hardware shall not be done if the voltage is too low.

7.8.2 Acquisition of data

The DAS hardware should be kept powered on and running during the entire trip. To ensure that the host power system is not overexerted, the power management unit must continuously monitor the power supply and initiate shutdown if a permanent voltage fall is detected. The system must not shut down on temporary variations such as the drop during engine crank. For such circumstances, an energy reserve such as a battery may be required.

7.8.3 Shutdown

When the DAS recording has stopped, the DAS should be kept running for a short time (typically a few minutes) in case the vehicle is started again. Otherwise the DAS hardware should shut down itself as fast as possible. The power management unit should then, after a short interval, cut the power supply to the DAS, regardless of it is properly shut down or not.

7.9 Synchronisation

FOTs are considered to be used for logging of data and not as part of a hardware-in-the-loop component (for example prototype system development). This means that data from the different data sources do not necessarily have to be available for storage close to the real-world event, as long as they are individually time stamped for off-line recreation of the time-line.

7.9.1 Time stamping versus real world event

Latency is the time between when the actual real world event takes place and when the data from each respective sensor is time stamped in the logger. In most FOTs there is a need to specify the requirements for allowed levels of latency, based on the hypothesis. It is recommended to explicitly evaluate what the latency is for each data source (and for some sources individual measures) and compare this with the defined requirements (based on hypothesis; needs to be done as part of the hypothesis and performance indicator generation). If latency has been measured properly and there is limited jitter, the time stamp time can be corrected by offsetting with the latency.

Note that for data sources that are not controlled by the DAS implementers, such as vehicle CAN, it may be even more difficult to obtain the necessary information for the latency from real world event until time stamping.

7.9.2 Integrated sensing synchronisation

Depending on the methods of analysis and the implementation in the database, the needed level of synchronisation as well as the importance of measuring latency between different integrated data sources in a vehicle will differ. In Table 7-2, issues and methods for calculating the latency for some in-vehicle data sources are shown.

Table 7-2: Methods and issues in calculating latency for in-vehicle data sources.

7.9.3 Synchronisation with nomadic devices

The recommended method to realise synchronisation between the different sources of data is to use GPS UTC time. Thus, all acquired data should be associated with a time stamp that is represented as absolute GPS derived time. It is recommended to use nomadic devices that have easy interface with GPS, so that the absolute time information is available.

7.9.4 Synchronisation of infrastructure systems

Also in the case of infrastructure, the UTC time derived from GPS if useful. Another possibility is to use an on-line time synchronisation service like NTP/SNTP. In the case of a traffic data and safety related FOT , data may have to be stored as raw data, accurately synchronised in time, to allow the reconstruction of the scenario in the following data analysis phases.

7.9.5 Synchronisation of cooperative systems

• Synchronisation of systems with communication between vehicles can also be realised without a central infrastructure. Also here, it is recommended to use the common reference time provided by GPS. For central ITS stations involved in cooperative systems FOTs and additional external sources it is advised to use a network time synchronization protocol, such as NTP.

7.9.6 Synchronisation with interviews and other subjective sensors

In many cases it is enough that the interviewers write date and time (hours and minutes) of the interview or questionnaire. If the subject is requested to indicate and/or comment events during the driving (for example by pushing a button), this should be time stamped when logged if possible, to enable synchronisation of the event/comment with other data. The accuracy needed is in most cases less than 5 seconds. For post-hoc structured comments or questionnaires on video or events, it is important to define a process of linking the events to absolute time.

7.10 DAS status and malfunction management

To simplify laymen feedback on system status to the responsible technicians at times of system problems, LEDs or similarly externally viewable information about the system status can help and are recommended.

7.10.1 System status uploads

In order for the people responsible for DAS and sensing in the project to be able to continuously monitor the status of the test vehicles while on the road, a remote (wireless) transfer of the system and sensing status is preferred. (See section 7.14.1 for details.)

7.10.2 Malfunction management

A process for identification of problems in the vehicle, contacting the driver and exchanging sub-systems or the entire vehicle should be developed.

7.10.3 Spare system management

For quick and efficient problem solving it is recommended to keep spare parts for all components pertaining to the DAS. It has been suggested that a 10 % extra number of spare parts should be kept within the FOT , and that this number be added specifically when estimating the total project cost. All spare parts are to be managed by the inventory management system. Testing and calibration of spare parts should preferably be planned and performed within the process. Supporting management procedures need to be developed as well. At least one fully equipped DAS system should be kept “on the shelf”; prepared and calibrated for immediate use.

7.11 System installation

7.11.1 Installation procedures

Before initiating the installation procedures, an installation specification document shall be prepared. The installation specification must in detail describe how each component of the system shall be installed. Specifically, the installation specification shall provide solutions to the following topics:

Mounting: positioning, means of attachment, accessibility, safety and security;
Cabling: dimensions, shielding, drawing, mounting, tolerance and labelling;
Connectors: soldering/pressing, robustness, impedance and labelling to avoid mix up;
Power supply: consumption, fuse, voltage, source and switching;
Environmental endurance: effects on electromagnetic disturbances (EMC), temperature, humidity, vibration, shock, electric safety and dirt.

It is of great importance that the FOT system installation is adapted to the requirements set by all other systems in the vehicle. If this is not done properly the installed system could generate disturbances that might void warranty of the original systems – or even an entire vehicle. All systems that possibly may be in conflict with the installed system need to be identified. An adaptation plan must then be developed for each system to ensure that they will be able to operate properly after installation.

The actual installation work needs to be done by operators that are authorised to work on the actual host system, and during the installation work, all changes to the host system (if any) must be documented in detail. Depending on the study and region, the authorising body could for example be the OEMs, insurance companies (voiding warranty etc.), the project and/or a legal entity.

If several vehicles are to be installed, it is recommended to select one vehicle as prototype. The prototype installation will then revise the installation specification continuously during the work.

7.11.2 Installation verification and calibration

When the system is installed it needs to be verified and calibrated before the data acquisition starts. The verification will refer to the installation specification and verify that all requirements are met. Monitoring of all potential sources of interference so that no conflicts are caused is important.

To ensure data validity and quality a calibration and verification scheme is recommended. For data quality aspects it is important that all installed systems of the same category are calibrated and verified using the same procedures. During the verification process a full dataset should be recorded for the analysts and quality management team, in order for them to verify that the installation adheres to the analysis requirements.

7.11.3 Dismounting the system

When the data acquisition is finished and the system is to be uninstalled, the installation documentation shall be used to ascertain that the host system is restored to its former condition. Finally, all proprietary data need to be removed from the FOT system before it is disposed or reused in future projects. Remember to include dismounting costs in the FOT budget.

7.12 Proprietary data

The concerns regarding proprietary data are to keep the CAN/LIN/MOST specifications OEM-confidential and to hide the actual system design to prohibit reverse engineering based on data collected within an FOT project. Regarding the first issue there are two cases to be distinguished.

When the OEM is strongly involved in the data acquisition process during the FOT execution, the confidentiality of the CAN/LIN/MOST specification is not an issue within the project.
When the OEMdoes not handle the data collection by himself, the usage of CAN gateways is proposed. The CAN gateway has to be programmed by the OEMto provide the data from the CAN/LIN/MOST bus according to the agreed logger specification.

The second issue – reverse engineering of functions and systems – is also an issue within the FOT project. Each project will have to handle this and define what is needed. In some cases it may be necessary for the OEMs to handle detailed low level data and aggregate it on a certain level before it is provided to the project partners responsible for data analysis. A general recommendation to future FOT projects is to define in advance, what level of system data is needed to answer a specific 'research question and whether the involved 'OEMs are able to provide this data to the project.

In some FOTs OEMs might be interested in the acquisition of additional data, which is not directly related to the project and proprietary to the OEM. This should be allowed. The OEMcould separate the additional data from the project data before the data is provided to the further project for analysis.

7.13 Personal integrity and privacy issues in data acquisition and analysis

Recommendations for the definition of necessary legal arrangements depending on the specific FOT are not covered here. See Chapter 3 for further information.

Different levels of data security should be implemented in order to cover personal and privacy issues properly. The data access right of a project partner should depend on his specific role in the project.

Several different levels of data security should be implemented. Driver data allowing direct conclusions about the identity of the driver is considered to have highest requirements regarding data security. Video data of the driver’s face and GPS data are typical examples of the data that belongs to this category. Some types of metadata like the car serial number also belong to this category of data. For this kind of data anonymisation via monikers is required before upload into a database.

It might also be necessary to implement a GPS data based control, which deactivates the video recording, when required (e.g. when driving in countries with specific legal requirements).

7.14 On-line quality management procedures

In all FOTs assuring data quality in the data collection and data management process is very important. The procedures for data quality assurance before, during and after the data collection should be well defined. Specifications and plans should be written for each individual FOT.

It is recommended that a quality management team is appointed for each individual FOT with roles such as: daily quality overview, OEM contact person, subject contact person, DAS and sensor maintenance person, and vehicle maintenance person.

7.14.1 Remote automatic upload

In most state-of-the-art FOT wireless transfer of vehicle and data status has been used, in order to assess the status of vehicle, DAS and data without having to physically access the vehicle when the vehicles in the study are on the road. Different transfer techniques such as simple text messages (SMS) or GSM/3G, have been used for status uploads. A maximum delay from the time of actually collecting the data until it has been analysed for quality and status should be defined. Otherwise the project risks that the vehicles on the field are potentially not collecting the required data. The maximum delay should be set based on the accepted levels of data loss and the length of the study.

For a thorough listing of example variables/measures that may be of interest to store in the vehicle DAS summary files for per trip data upload, see section 4.1.1 in Deliverable D2.2].

When the data has been uploaded and put into a database trip statistics can be calculated per vehicle or driver. It is recommended to use this to identify extreme/abnormal driver/usage behaviour early in the study so that if necessary drivers can be exchanged. This driver monitoring early in the study is highly recommended so that the study schema of the specific FOT is kept.

If an FOT is to be executed across country boarders and include roaming for the wireless services, investigation of the cost/benefit of using the quality data upload systems outside of the “home country”, should be made.

7.14.2 Automatic and manual quality checks

It may be tempting to do the quality assessment fully automatic, but state-of-the-art FOTs have indicated that by doing this there is the risk to contact the driver in cases where the error or anomaly is not significant for the study. Due to this, it is recommended that the quality assessment should be set up in different steps and that before (if) employing a fully automatic system, the algorithms for the assessment should be thoroughly validated. An automatic thresholds based warning system can be applied for some hard and very important measures. A tool for maintaining the warning system should preferably be checked by a responsible person each day.

For a description of a process for on-line data quality checking, see section 4.1.2 in Deliverable D2.2.

7.14.3 DAS and sensor maintenance

It is recommended that a specific subject/driver only has one contact person throughout the study. The process for contacting the driver should be clear and preferably there should be a list available within the project with contact information for the contact-responsible for each vehicle. All communication with the driver should then be at least initiated via this person. For a description of a process for fixing DAS problems/maintenance, see section 4.1.3 in Deliverable D2.2.

FESTA handbook Guidelines for Data Acquisition