Practical Trial Amsterdam (PraktijkProef Amsterdam), Phase 1, In-car tests, Amsterdam Onderweg consortium
Isabel Wilmink, TNO & TrafficQuest, firstname.lastname@example.org Eline Jonkers, TNO, email@example.com
Timing and duration of tests
15 January – 15 October 2015
Location(s) of tests
Amsterdam, the Netherlands
Pre-trip departure time and route advice, on-trip route advice, navigation
Level of automation tested
0 – no automation
Tested use cases
Daily traffic, Event traffic (on Amsterdam road network)
Tested transport system
Road Transport (able to use a smartphone app while driving)
Purposes of testing
- Assessment of driver behaviour (human, vehicle) / road user behaviour
- Assessment of impacts on environment (short-term, long-term)
- Assessment of impacts on safety (short-term, long-term)
- Assessment of impacts on traffic efficiency/mobility (short-term, long-term)
- Assessment of user (driver, traveller, etc.) acceptance, usability, take up, etc.
- Technical assessment, proof of concept (incl. vehicle, background support systems such as communication)
- Scaling up, quick scan CBA
Definition of baseline
For most analyses, there was no baseline (it was not deemed feasible to ask participants to drive around with the app draining the battery and not giving advice). For the traffic efficiency analyses, the baseline was the same period in the previous year.
Drivers could download the app and register (drivers were recruited using license plate cameras, social media, ticket vendors etc.) With the app running in the background, trips were logged (GPS traces). With the app running in the foreground, trip details were stored in the back office (requests for departure time & route advice, responses of the back-office, deviations from the advised route, GPS traces). Users were also asked to fill in questionnaires. Interviews were held with road authorities / traffic management centres.
Method of testing
Interviews, Naturalistic field tests, Questionnaires
Test fleet, participants and environment
Number and make of vehicles
Description and number of participants/drivers
Over 28,000 registered users, 21,000 downloads of the app, over 19,000 participants made at least one trip, over 950,000 trips logged (app running in background or foreground), over 15,000 trips of which the compliance rate could be determined (and a lot more trips for which pre-trip advice was given)
Tested environment and facilities
Amsterdam region (app worked in all of the Netherlands, but with more detailed data in Amsterdam region)
Legal and ethical aspects
Privacy aspects (data treated according to data privacy laws). Also, HMI was designed to minimise driver distraction
Duration of testing
9 months. All roads in Amsterdam region, and main roads outside Amsterdam region. Number of journeys evaluated: thousands
Input parameters and assumptions of simulation tests
- Communication data (V2V, V2I, V2U, etc.)
- Map data
- Other cameras
- Situational data (e.g. weather, traffic situation)
- System internal data
- loop detector data, Bluetooth detector data, camera data (travel times, speeds, traffic volumes)
Specifications for the data sources
- Communication data (V2V, V2I, V2U, etc.), specify: route requests, route responses, location data
- System internal data, specify: uptime of system in back-office
- Situational data (e.g. weather, traffic situation), specify: incidents, adverse weather, large events, calamities/crises, technical problems (app & road infrastructure), road construction (recorded per day/time period); also, some participant information, such as familiarity with the Amsterdam network, indicated flexibility regarding departure time and route choice.
Key Performance Indicators (KPIs)
Many, in the following categories: app usage, trips made, compliance, changes in departure time and route choice behaviour (objective & subjective), feedback from the users about usefulness, satisfaction, etc. Also traffic efficiency indicators such as trip and route travel times, delays (vehicle hours lost in the network), km’s travelled in the network.
Situational data available
Incidents, adverse weather, large events, calamities/crises, technical problems (app & road infrastructure), road construction – only when the situation was bad enough that it would impact travel times across the network. Some participant characteristics that could influence their choices / compliance levels.
Subjective data collected
Surveys at the start, in the middle and at the end of the of the trial period, or after each event. Asking for feedback about the app, impact on travel choices and suggestions for improvement of the app.
Issues that affected the impact assessment
smartphone operating systems, messing up the GPS traces
Results are expected to be published in March 2016
Wilmink, I., E. Jonkers, A. Jöbsis, T. Djukic, E.J. van Ark, M. Duijnisveld & R. Haanstra (2016), “Eindrapport Evaluatie Praktijkproef Amsterdam IN CAR – Perceel Regulier Verkeer”, Delft, TNO, Rapportnr. TNO 2016 R10044.
Jonkers, E. I. Wilmink, A. Jöbsis, T. Djukic, E.J. van Ark, M. Duijnisveld & R. Haanstra (2016), “Eindrapport Evaluatie Praktijkproef Amsterdam IN CAR – Perceel Evenementen Verkeer”, Delft, TNO, Rapportnr. TNO 2016 R10043.
Djukic, T., I. Wilmink, E. Jonkers, M. Snelder & B. van Arem (2016), “Exploratory analysis of traveller’s compliance with smartphone personal route advice: a field trial Amsterdam”, Paper presented at the 95th Annual Meeting of the Transportation Research Board, January 2016.
Calvert, S. C., M. Snelder, T. Bakri, B. Heijligers, and V. L. Knoop (2015), Real-Time Travel Time Prediction Framework for Departure Time and Route Advice. Paper presented at the 94th Annual Meeting of the Transportation Research Board, January 2015.
Other things to report
The FESTA methodology (FESTA-V steps) was applied (this was required) and worked very well
The Amsterdam Practical Trial continues; Phase 2 starts in 2016