Mack Intelligent Vehicle Initiative Field Operational Test

From FOT-Net WIKI
Jump to: navigation, search
Mack Intelligent Vehicle Initiative Field Operational Test
General information
Type: Field operational test
Tested system/service: Autonomous Systems
Countries: USA  ? test users
8 partners  ? vehicles
Active from 09/1999 to 09/2006
Contact
Evaluation Final Report
John Orban
 ?
URIs of the form "?" are not allowed.
Battelle Memorial Institute
Catalogue entries
Data catalogue Tools catalogue
Data sets used in this FOT:

No data set is
linked to this FOT

Click here to create a new
data set or here to browse
existing data sets to link
them to this FOT

The following tools
were used in this FOT:

No tool is linked
to this FOT

Click here to create a
new tool or here to
browse
existing tools
to link them to this FOT

The Mack Field Operational Test (FOT) focused on the testing and evaluation of a Lane Departure Warning System (LDWS) for large trucks. The Mack project team designed and implemented the vehicle subsystems, operated the test vehicles, ensured all systems were functioning properly during the data collection period, and collected the test data. The data was reviewed by an independent evaluator, Battelle, and the costs and benefits of the LDWS were calculated. Subjective system evaluations via driver surveys were also conducted.

The purpose of the FOT was to evaluate a LDWS in terms of safety performance and driver acceptance.

Key milestones of the project

  • September 1999: Intelligent Vehicle Initiative
  • Late 2000: pre-tests
  • January 2003: the hardware and software were certified for use in the FOT
  • March 2004: Start of data collection
  • September 2006: Evaluation Final Report

Details of Field Operational Test

Start date and duration of FOT execution

Geographical Coverage

Nationwide with an emphasis on eastern United States

Link with other related Field Operational Tests

In 1999, USDOT entered into cooperative agreements with four partnerships to conduct Generation 0 Field Operational Tests (FOTs) of advanced intelligent vehicle safety systems (IVSS). The 4 partnerships are:

  • Minnesota DOT tested technologies designed to provide operators of snowplows, ambulances, and patrol cars a means to maintain desired lane position and avoid collisions with obstacles during periods of low visibility. Key among these technologies are vision enhancement, lateral guidance, and collision warning systems.
  • Volvo Trucks North America, Inc., in partnership with U.S. Xpress, tested a forward collision warning system, a blind spot warning system (not under evaluation), an adaptive cruise control, and an advanced electronic braking system for commercial vehicles
  • Mack Trucks, Inc., in partnership with McKenzie Tank Lines, will test a trucker safety advisory system and a lane departure warning system for commercial vehicles
  • Freightliner Corporation, in partnership with Praxair, tested a roll stability advisor and a roll stability control to assist commercial vehicle drivers in avoiding rollover crashes.

Objectives

According to Mack Trucks, their primary focus was on the viability of this technology for their future product line, in terms of technical performance, system robustness, and fleet/driver acceptance. McKenzie Tank Lines, Inc. was interested in determining if the LDWS was worth the investment by using the knowledge gained through in-depth experience with these systems, and by objectively assessing the prevalence of lane departures and the system’s effectiveness.

The primary goals of the independent evaluation included:

  • Achieve an in-depth understanding of system benefits
  • Obtain measures of driver performance and evaluate user acceptance
  • Ascertain the performance and capability potential of the LDWS system
  • Assess product maturity for deployment
  • Address institutional and legal issues that might impact deployment

Results

By participating in this FOT, Mack Trucks gained the opportunity to evaluate the viability of several systems used by large trucks. The TSA (Truck Safety Adviser) and ACN (Automatic Crash Notification) systems were viewed as telematics features offering high value to fleets. Although full data collection and evaluation of these systems did not occur during the FOT, Mack Trucks assessed them from a business perspective during the prototyping and testing phase of this project. These products were not brought to market due to the downturn in the telematics business in 2001-2002.

For McKenzie Tank Lines, their overall experience of the SafeTRAC™ LDWS was positive. In general, drivers found the system valuable and, in at least one case, a driver who participated in the FOT complained when the system was turned off, which was done intentionally as part of the experimental plan.

The Mack FOT independent evaluation revealed that the use of the LDWS could reduce crashes, injuries, and fatalities in crashes involving large trucks.

The FOT results showed that under the conditions observed in the FOT, the LDWS can reduce driving conflicts by 31 percent on straight roads and 34 percent on curves. Also, under similar conditions as the Mack FOT, the deployment of the LDWS would result in an approximate 21 percent to 23 percent reduction in single vehicle roadway departure crashes and 17 percent to 24 percent reduction in rollover crashes.

The results also indicated that the system can improve safety-related driving behavior, even with experienced drivers.

From a societal benefit-cost perspective, the system was economically justified for tractors pulling tanker-trailers and for tractors pulling HAZMAT (Hazardous Material) tanker-trailers under conditions similar to the FOT. Advantages of the LDWS included improvements in aiding alertness, concentration, and driving under poor driving conditions. Disadvantages included the distracting alerts and the location of the unit on the dash. The main behavioral effect of the LDWS was increased attentiveness to lane keeping, which constituted a major safety objective of this technology.

Lessons learned

This FOT revealed that a risk-based independent evaluation plan developed prior to the FOT is vital to prevent problems that may occur.

Proper planning by the independent evaluator should be in place and should be followed in a risk-based independent evaluation plan to ensure that driver training and driver surveys are performed successfully. Another important element of the plan was to install the LDWS and data acquisition systems on all trucks in the test fleet during a specific 1-to-2-month period. However, delays caused by the need to complete the development and verification of the on-board systems resulted in losing this window of opportunity, which could not be rescheduled because it would have disrupted McKenzie Tank Lines’ operations.

During the FOT, the participating truck fleet’s desire for minimal disruption of normal operations needs to be balanced by the independent evaluator’s need to execute the experimental design. Lengthy delays in the development and verification of the on-board systems, coupled with McKenzie Tank Lines’ need to deploy trucks to meet their business needs, forced significant changes in the initial research plan. As a result, the inability to control the VMT (Vehicle Miles Travel) in the baseline, active, and post-active periods resulted in a reduced amount of data that could be used in the safety-benefits estimate. Also, many of the challenges encountered in contacting drivers and checking and servicing test equipment could probably have been avoided if the drivers and vehicles had not been based at many widely dispersed terminals.

Main events

Financing

Summary, type of funding and budget

Overall

1.2 Mio $

Public

70% funded under the United States Department of Transportation (USDOT) Intelligent Vehicle Initiative (IVI)

Private

30% funded by the Mack project partners

Cooperation partners and contact persons

  • Public Authorities: United States Department of Transportation (USDOT)
  • Industry
    • Vehicle Manufacturer: Mack Trucks
    • Supplier: XATA Corporation. Assistware, Inc. and Aonix also played major roles on a contract basis.
  • Users:
  • Universities:
  • Research Institutes:
  • Others (specify): McKenzie Tank Lines (fleet operator), Bishop Consulting, Vehicle Enhancement Systems, Inc

Applications and equipment

Applications tested

Lane Departure Warning Systems

LDWS are in-vehicle electronic systems that monitor the position of a vehicle within a roadway lane and warn a driver if the vehicle deviates or is about to deviate outside the lane. The tested LDWS are forward-looking, vision-based systems that use algorithms to interpret video images to estimate vehicle state (lateral position, lateral velocity, heading, etc.) and roadway alignment (lane width, road curvature, etc.). LDWS warn the driver of a lane departure when the vehicle is travelling above a certain speed threshold and the vehicle’s turn signal is not in use. LDWS also notify the driver when lane markings are inadequate for detection, or if the system malfunctions. The LDWS does not take any automatic action to avoid a lane departure or to control the vehicle; therefore, drivers remain responsible for the safe operation of their vehicles.

The SafeTRAC™ LDWS, manufactured by Assistware, Inc., was tested in the FOT. SafeTRAC™ includes a digital camera and an image processing/user display unit. The system detects visual lane markings and can estimate some lane boundaries when visual lane markings are missing or of poor quality.

Driver feedback and warnings include:

  • A user display with an alphanumeric/graphical display to indicate vehicle position in the lane.
  • An “alertness measure” in the user display that indicates a driver’s consistency in maintaining a vehicle’s position within the lane.
  • An audible lane departure warning.
  • The ability to control vibrating (tactile/haptic) seats as an auxiliary warning device.

The Trucker Safety Adviser system provided drivers with an in-vehicle text advisory when entering a roadway zone that required increased vigilance. Examples of these Trucker Advisory Zones (TAZ) are areas with high winds, narrow shoulders, tight curves, or high crash rates.

The Automatic Crash Notification system provided notification to the fleet dispatch center, which could then contact local authorities within minutes after a crash with precise information about the load and crash situation.

Although these systems were installed on the test vehicles, the FOT focused on collecting data to evaluate LDWS technology.

Vehicle

22 long-haul freight vehicles (Due to technical issues, data collection for the full 36-vehicle fleet was not achieved as originally planned.)

Fleet from McKenzie Tank Lines

Equipment carried by test users

Infrastructure

None

Test equipment

The XATA Corp. Onvoy data collection system provided the computing and storage platform for logged vehicle position data via a Global Positioning System (GPS) receiver, and formatted messages sent via the cellular telephone connection to the project website. The Onvoy antenna, mounted on the roof of the vehicle, provided GPS and cellular reception in an integrated package.

Methodology

Pre-simulation / Piloting of the FOT

In late 2000, several tractors were configured with the LDWS for preliminary testing that was conducted both near the McKenzie headquarters and on extended runs. During this period, the LDWS was successfully tested, and McKenzie Tank Lines certified that the SafeTRAC™ design, operation, unit placement, and user interface was appropriate for use on their fleet vehicles.

Method for the baseline

The experimental plan involved 36 trucks equipped with LDWS to provide continuous data collection during an 8-month test period. The LDWS display was turned on and off during specific months of the test. To evaluate the system, data collected during periods with the display on were compared to the baseline data collected with the display off.

Techniques for measurement and data collection

Data records were transmitted to the host server via wireless communication when wireless communication coverage was available. If a crash event was detected, the transmittal of a crash notification record was given priority over other records. For the FOT, Cingular cellular service was used along with a Mobitex digital networking system. The host server archived all records and posted them on the XATANET website. Users with access (username and password) could select records and request that they be emailed to them. The host server was maintained by Aonix. The independent evaluator also maintained a separate FOT database archive.

Objective (logged data…):

Data were acquired from the following sources:

  • Eaton VORAD EVT-300 CWS – This original equipment manufacturer (OEM) rear-end CWS (Collision Warning System) was installed on all trucks in the FOT test fleet. Using radar, it collected information on the position and velocity of the truck relative to nearby vehicles. It also provided data on tractor yaw rate from a rate gyro, collected driver ID, and provided an indication that a vehicle was present on the passenger side of the truck.
  • VMAC III Modules – These modules were the standard Mack Truck electronic powertrain control modules that broadcasted operational data packets via the SAE J1708 data bus containing information on vehicle and engine speed, ambient temperature, throttle position, cruise control status, brake status, truck VIN, fuel consumption rate, and truck time in motion.
  • VES JBox Module – This module was installed on FOT trucks, and broadcasted, via the J1708 data bus, data on tractor roll angle (measured with a Kavlico Model TS904 Tilt Sensor), lateral acceleration (measured with a Kistler Model 8303A2 K-Beam Accelerometer), windshield wiper status, and turn signal status.
  • SafeTRAC LDWS – This system was the primary focus of the evaluation and, through sophisticated algorithms that interpret video images, provided critical information on truck lane-keeping behaviour, along with estimates of lane width, lane boundary type, and road curvature. The LDWS also provided data on the status of warnings enabled or disabled, and event codes that described its interpretation of an event (e.g., drifts and lane changes to the right and left).
  • Onvoy Module – This module provided GPS coordinates provided by its built-in GPS receiver, as well as date and time information

Subjective (questionnaires, focus groups…):

The assessment of driver acceptance and human factors relied upon two independent surveys, one conducted at the beginning of the evaluation period before drivers had experience with the LDWS and a second survey conducted after the drivers had accumulated experience with this system.

The primary objective was to assess driver responses to the LDWS. Survey questions were constructed to evaluate training and learning, understanding of the system capabilities, usability under real-world driving conditions, potential distraction effects of system operation, stress associated with system use, changes in perceived workload, usefulness and acceptance, and potential effects on driving behaviour and risk taking.

Recruitment goals and methods

31 participating drivers

Time of exposure : 8 months

Methods for the liaison with the drivers during the FOT execution

McKenzie Tank Lines’ normal operating procedures were a priority during the FOT data-collection period. As a consequence, liaison to the driver was done by McKenzie Tank Lines.

This leads to the fact that the VMT (Vehicle Miles Travelled) associated with each driver/truck combination could not be entirely controlled by the experimental design. Furthermore, driver training on the operation of the LDWS and other on-board systems installed for the FOT was limited to informational brochures and LDWS manuals.

Methods for data analysis, evaluation, synthesis and conclusions

During the FOT, the experimental plan execution was strongly influenced by the condition that McKenzie Tank Lines’ normal operating procedures (driver selection, truck assignments, route selections, etc.) were a priority during the FOT data-collection period.

The steps performed in the analysis were:

  1. the examination of historical crash data to determine the frequency of relevant crashes (rollover and roadway departure crashes) and the nature of pre-crash situations (driving conflicts) that led to these crashes,
  2. the comparison of the frequency of conflicts with and without drivers receiving LDWS feedback during the FOT, and
  3. the comparison of conflict severities or, equivalently, the conditional probabilities of a crash given that a truck is in a conflict. Additional analyses were performed to identify driving conditions or other factors that affect the efficacy of the LDWS in reducing numbers of crashes or the effects of the LDWS feedback on various surrogate measures of safe driving.

To determine how the LDWS could improve safety and reduce vehicle crashes, two important measures were considered:

  • Exposure of a vehicle to potential crash situations (driving conflicts)
  • Prevention of crashes when a vehicle is in a driving conflict

In addition to crash reduction estimates, other potential benefits of the LDWS were analyzed for drivers, fleets, and society.

To assess the maturity of the LDWS for deployment, interviews were conducted with system developers, Mack representatives who have experience with the system in their trucks, and consultants who were experienced with SafeTRAC and the other major LDWS currently on the market.

Sources of information

Evaluation Final Report: http://ntl.bts.gov/lib/51000/51200/51298/Evaluation-of-the-Mack-Intelligent-Vehicle-Field-Operational-Test-Sep2006.pdf

Final Report: http://www.fmcsa.dot.gov/facts-research/research-technology/report/Mack-FOT-finalreport.pdf