FESTA handbook Planning and Running a Field Operational Test
Back to FESTA Handbook main page: FESTA handbook
- 1 Planning and Running a Field Operational Test
Planning and Running a Field Operational Test
For a Field Operation Test(FOT) to proceed smoothly, a plan of action must be developed which documents the scientific, technical, administrative and procedural activities and tasks that are needed to successfully complete it. Given that the lifecycle of an FOT typically evolves through many phases, there are many issues to consider. In this chapter, the critical activities and tasks which are necessary to run a successful FOT are documented — in the form of a “FOT Implementation Plan” (FOTIP) — drawing on lessons learned from previous FOTs conducted in Europe, the United States, Japan, Australia and elsewhere.
The [ FOTIP] is contained in Annex B of the FESTA Handbook. In this chapter, the [ FOTIP] is introduced, described, explained and discussed.
The FOT Implementation Plan
The [ FOTIP] is intended to serve primarily as a checklist for planning and running FOTs:
- to highlight the main Activities and Tasks that would normally be undertaken in successfully completing an FOT ;
- to ensure that, in running an FOT, researchers and support teams are aware of critical issues that influence the success of the FOT; and
- by drawing on the experiences of previous FOTs, to highlight the “dos” and “don’ts” of running an FOT ;
- to provide a consistent framework for planning, running and decommissioning FOTs.
The [ FOTIP] presented in this Handbook is not intended to be prescriptive, but rather to serve as a generic guide in conducting FOTs. By their very nature FOTs are major projects – extensive and expensive. Significant previous FOTs that have not delivered their anticipated outcomes have not done so primarily because of failures to anticipate problems that compromised their successful execution. The [ FOTIP] attempts to map out all known critical issues that need to be taken into account in planning and undertaking an FOT .
The history of FOTs suggests that no two will be the same, and that there often are many unforeseen Tasks and Sub-Tasks that arise during its lifecycle. The list of Tasks and Sub-Tasks contained in the [ FOTIP] in Annex B of this Handbook is not, therefore, exhaustive. It is based on the collective wisdom of those that have been involved in planning and running previous FOTs. There may be specific requirements for future FOTs conducted in Europe that will need to be decided on a case-by-case basis.
The [ FOTIP] at Annex B describes what needs to be done, and approximately when, in running a successful FOT. Other relevant chapters in the FESTA handbook describe in detail why these activities are necessary and how they are to be accomplished.
Description of the FOT Implementation Plan
The [ FOTIP] in Annex B of this Handbook resembles a traditional Work Breakdown Structure (WBS), but without timelines. It is specifically designed in this way so that timelines can be inserted at a later date by those responsible for the overall planning and running of the FOT.
The [ FOTIP] is divided into three columns and two sections below each activity:
- Column 1 — Activities. An Activity is a high level task e.g. “Convene FOT research and support teams” that is usually needed to run an FOT.
- Column 2 — Tasks and Sub-Tasks. A Task directly supports an Activity e.g. “Appoint FOT project manager”. A Sub-Task directly supports a Task. Essentially, this column contains a series of action statements – “do this”; “do that”; etc. There are very few sub-tasks listed in this column, to contain the size of the document. The document is cross-referenced to other chapters of the FESTA Handbook, which identify the relevant Sub-Tasks that support these Tasks.
- Column 3 — Person/Organisation Responsible for Activity. This column identifies the person, team, organisation or combination thereof that would usually be responsible for completion of a Task. The FOT project manager is ultimately accountable for successful completion of all Tasks, and is therefore included for every Task.
- After section 1 – Critical Considerations (the “dos” and “don’ts”). This column contains critical advice for ensuring that an Activity or Task is successfully completed. e.g. “Be sure that the vehicle systems are designed so they do not drain the battery when the vehicle engine is not running.” e.g. “Do not underestimate the amount of time required to recruit company drivers for the FOT.”
- After section 2 – General Advice. This column provides general advice on how to maximise the likelihood of running a successful FOT e.g. “The FOT lifecycle is long. Hence, it is advisable to write separate reports on each critical stage of the FOT…” This column also contains explanatory notes, reference to other relevant documents (e.g., [ FOT] reports) and cross-referencing to other chapters in the FESTA Handbook.
The Activities and Tasks identified in the [ FOTIP] are consistent with those identified in the higher level “FOT Chain” that is described in Chapter 1 of this Handbook (see Figure 1.1), although the chronological order in which the Activities and Tasks are shown varies slightly between the two. For example, in the FOT Chain, in Figure 1.1, it is assumed that the first step when planning an FOT is the identification of systems and functions to be analysed. In the [ FOTIP], on the other hand, this task is identified later in the sequence of planning activities (within Activity 2), as there are other planning activities and tasks that necessarily precede the identification of systems and functions to be analyzed. The [ FOTIP] identifies the scientific, technical, administrative and procedural activities for planning and running an FOT; the FOT chain summarizes the key, high level, scientific and technical steps undertaken when performing an FOT, and the sequential links between them.
Development of FOT Implementation Plan
The content of the FOT Implementation Plan derives from several research activities undertaken in Work Package 2.5 of the FESTA project:
- a comprehensive review of the literature on previous FOTs undertaken in different parts of the world: the United States and Canada; the Asia-Pacific region (including Australia and Japan); Europe; and Scandinavia. This included reference to FOT project plans, internal reports, meeting minutes and related documents, where possible. A special literature review of FOTs of [ nomadic devices] was also undertaken, which encompassed all of these regions. References for the publicly available literature reviewed are listed at the end of the Handbook.
- a one-day workshop with FOT experts who had previously conducted FOTs, in Europe, the United States and Australia. This activity, along with the outputs of the literature reviews, identified critical Activities, Tasks and Sub-Tasks for successfully conducting FOTs, as well as the practical “dos” and “don’ts” of carrying out FOTs;
- an international teleconference with experts with experience in conducting FOTs and naturalistic driving studies. This augmented the information derived from the workshop;
- written feedback from FOT experts, who commented on an earlier draft of the FOT Implementation Plan;
- internal consultation with other FESTA Work package leaders, to identify critical scientific, technical and administrative activities arising from other FESTA research activities undertaken in developing other chapters of the FESTA Handbook; and
- Feedback received in FOT-Net.
Assumptions underlying the FOT Implementation Plan
There is no one way of conducting a successful FOT . The review of the literature on FOTs revealed that many different approaches have been taken in planning, running, analysing and decommissioning FOTs. The [ FOTIP] in Annex B of this Handbook draws together procedural activities that are most common to the known FOTs that have been conducted, and the collective wisdom of those who conducted them.
The FOT Implementation Plan is relevant to FOTs in which the ADAS and In Vehicle Information systems to be evaluated already exist as production systems in vehicles, or to studies in which the systems to be evaluated must be chosen by the FOT project team, purchased or developed, and installed (e.g., as in [ Regan] et al., 2006).
The FOT Implementation Plan provides only a general guide to the sequence in which Activities, Tasks and Sub-Tasks should be performed. Some need to happen early in the project and others at the end. Some need to immediately precede others. Other tasks need to proceed concurrently with others. Decisions about the scheduling of Activities, Tasks and Sub-Tasks are the responsibility of the FOT Project Manager. Table 2-1 lists the 22 Activities identified in the [ FOTIP], and highlights the main dependencies that exist between them. Within Activities, it is up to the FOT Project manager to further decide which Tasks and Sub-Tasks should proceed sequentially and in parallel.
Some of the major Tasks listed in the [ FOTIP] (e.g. “recruit participants”, within the Activity “Run FOT”) are given only a one-line description and, as such, may appear to be down played in the plan. A judgement had to be made about how much detail to include in the [ FOTIP]. Where such one-liners exist, this is because either the Task in question is one that most researchers would normally be familiar with (e.g., recruiting study participants) or because the Sub-Tasks involved are described in detail in other Chapters of the FESTA Handbook. Where appropriate, any known difficulties and concerns associated with major Tasks for which only a one-line description is given are emphasised.
Table 2-1: A generic guide to scheduling the 22 Activities described in the FOTIP in Annex B of the FESTA Handbook.
Using the FOT Implementation Plan
It is suggested that the [ FOTIP] be used as follows:
- read through the [ FOTIP] before starting to plan an FOT ;
- use the [ FOTIP] as a checklist for guiding the planning, design and running of the FOT — and as a quality control mechanism for ensuring during the study that nothing critical has been forgotten;
- read the [ FOTIP] in conjunction with other chapters in the FESTA Handbook, and refer to other chapters and other FOT reports for detail; and
- if desired, use the [ FOTIP] as the basis for the development of GANTT charts and other project management tools.