FESTA handbook Planning and Running a Field Operational Test

Back to FESTA Handbook main page: FESTA handbook

2. Planning and Running a Field Operational Test

2.1 Introduction

For a Field Operational Test(FOT) to proceed smoothly, a plan of action must be developed which documents the scientific, technical, administrative, and procedural activities that are needed to successfully complete it. Given that the lifecycle of an FOT typically evolves through many phases, there are many issues to consider. In this chapter, the critical activities and tasks which are necessary to run a successful FOT are documented — in the form of a “FOT Implementation Plan” (FOTIP) — drawing on lessons learned from previous FOTs conducted in different parts of the world.

The FOTIP is described in Annex B of the FESTA Handbook. In this chapter it is introduced, described, explained and discussed.

2.2 The FOT Implementation Plan

2.2.1 Purpose

The FOTIP is intended to serve primarily as a checklist for planning and running FOTs:

• To highlight the main activities and tasks that would normally be undertaken in successfully completing an FOT
• To ensure that, in running an FOT, researchers and support teams are aware of critical issues that influence the success of the FOT
• By drawing on the experiences of previous FOTs, to highlight the “dos” and “don’ts” of running an FOT
• To provide a consistent framework for planning, running and decommissioning FOTs.

The FOTIP presented in this Handbook is not intended to be prescriptive, but rather to serve as a generic guide in conducting FOTs. By their very nature FOTs are major projects — extensive and expensive. Previous FOTs that have not delivered their anticipated outcomes have not done so primarily because of failures to anticipate problems that compromised their successful execution. The FOTIP attempts to map out all known critical issues that need to be taken into account in planning and undertaking an FOT.

The history of FOTs suggests that no two will be the same, and that there often are many unforeseen tasks and sub-tasks that arise during its lifecycle. The list of tasks contained in the FOTIP in Annex B of this Handbook is not, therefore, exhaustive. It is based on the collective wisdom of those that have been involved in planning and running previous FOTs. There may be specific requirements for future FOTs conducted in Europe that will need to be decided on a case-by-case basis, especially where FOTs are extended to cover Naturalistic Driving Studies and, potentially, other transport modes such as public transport, cycling, walking or multi-modal studies.

The FOTIP in Annex B describes what needs to be done, and approximately when, in running a successful FOT. Other relevant chapters in the FESTA handbook describe in detail why these activities are necessary and how they are to be accomplished.

2.2.2 Description of the FOT Implementation Plan

The FOTIP in Annex B of this Handbook resembles a traditional Work Breakdown Structure (WBS), but without timelines. It is specifically designed in this way so that timelines can be inserted at a later date by those responsible for the overall planning and running of the FOT.

The FOTIP is divided into three columns and two sections below each activity:

• Column 1 — Activities. An Activity is a high level task e.g. “Convene FOT research and support teams” that is usually needed to run an FOT.
• Column 2 — Tasks and Sub-Tasks. A Task directly supports an Activity e.g. “Appoint FOT project manager”. A Sub-Task directly supports a Task. Essentially, this column contains a series of action statements – “do this”; “do that”; etc. There are very few sub-tasks listed in this column, to contain the size of the document. The document is cross-referenced to other chapters of the FESTA Handbook, which identify the relevant Sub-Tasks that support these Tasks.
• Column 3 — Person/Organisation Responsible for Activity. This column identifies the person, team, organisation or combination thereof that would usually be responsible for completion of a Task. The FOT project manager is ultimately accountable for successful completion of all Tasks, and is therefore included for every Task.
• After section 1 – Critical Considerations (the “dos” and “don’ts”). This column contains critical advice for ensuring that an Activity or Task is successfully completed. e.g. “Be sure that the vehicle systems are designed so they do not drain the battery when the vehicle engine is not running.” e.g. “Do not underestimate the amount of time required to recruit company drivers for the FOT.”
• After section 2 – General Advice. This column provides general advice on how to maximise the likelihood of running a successful FOT e.g. “The FOT lifecycle is long. Hence, it is advisable to write separate reports on each critical stage of the FOT….” This column also contains explanatory notes, reference to other relevant documents (e.g., [ FOT] reports) and cross-referencing to other chapters in the FESTA Handbook.

The Activities and Tasks identified in the FOTIP are consistent with those identified in the higher level “FOT Chain” that is described in Chapter 1 of this Handbook (see Figure 1.1), although the chronological order in which the Activities and Tasks are shown varies slightly between the two. For example, in the FOT Chain, in Figure 1.1, it is assumed that the first step when planning an FOT is the identification of systems and functions to be analysed. In the FOTIP, on the other hand, this task is identified later in the sequence of planning activities (within Activity 2), as there are other planning activities and tasks that necessarily precede the identification of systems and functions to be analyzed. The FOTIP identifies the scientific, technical, administrative and procedural activities for planning and running an FOT; the FOT chain summarizes the key, high level, scientific and technical steps undertaken when performing an FOT, and the sequential links between them. Decisions taken at earlier stages may have a big impact on the FOT development and the analysis phase. Based on this concept, involving since the beginning all experts that will be in charge of the activities in the successive phases, is theoretically the safest strategy. But this is not economically sustainable and also will create large teams analyzing each issue, with the results of expanding timing. For this reason a key person is the FOT project manager. He should ensure that sign-off of different activities is properly validated consulting or involving in specific meetings the right people. Although this preventive policy is applied, criticalities depending on decisions taken in previous activities/tasks may happen. In this cases, negative or partially negative sign-offs may happen and re-analysis of previous activities or tasks should be done. Figure 2-1 illustrates this potentially cyclic process.

Figure 2.1: Iterative development process

2.2.3 Development of FOT Implementation Plan

The content of the FOT Implementation Plan derives originally from several research activities undertaken in the FESTA project, based on comprehensive review of the related literature and several consultation (via workshops, teleconference, written feedback)of experts member of the consortium and external experts who previously conducted FOTs. It was the revised in rev 4 and rev 5 based on the feedback received in the FOT - Net and FOT - Net 2.

2.2.4 Assumptions underlying the FOT Implementation Plan

There is more than one way to perform a successful FOT. The review of the literature on FOTs revealed that many different approaches have been taken in planning, running, analyzing and decommissioning FOTs. The FOTIP in Annex B of this Handbook draws together procedural activities that are most common to the known FOTs that have been conducted, and the collective wisdom of those who conducted them.

The FOT Implementation Plan is relevant to FOTs in which the ADAS and In Vehicle Information systems to be evaluated are already available on the market, or to studies in which the systems to be evaluated must be chosen by the FOT project team, purchased or developed, and installed (e.g., as in [ Regan] et al., 2006).

The FOT Implementation Plan provides only a general guide to the sequence in which the required activities should be performed. Some need to happen early in the project and others at the end. Some need to immediately precede others. Other tasks need to proceed concurrently with others. Decisions about the scheduling of Activities, Tasks and Sub-Tasks are the responsibility of the FOT Project Manager. Table 2-1 lists the 22 Activities identified in the FOTIP, and highlights the main dependencies that exist between them. Within Activities, it is up to the FOT Project manager to further decide which Tasks and Sub-Tasks should proceed sequentially and in parallel.

Some of the major Tasks listed in the FOTIP (e.g. “recruit participants”, within the Activity “Run FOT”) are given only a one-line description and, as such, may appear to be down played in the plan. A judgement had to be made about how much detail to include in the FOTIP. Where such one-liners exist, this is because either the Task in question is one that most researchers would normally be familiar with (e.g., recruiting study participants) or because the Sub-Tasks involved are described in detail in other Chapters of the FESTA Handbook. Where appropriate, any known difficulties and concerns associated with major Tasks for which only a one-line description is given are emphasised.

Table 2-1: A generic guide to scheduling the 22 Activities described in the FOTIP in Annex B of the FESTA Handbook.

2.2.5 Using the FOT Implementation Plan

It is suggested that the FOTIP be used as follows:

• read through the FOTIP before starting to plan an FOT ;
• use the FOTIP as a checklist for guiding the planning, design and running of the FOT — and as a quality control mechanism for ensuring during the study that nothing critical has been forgotten;
• read the FOTIP in conjunction with other chapters in the FESTA Handbook, and refer to other chapters and other FOT reports for detail; and
• if desired, use the FOTIP as the basis for the development of GANTT charts and other project management tools.