Key Performance Areas of ORCHESTRA and research approaches
Thanks to the program theory evaluation framework, ORCHESTRA has identified 12 relevant research and Key Performance Areas for the evaluation of the Multimodal Traffic Management with the corresponding Research Questions and Key Performance Indicators.
In ORCHESTRA, different types of evaluations are defined:
- Effect evaluation – to evaluate short term outcomes of new measures in ORCHESTRA related to the introduction of the MTM system.
- Impact evaluation – to evaluate/predict long term impacts of the new ORCHESTRA measures.
- Process evaluation – to evaluate and learn from the result implementation/establishment process.
The 12 Key Performance Areas are:
- Policies, governance and regulations
- Data governance and sharing
- Smart infrastructure
- Technological solutions
- Stakeholders’ acceptance of traffic management operations and autonomy
- Operational practices and decision making for traffic orchestrators
- Business policy aspects
- Organisational aspects
- Traffic management effects
- Economic impact (increased cost-efficiency)
- Environmental impact
- Transport and mobility impact
For more information about the research questions and KPIs, please refer to D6.2, Intermediate evaluation results from Living Labs.
Figure 5 illustrates the generic steps in the research process and the selected research approaches and methods for the required activities. In general, the research methods require data collection to gather useful information from various sources and data analyses to provide findings and KPIs that can be evaluated to facilitate a conclusion where the results are discussed and reported with respect to lessons learned and answers to RQs. The results contribute to the evaluation of the MTME as a whole and by research area and KPA.
The research approaches and methods are selected based on the nature of the individual KPAs, KPIs and RQs. Hybrid approaches with several research methods are used to improve the reliability and validity of the results.
