The ORCHESTRA framework for evaluating the concept of Multimodal Traffic Management Ecosystem (MTME)
The ORCHESTRA evaluation framework is based on the Program theory (logic model) explaining the change mechanisms and how the MTME will achieve the intended effects and impacts (Lervåg, 2021). This framework can be applied from the early development stage of a concept until the final solution is ready for market implementation and provides a structured thinking process to identify the different factors intervening in the complex implementation process of an ecosystem.
The Program theory evaluation framework is beneficial for interdisciplinary knowledge development, and involvement of multiple types of stakeholders.
The starting point of the evaluation is the stakeholders’ analysis of how and why the planned MTM ecosystem components will lead to specific outcomes (i.e., measurable short-term effects and impacts defined as long-term effects). This comprehensive viewpoint is presented in figure 3 below explaining the relations between the inputs, effects, and impacts of the MTME, including the underlying mechanisms of change and contextual aspects.
For more information about the analysis of the needs and requirements of the various stakeholders, see D2.2, Pre-studies on environment analysis and drivers and summary provided in D6.1, Evaluation handbook.
Requirements and enablers represent the framework conditions that needs to be in place:
- Policies, governance, and regulations in terms of standardisation, legislation, ethics, etc.
- Data governance and sharing, e.g., cyber security, privacy, and real-time information sharing.
- Smart infrastructure for automation and integration of CAVs.
- Safety requirements for prevailing accidents and fatalities.
MTM functionalities to facilitate the every-day MTM operations, and to solve issues related to unexpected incidents and expected situations:
- Dynamic transport demand management.
- Demand-capacity balancing and coordination.
- Decision support.
Mechanisms of change identified in ORCHESTRA include:
- Stakeholders’ acceptance of MTME and autonomy.
- Operational practices and decision making for orchestrators e.g., skills and training.
- Business policy for transport service providers e.g., increased flexibility in contractual framework.
- Increased collaboration, coordination, and transhipment between organisations.
Traffic management effects represent:
- Better utilisation of capacity.
- Better handling of disruptions.
- Support for multimodal services like mobility-as-a-service (MaaS) and physical internet (PI).
- Support for connected and automated vehicles (CAVs).
- Increased use of green modes.
Impact on society is the long-term effect of improved sustainability with greener and more efficient transport and mobility.
The elaborated program theory of the MTME is presented in Figure 4 below:
Lervåg, L.E. (2021). Evaluation of intelligent transport systems. Development of a theory-driven evaluation methodology. PhD Thesis, Norwegian University of Science and Technology, 2021:103.