SOTIF risk mitigation based on unified ODD monitoring for autonomous vehicles

Wenhao Yu (School of Vehicle and Mobility, Tsinghua University, Beijing, China)
Jun Li (School of Vehicle and Mobility, Tsinghua University, Beijing, China)
Li-Ming Peng (Department of Vehicle Engineering, Hefei University of Technology, Hefei, China)
Xiong Xiong (Department of Decision and Control and Department of Aeronautical and Vehicle Engineering, KTH Royal Institute of Technology, Stockholm, Sweden)
Kai Yang (College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing, China)
Hong Wang (School of Vehicle and Mobility, Tsinghua University, Beijing, China)

Journal of Intelligent and Connected Vehicles

ISSN: 2399-9802

Article publication date: 28 June 2022

Issue publication date: 11 October 2022

1578

Abstract

Purpose

The purpose of this paper is to design a unified operational design domain (ODD) monitoring framework for mitigating Safety of the Intended Functionality (SOTIF) risks triggered by vehicles exceeding ODD boundaries in complex traffic scenarios.

Design/methodology/approach

A unified model of ODD monitoring is constructed, which consists of three modules: weather condition monitoring for unusual weather conditions, such as rain, snow and fog; vehicle behavior monitoring for abnormal vehicle behavior, such as traffic rule violations; and road condition monitoring for abnormal road conditions, such as road defects, unexpected obstacles and slippery roads. Additionally, the applications of the proposed unified ODD monitoring framework are demonstrated. The practicability and effectiveness of the proposed unified ODD monitoring framework for mitigating SOTIF risk are verified in the applications.

Findings

First, the application of weather condition monitoring demonstrates that the autonomous vehicle can make a safe decision based on the performance degradation of Lidar on rainy days using the proposed monitoring framework. Second, the application of vehicle behavior monitoring demonstrates that the autonomous vehicle can properly adhere to traffic rules using the proposed monitoring framework. Third, the application of road condition monitoring demonstrates that the proposed unified ODD monitoring framework enables the ego vehicle to successfully monitor and avoid road defects.

Originality/value

The value of this paper is that the proposed unified ODD monitoring framework establishes a new foundation for monitoring and mitigating SOTIF risks in complex traffic environments.

Keywords

Citation

Yu, W., Li, J., Peng, L.-M., Xiong, X., Yang, K. and Wang, H. (2022), "SOTIF risk mitigation based on unified ODD monitoring for autonomous vehicles", Journal of Intelligent and Connected Vehicles, Vol. 5 No. 3, pp. 157-166. https://doi.org/10.1108/JICV-04-2022-0015

Publisher

:

Emerald Publishing Limited

Copyright © 2022, Wenhao Yu, Jun Li, Li-Ming Peng, Xiong Xiong, Kai Yang and Hong Wang.

License

Published in Journal of Intelligent and Connected Vehicles. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence maybe seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

1.1 Motivation

The autonomous vehicle is one of the highly anticipated new technologies of the modern era (Huang et al., 2019). It encompasses research from diversified fields, such as control theory, computer science and orientation engineering (Kuutti et al., 2018; Rasouli and Tsotsos, 2019; Huang et al., 2018). Given that a large proportion of traffic accidents are caused by human mistakes, autonomous vehicles are considered a promising solution for reducing traffic accidents on the road (Wang et al., 2018). However, the emerging accidents involving autonomous vehicles alert us to the insufficiency of safety guarantees (Ren et al., 2019; Wang et al., 2015). The complex operating environment and performance insufficiency of artificial intelligence (AI) algorithms are triggering the safety risks of autonomous vehicles (Wang et al., 2022; Willers et al., 2020), which is beyond the scope of conventional safety concepts, including functional safety.

The Safety of the Intended Functionality (SOTIF) is a novel safety concept developed to address the emerging safety domain of autonomous vehicles. It considers the absence of unreasonable risk due to hazards resulting from the performance limitations/functional insufficiency or reasonably foreseeable misuses (ISO, 2019). Two necessary conditions would lead to the SOTIF risks: trigger condition and performance limitation/functional insufficiency of the system. Significant research efforts have been continuously devoted to improving the functionality of AI algorithms (Yang et al., 2021; Chen et al., 2020; Wang et al., 2020), while the trigger conditions in the operational design domain (ODD) of an autonomous vehicle have not been well addressed in the available research literature yet. Therefore, the main focus of this research is to develop a unified ODD monitoring framework to mitigate the SOTIF risks triggered by vehicles exceeding ODD boundaries in complex traffic scenarios.

1.2 Related research

ODD refers to the specific operating conditions under which an autonomous vehicle is designed to operate. Several aspects of ODD have been investigated and discussed (Xia et al., 2020; Talamini et al., 2020; Xiao et al., 2021), including weather conditions, vehicle behaviors, and road conditions.

Environmental perception in adverse weather conditions remains a major challenge to ensure the safety of autonomous vehicles. Several weather monitoring approaches have been used to improve the insufficiency of perception algorithms. For instance, Karlsson et al. (2021) proposed a probabilistic hierarchical Bayesian model to quantitatively estimate rainfall from the LiDAR point cloud sequences for better rain recognition. Bossu et al. (2011) improved the snowflake trajectory model and realized the recognition of raindrops and snowflakes. A camera-based method for fog ambiguity effect detection was proposed to distinguish the existence of fog in the street scene (Spinneker et al., 2014). To recognize various weather conditions, Lin et al. (2017) proposed the region selection and concurrency model to effectively screen out the discriminative regional features in outdoor images.

The main focuses of the aforementioned research are on optimizing the functionality of the AI algorithms, while it should be noted that sensor performance will decline significantly in adverse weather and then endanger vehicle safety. The speed limit is an effective way to ensure the safety of vehicles operating in adverse weather conditions with limited visibility. However, most of the current speed limit methods are based on the limitation of human visibility rather than sensor visibility. Therefore, the visibility limitation of sensors should be considered in the ODD monitoring in adverse weather to mitigate SOTIF risks.

In terms of vehicle behavior, an autonomous vehicle should adhere to traffic rules in the same way as a human driver would in a naturalistic driving environment. Aggressive vehicle behaviors that violate traffic rules may jeopardize the safety of autonomous vehicles or cause more unpredictable behavior of other traffic participants, which increases the SOTIF risk of the local driving environment. Several studies have been conducted to formalize and integrate traffic rules into the decision-making system. Li et al. (2018) proposed a Takagi–Sugeno fuzzy neural network decision-making model, which takes 16 factors related to ethical and legal vehicle-road-environment into account under red-light running situations. With regard to optimization approaches, Wei et al. developed an optimal control strategy that meets the complex specifications of traffic laws and cultural expectations of reasonable driving behaviors (Xiao et al., 2021). However, there is a lack of research on how to transform the logical expression of traffic rules into vehicle behavior norms to guide and standardize the driving behaviors of autonomous vehicles.

For the aspect of road conditions, various types of road defects will affect the dynamic stability of the autonomous vehicle, triggering the SOTIF risks. Improved recognition methods have been used to improve the detection performance of road defects. Xin et al. proposed a laser-based method to measure pothole properties comprehensively and precisely by using 3D line laser data for road defect recognition (She et al., 2021). To accomplish the detection and safety evaluation of road potholes, Wu et al. (2019) developed an algorithm that integrates the mobile point cloud and images. However, most detections are not used for detecting big enough defects that affect the safety of the vehicles, and safe decision-making avoiding road defects has not been well addressed in available research yet.

Furthermore, the aforementioned approaches of ODD risk mitigation for autonomous vehicles consider only one aspect of ODD in each literature. The existing models have their own imperfections, due to the neglect of the sensor degradation in adverse weather, noncompliance with traffic rules and complex road conditions. The conventional monitoring function is only designed to detect anomalies, whereas the problem-solving relies on the decision-making and control modules. However, any additional requirements might bring major reconstruction of the decision-making algorithms. As with functional safety's fault detection and diagnosis techniques, a modularized framework should be developed to assist the decision-making module to deal with the anomalies in ODD. Therefore, a unified model for the ODD risks monitoring triggered by vehicles exceeding ODD boundaries should be constructed for the practical application of autonomous vehicles.

1.3 Contribution

To the best of our knowledge, SOTIF risk mitigation based on ODD monitoring has not been explored yet. In brief, a unified ODD monitoring framework is proposed for SOTIF risk mitigation: for weather condition monitoring, an accurate safe speed limit method based on the performance decline information of sensors is proposed; for vehicle behavior monitoring, a digitalization framework of traffic rules is proposed; and for road condition monitoring, a recognition and evaluation method of road defects is proposed. Applications are simulated to verify the proposed unified ODD monitoring framework and its ability to mitigate SOTIF risks.

1.4 Paper organization

The rest of this paper is organized as follows. In Section 2, the unified ODD monitoring framework is modeled and discussed. Furthermore, three modules of the unified model are designed and analyzed in detail, respectively, including weather condition monitoring, vehicle behavior monitoring and road condition monitoring. In Section 3, applications of the unified ODD monitoring framework are carried out, and the performance of the three modules is analyzed with corresponding scenarios. The conclusion and future work are demonstrated in Section 4.

2. The unified operational design domain monitoring framework

Figure 1 depicts the overall structure of the unified ODD monitoring framework. For autonomous vehicles, the factors of ODD boundaries, which should be mainly focused on, can be classified into three main categories: weather category, vehicle category and road category. As is illustrated in Figure 1, there are mutual effects between different categories, which means that factors in one category may affect factors in another.

Specifically, the weather category encompasses adverse weather conditions, such as rain, snow and fog. The trigger conditions for SOTIF risks in the weather category include the following: degradation of sensors; the functional insufficiency of perception algorithms; and variation of the road friction coefficient.

The vehicle category primarily comprises the factors involving traffic-rule compliance, perception algorithms, prediction algorithms, etc. The trigger conditions of SOTIF risks in the vehicle category include:

  • illegal driving behaviors; and

  • the functional insufficiency of on-board AI algorithm.

The road category mainly includes the road conditions, such as road defects and slippery roads. The road category would affect the dynamic stability of an autonomous vehicle, which might cause the functional insufficiency of the autonomous system and trigger the SOTIF risks.

In this paper, the unified ODD monitoring framework is divided into three modules, including weather condition monitoring, vehicle behavior monitoring and road condition monitoring. Each module is responsible for monitoring the corresponding category and producing the reference signals to assist the original autonomous system in coping with anomalies. For brevity, the primary focus of each module will be assigned to specific factors.

2.1 Weather condition monitoring

There are drastic performance degradations of Lidar and variation of road friction coefficients on rainy days. A proper speed limit based on real-time weather conditions is the most effective method to guarantee driving safety and mitigate SOTIF risks. To enhance the safety of autonomous vehicles, an accurate safe speed limit method is proposed based on safe distance and detectable distance of sensors.

For longitudinal safety, safe distance models are mainly used. Many of them determine the safety states of a vehicle by analyzing the safe distance based on the relative movement between leading and following vehicles in real-time (Miller and Huang, 2002). The safe distance model from the Responsibility-Sensitive Safety (RSS) is used in the process of speed limit design. RSS safe distance is a dynamic distance, which cannot be used in speed limit calculation. Therefore, the modified RSS safe distance is defined under the assumption of a static leading vehicle, which is given as (Shalev-Shwartz et al., 2017):

(1) SSD=(v+ρamaxacc)22μg+vρ+0.5ρ2amaxacc
where SSD is the safe distance, v denotes the velocity of the following vehicle, ρ denotes the response time, amaxacc denotes the maximum acceleration of the following vehicle and µ denotes the road friction coefficient.

The depth of the water film is the most important factor affecting the road friction coefficient on rainy days, which can be attained by:

(2) d=0.046(LfI)0.5Sf0.5
(3) μ=0.21e1.8d+0.4
where d represents water film depth, Lf denotes the length of the flow path, I is the rain intensity and Sf is the flow path slope.

Detectable distance is the maximal distance under which an autonomous vehicle can detect objects. The detectable distance of human eyes is affected by both speed and rainfall intensity, whereas that of sensors is limited only by the rainfall intensity. Rainfall experiments should be conducted to determine the performance attenuation of Lidar on rainy days. Lidar can detect objects only when receiving energy is larger than the minimum receivable energy. Therefore, the maximum detectable distance of Lidar is determined by the received energy, which is presented by:

(4) PR=PEDR2ρrcosα4R2ηsysηAtm
where PR denotes the energy of the laser produced by lidar, PE is received laser energy, DR represents the radium of aperture and ρr stands for the reflection rate of objects. Furthermore, α is the incidence angle, ηsys refers to the transmission efficiency of the system and ηAtm is the transmission efficiency of the atmosphere.

The detectable distance in each rain intensity can be calculated using the information of the detectable distance in normal conditions. Then, the speed limit on rainy days can be calculated as follows:

(5) SSD(vlim)=Ddetectable
where Ddetectable is the detectable distance, and SSD is a function of the speed limit, which replaces the velocity of the following vehicle in equation (1).

In the meantime, hydroplaning happens when water film builds between the wheels of the vehicles and the road surface, which will lead to a decrease in the friction of the road. A rough prediction of the hydroplaning speed of the vehicle is calculated as follows (Ong and Fwa, 2007):

(6) vp=6.36p

At the same time, the speed limit of traffic rules from vehicle behavior monitoring should be considered. Therefore, the reference speed limit from the weather condition monitoring module is as follows:

(7) vweather=min{vlim,  vp}

2.2 Vehicle behavior monitoring

The performance of traffic-rule compliance is a critical factor in vehicle behavior monitoring for an autonomous vehicle. But the digitalization of traffic rules entails more than elaborating a certain traffic rule into a series of logical expressions. Additionally, it should also transform the logical expressions into vehicle behavior references to guide and standardize the driving behaviors of autonomous vehicles. To enhance the safety of autonomous vehicles, a digitalization framework of traffic rules is proposed to ensure compliance with traffic rules.

Figure 2 demonstrates the overview of the digitalization framework of traffic rules. Despite the variation of traffic rules in different countries, most of them regulate four aspects of driving behavior, including speeds, distances, actions and right of way. For speed constraints, upper and lower limits on the vehicular speed will be specified to restrict the vehicle behaviors in a certain scenario. For distance constraints, the distance between vehicles and other traffic participants is usually constrained by a specific distance in some specific scenes. In some cases, fuzzy descriptions, such as “maintain a safe distance,” might be given, necessitating the addition of additional thresholds for digitalization. For action constraints, it will specify whether the vehicle can pass, overtake, change lanes, stop, etc. under certain conditions. At the same time, it will incorporate the distance constraints as trigger conditions, such as prohibiting certain activities within a specified distance of certain facilities or road structures. For the right of way restriction, it will be stipulated when the vehicle will have priority right of way under specific situations, such as “straight forward or right forward vehicles go first.”

The digitalization framework of traffic rules contains two parts, including digitalization and monitoring. In the digitalization part, the logical definition is critical to the smooth operation of digitalization. It defines the computable quantitative expression of all the entities involved in rules and is the adaptive quantitative logical translation of those entities. The traffic rules are elaborated into logical definitions, which contain the trigger conditions and fuzzy descriptions. The trigger conditions are the prerequisites for the effectiveness of each traffic law, such as the presence of various traffic signs and markings, the presence of certain facilities or road conditions and the presence of certain weather conditions. For fuzzy descriptions, the thresholds should be defined in advance. For instance, the longitudinal TTC distance for safety is equal to 5 s. In the monitoring part, the reference outputs are classified into three main categories, including reference variables, state constraints and their combination. The working procedure of the digitalization framework of traffic rules is demonstrated in Pseudocode of the working procedure for the digitalization framework of traffic rules.

Algorithm 1 Pseudocode of the working procedure1: Initialization:2:   transform the traffic rules into logical    definitions;3:   sort out the trigger conditions    for specific traffic rules from the    logical definitions;4:   define the thresholds for the    fuzzy descriptions;5: While the autonomous vehicle is  proceeding:6:   Receive the state of the ego vehicle;7:   Receive the states of the driving     environments;8:   If the trigger conditions are satisfied:9:        search for the logical definitions      for the specific traffic rule;10:         search for the thresholds for the           fuzzy descriptions;11:   output the reference signals     

{reference variables, states 12:         

constraints,  their combination}13:   End14: End

2.3 Road condition monitoring

Road conditions have direct impacts on vehicular dynamics, whereas the majority of perception modules are focused exclusively on traffic participants or traffic signs. Road deformations should be detected to avoid hazards that might result in extreme changes in vehicle states and escalate into an out-of-control situation. A recognition and evaluation method of road defects is proposed to mitigate SOTIF risks caused by road conditions.

Figure 3 demonstrates the working flow of the recognition and evaluation method of road defects. The method is based on the point cloud information of Lidars. To begin with, the point cloud information of Lidars usually contains objects, including traffic participants, roadside objects and other irrelevant information. Therefore, it is necessary to extract the region of interest from the data by filtering out the point cloud of the road surface.

For the meshing and plane fitting section, the road should be meshed to minimize the deviation between the fitting plane and the road surface, due to the variation of the slope on the road. Then, a modified random sample consensus (RANSAC) algorithm is designed to fit the plane and build a pavement model. The inliers of the road surface planes are filtered out via the RANSAC algorithm, while the planes are fitted by the least square method. Specifically, the pseudocode of the modified RANSAC algorithm is demonstrated in The pseudocode of the modified RANSAC algorithm.

Part 1 of the modified RANSAC algorithm1: input: the set for all points in the grid unit, C; distance threshold dth2:  output: the estimated best-fitted plane for this grid, Pbest3:  N ← 04:  

nc card(C)5:  while N < Nmax do6:   S ← 3 random points in set C7:   P ← the plane constructed by S8:   A ← all points in set C whose distance to the plane P satisfy d < dth9:   if card(A) > card(Amax) then10:    PbestP11:    AmaxA12:  end if13:  if card(A)/ nc > ath1 then14:    return Pbest15:  end if16:  NN + 117: end whilePart 2 of the modified RANSAC algorithm1: input: the set for all points in a grid unit, C2: output: the chosen fitted plane for this grid, Pchosen3: nc ← card(C)4: Pchosen← Part 1(C, dth1)5: nP← the count of points in set C whose distance to the plane Pchosen satisfy d < dth6: if 

npnc< ath2 then7:  Pchosen← Part 1(C, dth2)8: end if

Furthermore, the outliers of the chosen plane are selected when the distances d between the plane and points are larger than a threshold dth0. Based on the relative distances, the outliers are classified into different clusters. The 3D bounding box and point cloud information of each road defect will be output for the suspension-response-based evaluation.

The road input of a detected road defect is given by:

(8) q=1npBzp
where q is the road input, B is the point set of the tire contact area, n = card (B) denotes the number of the points in B and zp denotes the z coordinate of the position p.

The suspension response of the road defect is simulated using a quarter-car model. To assess the risk of a road defect, the weighted RMS value of the acceleration aw will be calculated as follows:

(9) aw=[1T0Taw2(t)dt]12

Different reference lane Yref,i and speed vref,i will be given based on the corresponding weighted RMS values of the road defects from the road condition monitoring, which is as follows:

(10) (Yref,i,vref,i)=Ψ(aw,i)
where Ψ(awi) is a mapping set of (Yref,i, vref,i) and awi.

3. Application

In this section, the functionality of the proposed unified ODD monitoring framework is demonstrated by the design and analysis of three typical SOTIF-related applications. Figure 4 illustrates the simulation structure used in the applications of the proposed framework. The MPC-based decision-making system is used to take the reference signals and state constraints from modules of the unified ODD monitoring framework and perform the motion control of autonomous vehicles. For brevity, the functions of the MPC-based decision-making system are not illustrated here, which can be found in Wang et al. (2019).

3.1 Application I – weather condition monitoring

In this application, the performance of the proposed unified ODD monitoring framework under rainy weather conditions is demonstrated. The initial speed of the autonomous vehicle is 15 km/h and targets at 70 km/h. The rain will occur in 10 s with an intensity of 2.5 mm/h.

To get the detectable distance, a rainfall experiment is conducted to obtain the sensor’s performance on rainy days. A whiteboard (reference whiteboard), a dummy and a vehicle are used as obstacles in the experiment, with distances of 15, 20 and 40 m, respectively, as shown in Figure 5. The performance of Lidar in different rain intensities is demonstrated in Table 1. For Lidar, the most critical parameter that will affect driving safety is actually the reflectance, which determines the maximal detectable distance. As illustrated in Table 1, the car has the smallest reflectance because of the complex shape and thick water layer formed by raindrops. To ensure safety, the data of the car is used to calculate the proposed speed limit. Table 2 demonstrates the final speed limit for various rain intensities using the proposed speed limit method.

Figure 6 demonstrates the variation of velocity under rainy weather conditions. The red line represents the speed limit calculated by real-time weather conditions, while the blue line denotes the running speed generated by the MPC controller. Because of the absence of rain at the beginning, the speed limit is 60 km/h (shown in Table 2). The vehicle accelerates to reach the reference speed (it equals 0.85 times the speed limit). The velocity of the autonomous vehicle reaches the reference value at 6 s with a slight overshoot. At 10 s, light rain begins, reducing the speed constraint to 42 km/h. The vehicle will decrease its speed to reach the new reference value. This application demonstrates that the autonomous vehicle can make a safe decision based on the performance degradation of Lidar on rainy days using the proposed monitoring framework.

3.2 Application II –vehicle behavior monitoring

In this application, the compliance performance of the proposed unified ODD monitoring framework in traffic rules is demonstrated in a SOTIF-related scenario. The scenario is stetted that an opposing-driving vehicle is approaching the ego vehicle on a two-way street with no centerline. And the surrounding vehicle is positioned relatively close to the right edge of the road, as is depicted in Figure 7.

According to Article 48 of the regulations for the implementation of the road traffic safety law of the People's Republic of China: when another motor vehicle approaches from the opposite direction on roads without central isolation facilities or central lines, the vehicle must slow down and drive to the right, maintaining the necessary safety distance from other vehicles and pedestrians (CHINA T C P G, 2005). Based on the proposed digitalization framework, the logical definitions of the example traffic rule are shown in Table 3.

Based on the above logical definitions, the trigger conditions are easily determined as the logical sum of ①, ②, ③ and ④. Moreover, definitions ⑤, ⑥ and ⑦ can surely be used as the constraints of the states and output to the MPC decision-making system.

With the proposed digitization framework, the performance comparison of the ego vehicle is demonstrated in Figure 8. T1 denotes the timestamps of the surrounding vehicle, whereas T denotes the timestamps of the ego vehicle. Without considering the traffic rules, the ego vehicle controlled by optimization-based MPC will choose the left side of the road to avoid colliding with the surrounding vehicle, as the artificial potential field values of the surrounding vehicle are primarily on the right side of the road. This may result in hazardous situations, as the human-driven surrounding vehicle may turn to the left side (from the perspective of the ego vehicle) to comply with the traffic rule. On the other hand, a vehicle that follows the traffic rules will choose the right side to avoid colliding with the surrounding vehicle, which complies with the traffic rule and mitigates the SOTIF risk in this critical scenario.

3.3 Application III – road condition monitoring

In this application, the performance of the proposed unified ODD monitoring framework under road defect conditions is demonstrated. An irregular protuberance is in front of the ego vehicle on the road, which may endanger the vehicle’s dynamics. The experiment vehicle is demonstrated in Figure 9, which is equipped with IPC, Nov Atel GPS, Velodyne Lidar and a binocular camera. The parameters of the proposed recognition and evaluation method are listed in Table 4.

A 2 m × 2 m grid is used to mesh the road, as demonstrated in Figure 10(a). With the modified RANSAC algorithm, the fitting plane of the road is demonstrated in Figure 10(b). After the outlier searching and clustering, the road defect is recognized, which is lighted in the point cloud information in Figure 10(c).

Table 5 summarizes the behavioral decision-making strategies under various aw,i. As the detected road defect would cause a hitting on the stop block, the avoidance and deceleration behavior is selected. The reference signal from the unified ODD monitoring module is shown in Figure 11. With recognizing and evaluating the road defect, the reference signal (Yref, vref) is changed from (1.75, 10) to (5.25, 6) at 1 s to avoid the road defect and slow down. The trajectory of the ego vehicle is demonstrated in Figure 12. As is depicted in Figure 12, the proposed unified ODD monitoring framework enables the ego vehicle to successfully monitor and avoid road defects.

4. Conclusion and future work

This paper proposed a unified ODD monitoring framework for autonomous vehicles to mitigate the SOTIF risk triggered by vehicles exceeding ODD boundaries. The major factors of ODD boundaries, as well as their mechanisms, were analyzed and categorized. For monitoring the corresponding category, the unified model of the proposed unified ODD monitoring framework contains three modules, including weather condition monitoring, vehicle behavior monitoring and road condition monitoring. For weather condition monitoring, an accurate safe speed limit method is proposed to handle the drastic performance degradation of sensors in rainy conditions. For vehicle behavior monitoring, a digitalization framework of traffic rules is proposed to ensure the traffic rule compliance of an autonomous vehicle. For road condition monitoring, a recognition and evaluation method of road defects is proposed to avoid hazards from the road conditions. Three SOTIF-related applications were constructed to validate the effectiveness of the proposed framework. The simulation results indicated that the proposed unified ODD monitoring framework can effectively mitigate the SOTIF risk in various conditions for autonomous vehicles.

In future work, more complicated scenarios with heterogeneous conditions will be investigated using the proposed framework. Additionally, the corresponding real-world vehicle road tests will be carried out.

Figures

The overall structure of the unified ODD monitoring framework

Figure 1

The overall structure of the unified ODD monitoring framework

The overview of the digitalization framework of traffic rules

Figure 2

The overview of the digitalization framework of traffic rules

The flowchart of the recognition and evaluation method of road defects

Figure 3

The flowchart of the recognition and evaluation method of road defects

The simulation structure of applications

Figure 4

The simulation structure of applications

Rainfall experiment

Figure 5

Rainfall experiment

The variation of velocity under the rainy weather condition

Figure 6

The variation of velocity under the rainy weather condition

The obstacle avoidance scenario

Figure 7

The obstacle avoidance scenario

The performance comparison of the ego vehicle

Figure 8

The performance comparison of the ego vehicle

The experiment vehicle

Figure 9

The experiment vehicle

The outcomes of the recognition process: (a) the meshing of the road, (b) the fitting plane of the road and (c) the point cloud information

Figure 10

The outcomes of the recognition process: (a) the meshing of the road, (b) the fitting plane of the road and (c) the point cloud information

Reference signals from the unified ODD monitoring module: (a) Y reference and (b) velocity reference

Figure 11

Reference signals from the unified ODD monitoring module: (a) Y reference and (b) velocity reference

The trajectory of the ego vehicle

Figure 12

The trajectory of the ego vehicle

The performance of Lidar in different rain intensities

Tele-15
Rain intensity Object type Average number of point clouds Average reflectance Reflectance information entropy
No rain White box 453 70 2.1
Dummy 613 17 2.1
Car 1583 14 1.43
Light rain (2.5 mm/h) White box 402 2 0.02
Dummy 732 8 0.03
Car 813 2 0.47
Medium rain
(8 mm/h)
White box 421 10 0.85
Dummy 716 9 0.37
Car 1147 7.9 0.52
Heavy rain (16 mm/h) White box 476 18 1.45
Dummy 697 10 0.94
Car 1239 9 0.61
Note:

*Tele-15 is a high-quality Lidar used in the rain experiment

The final speed limit in different rain intensities

Rain performance
Rain intensity
(mm/h)
Detectable
distance (m)
Friction
coefficient
Speed limit
(km/h)
0 174 0.8 60
2.5 90 0.41 41
8 105 0.378 43
16 120 0.348 47

The logical definitions of the example traffic rule

Semantic description Logical definition
1. Without central isolation facilities Cent_isoF_Flag!=1
2. Without central lines LineColor!=Yellow
3. Comes in the opposite direction vEgo×vTgt<0
4. Motor vehicle MotorVehicleFlag=1
5. Slow down vv0
6. Drive to the right yy0
7. Keep necessary safety distance (yEgoyTgt)2>0
&(xEgoxTgt)2>0

The parameters of the recognition and evaluation method

Parameters Value Parameters Value
dth1 0.01 m ath1 0.98
dth2 0.02 m ath2 0.9
dth0 0.02 m ath 16
dthg 0.15m athg 25

Behavioral decision-making strategies under different awi

Stop block No hitting on the stop block Hit on the stop block
Comfort level comfort A little discomfort discomfort Extreme discomfort
aw,i <0.315 0.315∼0.56 0.56∼2.25
Pass
Decelerate
Avoid
Avoid and decelerate

References

Bossu, J., Hautiere, N. and Tarel, J.-P. (2011), “ Rain or snow detection in image sequences through use of a histogram of orientation of streaks”, International Journal of Computer Vision, Vol. 93 No. 3, pp. 348-367.

Chen, Y., Lu, C. and Chu, W. (2020), “A cooperative driving strategy based on velocity prediction for connected vehicles with robust path-following control”, IEEE Internet of Things Journal, Vol. 7 No. 5, pp. 3822-3832.

Chu, L.J. and Fwa, T.F. (2018), “Pavement skid resistance consideration in rain-related wet-weather speed limits determination”, Road Materials and Pavement Design, Vol. 19 No. 2, pp. 334-352.

Huang, Y., et al. (2018), “A review of power management strategies and component sizing methods for hybrid vehicles”, Renewable and Sustainable Energy Reviews, Vol. 96 No. 1, pp. 132-144.

Huang, Y., et al. (2019), “A novel local motion planning framework for autonomous vehicles based on resistance network and model predictive control”, IEEE Transactions on Vehicular Technology, Vol. 69 No. 1, pp. 55-66.

ISO (2019), “Pas 21448-road vehicles-safety of the intended functionality”.

Karlsson, R. et al. (2021), “Probabilistic rainfall estimation from automotive lidar”, arXiv preprint arXiv:2104.11467.

Kuutti, S., et al. (2018), “A survey of the state-of-the-art localization techniques and their potentials for autonomous vehicle applications”, IEEE Internet of Things Journal, Vol. 5 No. 2, pp. 829-846.

Li, S., et al. (2018), “Ethical and legal dilemma of autonomous vehicles: study on driving decision-making model under the emergency situations of red light-running behaviors”, Electronics, Vol. 7 No. 10, p. 264.

Lin, D., et al. (2017), “RSCM: region selection and concurrency model for multi-class weather recognition”, IEEE Transactions on Image Processing, Vol. 26 No. 9, pp. 4154-4167.

Miller, R. and Huang, Q. (2002), “An adaptive peer-to-peer collision warning system. Vehicular technology conference”, IEEE 55th Vehicular Technology Conference, VTC Spring 2002 (Cat. No. 02CH37367). 2002. IEEE.

Ong, G.P. and Fwa, T.F. (2007), “Prediction of wet-pavement skid resistance and hydroplaning potential”, Transportation Research Record: Journal of the Transportation Research Board, Vol. 2005 No. 1, pp. 160-171.

Rasouli, A. and Tsotsos, J.K. (2019), “Autonomous vehicles that interact with pedestrians: a survey of theory and practice”, IEEE Transactions on Intelligent Transportation Systems, Vol. 21 No. 3, pp. 900-918.

Ren, K., et al. (2019), “The security of autonomous driving: threats, defenses, and future directions”, Proceedings of the IEEE, Vol. 108 No. 2, pp. 357-372.

Shalev-Shwartz, S. Shammah, S. and Shashua, A. (2017), “On a formal model of safe and scalable self-driving cars ”, arXiv preprint arXiv: 1708.06374.

She, X., et al. (2021), “Feasibility study of asphalt pavement pothole properties measurement using 3D line laser technology”, International Journal of Transportation Science and Technology, Vol. 10 No. 1, pp. 83-92.

Spinneker, R., et al. (2014),. “Fast fog detection for camera based advanced driver assistance systems”, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

Talamini, J., et al. (2020), “On the impact of the rules on autonomous drive learning”, Applied Sciences, Vol. 10 No. 7, p. 2394.

Wang, J., et al. (2018), “Networking and communications in autonomous driving: a survey”, IEEE Communications Surveys & Tutorials, Vol. 21 No. 2, pp. 1243-1274.

Wang, H., et al. (2019), “Crash mitigation in motion planning for autonomous vehicles”, IEEE Transactions on Intelligent Transportation Systems, Vol. 20 No. 9, pp. 3313-3323.

Wang, H., et al. (2020), “Ethical decision-making platform in autonomous vehicles with lexicographic optimization based model predictive controller”, IEEE Transactions on Vehicular Technology, Vol. 69 No. 8, pp. 8164-8175.

Wang, H., et al. (2022), “Ethical decision making in autonomous vehicles: challenges and research progress”, IEEE Intelligent Transportation Systems Magazine, Vol. 14 No. 1, pp. 6-17.

Wang, J., Wu, J. and Li, Y. (2015), “The driving safety field based on driver–vehicle–road interactions”, IEEE Transactions on Intelligent Transportation Systems, Vol. 16 No. 4, pp. 2203-2214.

Willers, O., et al. (2020), “Safety concerns and mitigation approaches regarding the use of deep learning in safety-critical perception tasks”, International Conference on Computer Safety, Reliability, and Security, Springer.

Wu, H., et al. (2019), “Road pothole extraction and safety evaluation by integration of point cloud and images derived from mobile mapping sensors”, Advanced Engineering Informatics, Vol. 42 No. 1, p. 100936.

Xia, J., et al.. (2020), “ResNet15: weather recognition on traffic road with deep convolutional neural network”, Advances in Meteorology, Vol. 2020.

Xiao, W., et al. (2021), “Rule-based optimal control for autonomous driving”, Proceedings of the ACM/IEEE 12th International Conference on Cyber-Physical Systems.

Yang, K., et al. (2021), “Comparative study of trajectory tracking control for automated vehicles via model predictive control and robust H-infinity state feedback control”, Chinese Journal of Mechanical Engineering, Vol. 34 No. 1, pp. 1-14.

Further reading

Berman, M., Liu, J. and Justison, L. (2009), Caltrans Fog Detection and Warning System, CA. Dept. of Transportation.

Rizaldi, A., et al. (2017), “Formalising and monitoring traffic rules for autonomous vehicles in Isabelle/HOL”, International conference on integrated formal methods, Springer.

Vanholme, B., et al. (2012), “Highly automated driving on highways based on legal safety”, IEEE Transactions on Intelligent Transportation Systems, Vol. 14 No. 1, pp. 333-347.

CHINA T C P G (2019), “The regulations for the implementation of the road traffic safety law of the people's republic of China[EB/OL]”.

Acknowledgements

The authors would like to appreciate the financial support of the National Key R&D Program of China (Grant No. 2020YFB1600303), the National Science Foundation of China Project: (Grant Nos. U1964203 and 52072215).

Corresponding author

Li-Ming Peng can be contacted at: alaric.peng@gmail.com

Related articles