Evaluation and design of DNN system for Automated Vehicle through smart sampling

Mcity research project with Prof.Ding Zhao

Time: Aug - current

Introduction:

A key to the success of automated roadway vehicles is to make the systems safe, reliable. There are various systems for autonomous vehicle control from traditional PID control to Neural Network system. Researchers developed systems for autonomous vehicle based on Neural Network: Pomerleau[1] built artificial neural networks for autonomous navigation in 1991. Ding [2] proposed a neural network for predicting driver’s lane change trajectory in 2013. Wang[3] built deep reinforcement learning architecture for on-ramp merge in 2017.

Among these different methods, we may come up with one questions: how to evaluate Deep Neural Network for autonomous vehicles and how to design a neural network architecture that could have better performance? This project will develop an algorithm for evaluating and designing Deep Neural Network for autonomous vehicle through smart sampling. The lane change (cut-in) scenario is used as an example to show the benefits of the proposed method.

Technical Approach：

Task 1: Build mathematical model for lane change scenario and Importance sampling

The basic mathematical model with definitions of variables and specific lane change scenario is needed for the further discussion and model-building[4]. This task aims to define a specific lane change model and observable variables such as velocities, locations, distance and acceleration.

Original mathematical model is very likely to get a biased crash estimate results due to non-ramdom sampling [5]. Importance sampling can skew back the biased estimates so that the statistical inferences are validly representing the real world characteristics.

Task 2: Build reinforcement model for autonomous car

This task aims to build a dynamic model for autonomous vehicle in lane change scenario, which can decide the optimal actions based on historical impact of interactive environment. This problem need to take states of other vehicles into consideration. Moreover, the vehicles’ state space and action space is continuous, which make this problem impractical to use basic Q-learning. We will build a Deep reinforcement learning model with Q-network as approximator and another Q-network for updating parameters. At each time step, this network will take observations (current autonomous vehicle speed, lane change vehicle speed, the distance between them) and last time step’s action (accelerator of autonomous vehicle) as input, then product the current action in Q-network and update parameters in Q-update-network.

Task 3: Evaluate Deep Neural Network

In order to evaluate different models, we need to find general metrics valid for different systems. For lane change scenario, the most important metric is safety. We can use crash possibility for thousands of simulations during same time period as the safety metric. On the other side, under real world situation, our autonomous vehicle may meet some corner cases, which may not show in our simulation. Therefore, we want our system to demonstrate potential for displaying various behaviors. Neuron Coverage can be another metric for evaluating deep neural networks. This task is the key to the entire project with two proposes: 1) we can compare deep neural networks with traditional PID controlled systems with safety metrics; 2) we can compare different deep neural networks with safety metrics and neuron coverage metrics.

Task 4: Design Deep Neural Network

This task aims to get a general standard for adjusting deep neural network architecture. We can make this problem to be a joint optimizing problem, which decrease the probability for crash and increase neuron coverage. Using the number of neurons of each fully connected layer and the number of fully connected layers as parameters to find the optimal architecture for joint optimizing problem.

The deliverables associated with these tasks are as follows:

• A framework to build a deep reinforcement learning model for lane change scenario;

• Algorithm to compare traditional PID control system with deep neural network for the same scenario;

• Algorithm to evaluate different deep neural networks;

• Algorithm to design architecture of deep neural network;

Reference:

[1] D. A. Pomerleau, "Efficient training of artificial neural networks for autonomous navigation," Neural Computation, vol. 3, no. 1, pp. 88-97, 1991.

[2] C. Ding, W. Wang, X. Wang, and M. Baumann, "A neural network model for driver’s lane-changing trajectory prediction in urban traffic flow," Mathematical Problems in Engineering, vol. 2013, 2013.

[3] P. Wang and C.-Y. Chan, "Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge," arXiv preprint arXiv:1709.02066, 2017.

[4] D. Zhao et al., "Accelerated evaluation of automated vehicles safety in lane-change scenarios based on importance sampling techniques," IEEE transactions on intelligent transportation systems, vol. 18, no. 3, pp. 595-607, 2017.

[5] Z. Huang, Y. Guo, H. Lam, and D. Zhao, "A Versatile Approach to Evaluating and Testing Automated Vehicles based on Kernel Methods," arXiv preprint arXiv:1710.00283, 2017.