Project Details
Summary:
Self-driving Cars: Semantic Segmentation
Scene understanding is a critical capability for autonomous vehicles. It enables them to interpret and respond to dynamic environments. This project focuses on precise identification of objects in dynamic environments using image segmentation. We will study and train semantic segmentation neural networks to identify key objects in driving scenes, such as roads, pedestrians, vehicles, and more.
Description:
This project focuses on semantic segmentation for self-driving cars, aiming to accurately identify and classify key objects in driving environments. We will work with a suite of segmentation models, including SegNet, FRRN, MobileNets, DeepLab, Encoder-Decoder, DenseNets, and more, to process and analyze real-world driving data. By applying these models to video data captured from vehicle-mounted sensors, participants will develop a deeper understanding of image processing and computer vision techniques used in modern transportation technology.
Environment
Training a semantic segmentation model in this project is computationally intensive and requires an Nvidia GPU and a Linux operating system. A Docker container will be used to ensure a consistent and efficient training environment.
- Install Docker and verify GPU support: https://saturncloud.io/blog/how-to-use-gpu-from-a-docker-container-a-guide-for-data-scientists-and-software-engineers/
Dataset
- CityScapes Dataset: https://www.cityscapes-dataset.com
Register and request to download the following:
- gtFine_trainvaltest.zip (241MB)
(fine annotations for train and val sets (3475 annotated images) and dummy annotations (ignore regions) for the test set (1525 images))
- leftImg8bit_trainvaltest.zip (11GB)
(left 8-bit images - train, val, and test sets (5000 images))
Codebase
- Semantic Segmentation Suite for Self-driving Cars: https://github.com/Lingrui/self_driving_car/tree/master
References
SegNet: https://arxiv.org/abs/1511.00561
MobileNets: https://arxiv.org/abs/1704.04861
FRRN: https://arxiv.org/abs/1611.08323
Encoder-Decoder: https://arxiv.org/abs/1802.02611
DenseNets: https://arxiv.org/abs/1611.09326
Project Prepared By:
Kaiman Zeng, Ph.D., Assistant Professor of Computer Science at California State University, Stanislaus