CarlaDataset: A PyTorch Dataset for Autonomous Driving with Carla Simulator

This code implements a CarlaDataset class in PyTorch, designed to handle data from the Carla simulator for autonomous driving applications. The dataset includes various modalities like images, depth maps, semantic segmentations, and more. This class inherits from torch.utils.data.Dataset, making it readily usable with PyTorch's data loading mechanisms.

Initialization:

When initializing CarlaDataset, you need to provide:

• root_dir: The root directory where your Carla dataset is stored.
• is_train: Whether you are loading the training or validation set.
• cfg: A configuration object containing parameters like receptive field size, future frame count, and sampling settings.

Preprocessing:

The dataset preloads data from the specified directory. It also includes several helper functions for:

• Image normalization: Applying standard normalization to images using torchvision's Normalize transform.
• Depth normalization: Converting depth values to a standardized format.
• HD map loading: Loading and processing high-definition map information.
• Label processing: Extracting segmentation and pedestrian labels from top-down views.
• Trajectory sampling: Generating trajectory samples based on vehicle velocity and steering.
• Camera parameter extraction: Obtaining camera intrinsics and extrinsics.

Data Loading (__getitem__):

The __getitem__ method handles data retrieval for a given index. It retrieves sequences of:

• Images: Front, left, right, and rear cameras.
• Depths: Corresponding depth maps for each camera view.
• Semantic segmentation: Segmentation labels from top-down views.
• Pedestrians: Pedestrian labels from top-down views.
• Extrinsics: Camera-to-ego transformations.
• Intrinsics: Camera intrinsic parameters.
• HD map: High-definition map information.
• Ground truth trajectory: Future trajectory of the vehicle.
• Target point: The goal point for the vehicle to navigate towards.
• Control signals: Steering, throttle, brake, and command information.
• Future egomotion: Relative motion of the vehicle in the future.
• Sampled trajectory: Hypothetical trajectories based on control inputs.

Helper Functions:

The code includes several utility functions for:

• Scaling and cropping images: Resizing and cropping images to the desired size.
• Transforming 2D points: Transforming points between different coordinate frames.

Example Usage:

import torch.utils.data as data
from carla_dataset import CarlaDataset

# Load the dataset
dataset = CarlaDataset(root_dir='/path/to/dataset', is_train=True, cfg=config)

# Create a data loader
data_loader = data.DataLoader(dataset, batch_size=32, shuffle=True)

# Iterate over the data loader
for batch in data_loader:
    # Access the data
    images = batch['image']
    depths = batch['depths']
    # ...

Key Features:

• Efficient loading of Carla simulator data for autonomous driving.
• Multi-modal data handling including images, depth, segmentation, and controls.
• Preprocessing steps for normalization and transformation.
• Integration with PyTorch for seamless use in training models.
• Trajectory generation and sampling capabilities.
• Helper functions for image processing and coordinate transformations.

This dataset serves as a valuable tool for training and evaluating autonomous driving models using data collected from the Carla simulator.