Modeling

Compare three influential segmentation architectures, then choose how much of a pretrained model to retrain. Focus on the required output, the information carried by skip connections, and validation performance on the driving dataset.

Segmentation architectures

FCN, U-Net, and Mask R-CNN are useful architectures to study, not a claim about the current leaderboard. Their original papers establish the ideas discussed here: FCN, U-Net, and Mask R-CNN.

FCN

Fully convolutional networks adapt image-classification features to dense, pixel-wise prediction.

Note: Segmentation is a dense prediction task of pixel-wise classification.

A typical FCN operates by fine-tuning an image classification CNN and applying pixel-wise training. It first compresses the information using multiple layers of convolutions and pooling. Then, it up-samples these feature maps to predict each pixel’s class from this compressed information.