Tunnel segments for large traffic tunnels are subject to the highest quality standards in the concrete precast industry. Given their large dimensions, it is inevitable that various defects occur during production or transportation. This paper presents a novel approach to the quality control of the segments at the latest possible location inside the tunnel boring machine (TBM) before the final ring erection. An artificial intelligence (AI) based computer vision (CV) software was developed to check images of the segments, with the images being collected by four cameras inside the TBM. The system “segment check” is able to detect missing dowels and misplaced seals, as well as cracks and spallings. By adopting various AI-based image processing modules, this work is on the forefront of applying AI in the tunnelling world. Results show high accuracy for a 3D pose estimation network of the segment, as well as for the segmentation and classification networks which are used to perform the quality inspection.