trainYOLO has been the go-to platform for object detection, instance segmentation, keypoint detection, and human pose estimation. Yet, there was one essential task that was missing from our arsenal: semantic segmentation. But worry not, as we are excited to announce the integration of an easy-to-use semantic segmentation interface into our platform. And if that’s not all – we are also excited to share that we have open-sourced Segment, a semantic segmentation library developed in PyTorch and integrated with our platform. With this integration, you can effortlessly train popular segmentation models like Unet or Deeplab from scratch immediately after labeling your images. And as usual, you can upload your trained models directly to our platform, and take advantage of our built-in model-assisted labeling to boost your labeling speed.

What is Semantic Segmentation?

Semantic segmentation is a computer vision technique that assigns a semantic label to each pixel in an image. Unlike instance segmentation, there is no distinction between different objects. Semantic segmentation is frequently used in the following industries:

• Autonomous driving: Semantic segmentation plays a crucial role in understanding the surrounding environment like free space and road boundaries.
• Medical Imaging: In medical images, semantic segmentation is used for analyzing and interpreting medical scans, such as MRIs, CT scans and histopathology images.
• Robotics: Semantic segmentation is utilized in robotics for various applications, but most importantly for perceiving the environment.
• Agtech: In agtech, semantic segmentation is often used for crop segmentation.
• Satellite Imagery and Remote Sensing: Semantic segmentation is employed in satellite imagery analysis to identify and classify land cover, vegetation, water bodies, buildings, roads, and other geographical features.
• Augmented and Virtual Reality: Semantic segmentation is used in AR and VR applications for real-time scene understanding and virtual object placement. By segmenting the scene into meaningful regions, it enables virtual objects to be seamlessly integrated into the real-world environment.

Labeling interface

The trainYOLO platform offers a variety of annotation tools to suit different labeling requirements.

a) Brush Tool: With the brush tool, annotators can easily label or remove regions of interest. With its adaptive size, it is easy to use on both small and large regions and ideal to make small corrections in combination with either SAM or model-assisted labeling. 

b) Polygon Tool: The polygon tool provides an easy way to label regions with clearly defined contours. It enables annotators to create accurate shapes that align with the object boundaries, ensuring precise masks.

c) SAM (Segment Anything Model): Our integration of Meta’s Segment Anything Model makes it easy to label regions with a few clicks. If more precision is required, you can easily add or remove the mask by making use of the brush tool.

Segment library: easily train UNet, DeepLab, and other Segmentation models

‍

To lower the threshold to get started with semantic segmentation, we are releasing our Segment library, a semantic segmentation training framework written in PyTorch. So after labeling your images, you can directly train popular semantic segmentation networks like UNet or DeepLab(V3+) with all sorts of encoders like Resnet, Resnext, MobileNet, EfficientNet, and even the recently released Mix Vision transformer. This seamless integration eliminates the need for complex setup procedures, enabling users to focus on model development and improvement.

‍

More importantly, is that you can upload your trained segmentation models to our platform for model-assisted labeling. By leveraging the predictions from your trained models, you can significantly reduce annotation time and effort. Annotators can review and refine the model predictions, ensuring high-quality annotations while boosting the labeling process.

Conclusion

We have added semantic segmentation to the trainYOLO platform and released our Segment library, which makes it easy to train segmentation models on your labeled images.

‍