Creation of Coco Text and MLT
Segmentation Datasets

For a complete overview of the datasets generation procedure please refer to our research publications:

COCO_TS Dataset: Pixel–Level Annotations Based on Weak Supervision for Scene Text Segmentation (arXiv preprint arXiv:1904.00818).
Weak Supervision for Generating Pixel-Level Annotations in Scene Text Segmentation (https://doi.org/10.1016/j.patrec.2020.06.023).

Overview

The COCO_TS, the MLT_S and the Incidental Scene Text Segmentation datasets provide pixel–level supervisions for the COCO–Text, the MLT and the Incidental Scene Text dataset, respectively. The supervision is obtained from the available bounding–boxes annotation exploiting a weakly supervised algorithm.

The supervision generation consists of two different steps.

A background–foreground network is trained on synthetic data to extract text from bounding–boxes.
The background–foreground network is then employed to generate pixel–level supervisions for real images of the COCO-Text, the MLT and the Incidental Scene Text datasets.

A background–foreground network is trained to segment the text inside a bounding–box. To train this network 1,000,000 bounding–box crops have been extracted from a synthetic dataset. After the training phase, the background–foreground network is applied on the bounding–boxes extracted from COCO-Text, MLT and Incidental Scene Text datasets. For each image, the pixel–level supervision is obtained combining the probability maps (calculated by the background–foreground network) for all the bounding–boxes inside the image. To provide a more reliable supervision, an uncertainty region is defined, based on the network probability maps. Furtermore, bounding–boxes that are not labeled as legible, machine printed and written in English have been added to the uncertainty region.

Note that images that only contain regions labeled as uncertain are not considered in the supervision generation procedure.