Text SAM AI Model

Text SAM is an open-source geospatial AI solution that extracts and segments objects in images based on free-form text prompts. It leverages Grounding DINO and the Segment Anything Model (SAM) to convert text prompts to geographic information system (GIS) features. With this technology, people can recognize the objects they need in various types of geospatial imagery without being limited to predefined object classes.

The work process starts with Grounding DINO, an open-set object detection model that recognizes the input text prompt and finds objects matching the text prompt in an image. Grounding DINO does not need to know the objects in advance because it detects the objects depending on their description provided by the user. The found objects are marked as bounding boxes showing the approximate place and extent of each object.

Then the bounding boxes are sent to the Segment Anything Model (SAM). With the help of these detected bounding boxes and some user-specified points as prompts, SAM provides image segmentation and generates the masks around objects.

The segmented objects are then processed and made into vector polygons, where the objects are then usable as GIS-ready vector objects. These polygons may then be integrated into various geospatial workflows, such as mapping and spatial analysis, among others.

Text SAM may be utilized to extract a diverse range of objects from images, such as cars, swimming pools, ships, aircraft, solar panels, buildings, roads, trees, and many others that are visually apparent in the imagery. All that users need to do is provide the model with the appropriate text prompt for the object to be detected and extracted.

Through the use of natural language prompting, open-set object detection, image segmentation, and vectorization, Text SAM is a streamlined method of automated geospatial feature extraction. This method may be useful in reducing manual digitization efforts and improving mapping workflows, especially when analyzing large quantities of aerial or satellite imagery.