OpenMVG Tutorial: Build 3D Models from Images

Image-based 3D reconstruction is one of the essential methods that are widely used in the field of computer vision, photogrammetry, robotics, and geospatial data analysis. The process allows one to calculate the camera locations, estimate the geometry of scenes, and create 3D reconstructions based on overlapping 2D photos. OpenMVG is an open-source computer vision library that was developed to perform these tasks by means of multiple-view geometry, feature matching, camera calibration, and SfM.

The capabilities of the OpenMVG library include estimation of camera locations and reconstruction of the sparse 3D point cloud based on images. In combination with MVS software, like OpenMVS, the sparse reconstruction may be further converted into a dense point cloud, textured mesh, etc.

OpenMVG

What Is OpenMVG?

OpenMVG is a library for multiple view geometry and image-based 3D scene reconstruction. It contains algorithms and command-line programs to process images, extract local features, match correspondences, estimate camera parameters, and compute the 3D scene structure.

OpenMVG is very handy for studying and implementing Structure from Motion pipelines. SfM reconstructs the 3D coordinates of the points in the scene and the positions of the cameras that took the pictures. As the reconstruction is typically incomplete, dense reconstruction software is needed to create a full picture.

OpenMVG can be used for:

• Photogrammetry. Reconstruction of objects, buildings, and environments from sets of overlapping photos.

• Robotics/Autonomy. Camera pose estimation and visual environment mapping.

• Documentation of cultural heritage. Scanning of monuments, objects, and architectural constructions.

• Aerial image-based mapping. Reconstruction of camera positions and scene geometry using drone-based image collections.

• Computer Vision research. Feature detector evaluation, image matching, camera model estimation, geometric estimation.

• 3D asset creation. Scene geometry reconstruction for visualization, simulation, and digital content generation.

OpenMVG is mostly focused on the geometric reconstruction part of the pipeline. It cannot generate a full and metrically correct 3D textured scene model by itself from an arbitrary set of images.

Understanding the OpenMVG Reconstruction Pipeline

A standard workflow of OpenMVG usually entails steps like pre-processing of images, extracting features from the images, matching the features, verifying geometry, reconstructing cameras, and sparse structure estimation. Dense reconstruction is normally done by a downstream compatible tool.

1. Image Acquisition and Preprocessing

Input images play a big role in determining the success of reconstruction. The images have to have enough overlap, have stable features, and be well-exposed.

In the case of object reconstruction, the images must be captured from different angles relative to the object. In the case of aerial photography, systematic flights are needed.

Significant points to be considered during the process of acquisition are as follows:

• Image Overlap: It is important that there should be adequate overlap between neighboring images.

• Sharpness: Avoid motion blur and defocused images that reduce feature localization accuracy.

• Texture: Images should have corners and edges; uniform textures are hard to reconstruct.

• Exposure Variation: Avoid variations in exposure.

• Stability of Camera: The information of cameras should be recorded; the variation in focal length should be avoided.

• Motion of Rolling Shutter: High-speed movement of cameras leads to the geometric distortion of images.

Images containing repetition, reflection, transparency, motion, and large homogeneous areas cause ambiguity in feature correspondence.

2. Feature Detection and Description

The local image feature extraction carried out by OpenMVG allows the recognition of features in several photos. Local features usually refer to some specific parts of an image such as blobs, corners, and textures.

The feature detector detects candidate positions, while the descriptor describes the local appearance around the detected positions. Descriptors make it possible to find corresponding features in the other images in the pipeline.

Depending on the chosen configuration and implementation, the algorithms that can be used for feature extraction are SIFT and other methods compatible with the local features approach.

For image (I_i), the following formula represents the output of the feature extraction stage:

[

F_i={(x_{ij},d_{ij})}_{j=1}^{N_i}

]

where:

(x_{ij}) stands for the image coordinates of the keypoint (j) in image (i).

(d_{ij}) is the descriptor for that keypoint.

(N_i) is the number of features detected in the image.

The quality of features is more important than their amount. A high amount of unreliable and repetitive features would not contribute to the improvement of the result.

3. Feature Matching Across Images

Feature matching locates possible correspondences between images using descriptor distances and filtering techniques that eliminate potential ambiguities.

The typical approach makes use of the distance ratio test of nearest neighbors. In case (d_1) and (d_2) represent the distances to the nearest and the second nearest descriptor correspondences, respectively, a correspondence may be selected when:

[

\frac{d_1}{d_2}<\tau

]

Here, (\tau) denotes a user-defined threshold. In general, a smaller threshold yields more selective correspondences, even though the optimal value varies depending on the images and the descriptors.

Matching each image with all others entails roughly:

[

\frac{N(N-1)}{2}

]

comparisons for (N) images. Therefore, exhaustive matching can be quite costly in case of big image sets.

In order to deal with large data sets, sequential, GPS-guided, or other kinds of image pair selection approaches may be applied, depending on the image sequence, available metadata, and overlapping scenes.

4. Geometric Verification Using Robust Estimation

The similarity in descriptors does not prove that the two observations belong to the same physical entity. OpenMVG uses geometric verification to rule out any mismatches between pairs of feature matches.

For calibrated cameras, the association between corresponding points can be stated in terms of the essential matrix:

[\mathbf{x}_2^T\mathbf{E}\mathbf{x}_1=0]

where (\mathbf{x}_1) and (\mathbf{x}_2) represent corresponding points in normalized camera coordinates.

For uncalibrated image coordinates, the epipolar constraint between corresponding points is stated in terms of the fundamental matrix:

[\mathbf{p}_2^T\mathbf{F}\mathbf{p}_1=0]

Robust estimation techniques, such as the RANSAC algorithm, can help in estimating the geometrical relationship between two camera images along with ruling out any outliers.

5. Structure-from-Motion and Camera Pose Estimation

Structure-from-Motion determines the camera pose and three-dimensional scene points using multiple overlapping images.

For a reconstructed point (\mathbf{X}_j), its projection on image (i) can be represented as:

[

\tilde{\mathbf{x}}_{ij}\sim

\mathbf{K}_i

\left[

\mathbf{R}_i\mid\mathbf{t}_i

\right]

\tilde{\mathbf{X}}_j

]

where:

(\mathbf{K}_i) is the intrinsic calibration matrix.

(\mathbf{R}_i) is the camera rotation matrix.

(\mathbf{t}_i) is the camera translation vector.

(\tilde{\mathbf{X}}_j) is the three-dimensional point in homogeneous coordinates.

(\tilde{\mathbf{x}}_{ij}) is its projection in the image plane.

The process estimates camera poses incrementally or in a global manner, triangulating corresponding observations from images and recovering 3D points.

Since image-based reconstruction without external scaling information is essentially a process of determining a geometric configuration only up to a similarity transformation, the resulting model has an arbitrary origin, orientation, and scale. Known camera poses, ground control points, measured distances, or any other type of external information may be used to define a geographic coordinate system.

6. Bundle Adjustment and Sparse Reconstruction Refinement

Bundle adjustment optimizes both camera parameters and three-dimensional points simultaneously by solving for the minimum reprojection error of images.

A more simplified optimization objective function is given below:

[

\min_{\mathbf{R},\mathbf{t},\mathbf{X}}

\sum_{i,j}

\rho\left(

\left|

\mathbf{x}_{ij}-

\pi(\mathbf{K}_i,\mathbf{R}_i,\mathbf{t}_i,\mathbf{X}_j)

\right|^2

\right)

]

where (\pi) is the projection function of the camera and (\rho) is a loss function that could be applied.

Bundle adjustment could solve some inconsistencies in geometry through the process of refining the reprojection error. However, bundle adjustment does not make up for poor geometry in the images, miscalibrations, and wrong correspondences.

A set of sparse 3D points will be the output, with optimized camera pose and calibration parameters, which will serve as a basis for dense reconstruction.

OpenMVG Tutorial: Building a 3D Model Step by Step

The following workflow demonstrates the main command-line stages of OpenMVG. Executable names, available options, and file locations may vary by release, build configuration, and operating system. Verify the commands against the installed version before running the pipeline.

Step 1: Install OpenMVG

OpenMVG can be built from source or installed using a compatible prebuilt package when available. Building from source provides greater control over dependencies and enabled features.

Typical dependencies include:

• A supported C++ compiler and build system.

• CMake for configuring the build.

• Eigen for linear algebra.

• An image-processing library and other dependencies required by the selected build configuration.

Clone the official repository:

git clone https://github.com/openMVG/openMVG.git
cd openMVG

Initialize and retrieve any required submodules according to the repository's instructions, then configure and compile the project using the documented build process for your platform.

For example, a CMake-based build may begin with:

cmake -S. -B build
cmake --build build --config Release

These commands illustrate the general build pattern. Some OpenMVG versions require additional configuration flags, dependencies, or build-directory conventions. Consult the official repository documentation before compiling.

After installation, locate the generated OpenMVG command-line binaries and

make them accessible to your shell or scripts.

Step 2: Organize the Input Images

Create a directory containing the images to be reconstructed:

project/
├── images/
│   ├── image_001.jpg
│   ├── image_002.jpg
│   ├── image_003.jpg
│   └── image_004.jpg
└── reconstruction/

Use images from the same scene and ensure that neighboring photographs have adequate overlap. If the camera's focal length or imaging configuration changed during capture, account for those differences in camera calibration.

For reproducible processing, preserve the original image files and store processing outputs separately.

Step 3: Initialize the OpenMVG Scene

OpenMVG's openMVG_main_SfMInit_ImageListing application prepares the image listing and initializes the scene description.

A representative command is:

openMVG_main_SfMInit_ImageListing \
  -i ./project/images \
  -o ./project/reconstruction/matches \
  -d /path/to/sensor_width_database.txt

The sensor database path is illustrative and must point to a valid file when required by the selected camera-initialization workflow. Some configurations can use explicitly supplied camera parameters or other initialization options.

This stage identifies the input images, associates camera metadata where available, and creates the scene structure used by subsequent processing steps.

Check the generated scene file and verify that camera models, image dimensions, and intrinsic parameters are reasonable before proceeding. Incorrect focal lengths or sensor information can cause systematic geometric errors.

Step 4: Extract Local Features

Run the feature-extraction application:

openMVG_main_ComputeFeatures \
  -i ./project/reconstruction/matches/sfm_data.json \
  -o ./project/reconstruction/matches

This step computes keypoints and descriptors for the images and saves the associated feature data.

Feature-extraction options depend on the OpenMVG version and selected descriptor implementation. For large image collections, monitor processing time and memory usage. High-resolution imagery can substantially increase descriptor storage requirements.

Step 5: Match Image Features

Compute putative feature correspondences:

openMVG_main_ComputeMatches \
  -i ./project/reconstruction/matches/sfm_data.json \
  -o ./project/reconstruction/matches

This command represents the matching stage. Specific matching modes, geometric verification settings, and output files may differ by release.

Inspect the match results and look for a sufficient number of geometrically consistent correspondences between overlapping images. Weak connectivity between image pairs can prevent the reconstruction from growing into a single coherent scene.

Step 6: Recover Camera Poses and Sparse 3D Geometry

Use the incremental Structure-from-Motion application:

openMVG_main_IncrementalSfM \
  -i ./project/reconstruction/matches/sfm_data.json \
  -m ./project/reconstruction/matches \
  -o ./project/reconstruction/sfm

The incremental reconstruction estimates an initial camera configuration, registers additional views, triangulates scene points, and refines the solution.

The exact command-line interface and required arguments should be checked against the installed build. OpenMVG also provides other reconstruction strategies, and the most appropriate method depends on camera overlap, scene complexity, and initialization quality.

After processing, inspect the sparse model and registered camera count. A successful process exit does not necessarily mean that every image was registered or that the reconstruction is geometrically reliable.

Step 7: Evaluate the Sparse Point Cloud

Render the reconstructed scene via a compatible viewer or export the sparse geometry into a format readable by your post-processing software.

Look out for:

• Disconnected camera groups.

• An abundance of unconnected points.

• Unrealistic camera angles.

• Distortion in geometry.

• Reprojection errors that are too high.

• Inconsistent scale or orientation in relation to known information.

A sparse point cloud can help verify the geometry of the scene, but it rarely includes enough detail to create an accurate 3D model.

Step 8: Dense Reconstruction Model

In order to create a dense model, transfer the reconstructed scene and its camera data into a Multi-View Stereo pipeline. One of the tools you can use here is OpenMVS.

The dense reconstruction procedure generally looks as follows:

1. Creating a dense reconstruction scene based on the data provided by OpenMVG.

2. Calculating depth maps using multiple views.

3. Fusing multiple depth maps into a dense point cloud.

4. Surface reconstruction.

5. Creating a texture map using input images.

This particular conversion utility and commands depend on the versions of OpenMVG and OpenMVS you are using. Make sure that camera intrinsics, extrinsics, image locations, and coordinate systems are correctly transferred in the process.

Dense reconstruction depends on image overlap, texture, exposure, occlusion, and quality of calibration. More images will not give you a better reconstruction result in case there is blur, parallax issues, or viewpoint inconsistencies.

Step 9: Exporting and Validating the Final Model

The final output can comprise the dense point cloud, triangle mesh, texture, and camera information depending on the reconstruction software that follows the process. Popular exchange file formats used include PLY and OBJ, among others that the chosen tool suite supports.

For practical application, the model should be tested against independent measurements and data. When the goal of the reconstruction is geospatial applications, check the coordinate reference system, scale, orientation, and vertical datum.

A realistic-looking model does not guarantee quality or survey precision. It is important to quantify the model’s accuracy prior to applying the output to any decisions or calculations.

The OpenMVG project gives an adaptable, open-source framework for estimating the three-dimensional structure of a scene using sets of overlapping images. This process includes detecting features in an image, matching descriptors between different images, verifying correspondences geometrically, estimating the camera pose, doing triangulation, and adjusting the camera parameters through bundle adjustment.

When OpenMVG is combined with densification software like OpenMVS, one can take the output of the sparse reconstruction process and convert it into a full-fledged point cloud, mesh, and even textured 3D model. Accurate results hinge upon proper imaging, camera calibration, good correspondences, parallax, and independent verification.

For programmers and researchers interested in computer vision, drone mapping, photogrammetry, and geospatial analysis, OpenMVG is a handy way to learn the underlying mathematics of image-based 3D reconstruction.

To learn more about OpenMVG and its geospatial capabilities, click here.

For more information or any questions regarding OpenMVG, please don't hesitate to contact us at

Email: info@geowgs84.com

USA (HQ): (720) 702–4849

GeoWGS84AI

(A GeoWGS84 Corp. Company)

https://www.geowgs84.ai

https://www.geowgs84.com/services/deep-learning-with-geospatial-data

Visit www.lizardtech.com - The Creators of the MrSID Format