Non-Rigid 3D Reconstruction refers to the process of reconstructing the geometric information and dynamic evolution of 3D objects with time-variability, deformability, and translucency from multi-view image sequences. It holds significant value in scientific research and industrial applications: In fluid mechanics, it is used to reconstruct the 3D shapes of high-speed water jets and sprays, serving to verify the accuracy of computational fluid dynamics simulations. In biomechanics, it enables the reconstruction of soft tissue movements and simulation of blood flow, providing a 3D visualization tool for medical research. In industrial engineering, it reconstructs flames, electric arcs, and fluids, facilitating the optimization of process parameters such as spray combustion, welding processes, and fluid machinery, thereby improving energy efficiency and safety.
However, this method essentially still relies on the texture of the non-rigid surface for image feature matching. Thus, it is suitable for biological soft tissues such as skin and faces, but cannot reconstruct non-rigid objects without texture features, such as fluids and smoke, or those that are transparent or luminous.
To overcome the above limitations, R&D engineers at HF Agile Device Co.,Ltd proposed an improved multi-view non-rigid 3D reconstruction system, innovatively incorporating the Synchronous Multiplicative Algebra Reconstruction Technique (SMART). This technology no longer depends on local feature matching. Instead, it divides the 3D space into a voxel grid, assuming each voxel has a certain emission or scattering intensity. By treating multi-view 2D images as projections of this 3D field in different directions, it inverts the voxel intensity distribution to ultimately obtain the 3D structure, solving the problem of reconstructing non-rigid objects without textures.
Hardware Components of Revealer's Improved Multi-View Non-Rigid 3D Reconstruction System:
High-speed camera acquisition array: Composed of 4 Revealer ultra-high-definition high-speed cameras (Model G2110_Pro) with a specification of 5120×4096 @1000fps. These cameras are arranged in a surrounding manner to ensure stereo view coverage of the non-rigid object.
Synchronous trigger system: Achieves nanosecond-level synchronous exposure through B-code timing, ensuring that the multi-view image sequences captured by the high-speed cameras are strictly aligned to a specific physical moment.
Calibration system: Completes the joint calibration of internal and external camera parameters using a chessboard or spherical calibration target, establishing a pixel-voxel mapping relationship to provide a geometric basis for 3D inversion.
GPU computing platform: Used to execute the SMART algorithm, supporting large-scale parallel computing in voxel space.
Technical Implementation Steps of Revealer's Improved Multi-View Non-Rigid 3D Reconstruction System:
(1) The 4 high-speed cameras perform synchronous exposure to acquire instantaneous observation datasets.
(2) Define the 3D reconstruction space as a regular voxel grid, and initialize each voxel with a non-negative intensity value.
(3) For each high-speed camera, establish a projection mapping from the voxel space to the image plane.
(4) Adopt the Synchronous Multiplicative Algebra Reconstruction Technique (SMART) to iteratively optimize the initial voxel values, correcting projection errors round by round to approach the real voxel distribution.
(5) After the voxel intensity field converges, extract voxels with intensity higher than the threshold to generate the initial 3D point cloud. Then, use the Poisson surface reconstruction algorithm to process the sparse point cloud, fitting it into a smooth closed surface, and further generating a 3D model with correct topology.
Compared with extended MVS technologies, Revealer's improved multi-view non-rigid 3D reconstruction technology has distinct characteristics in terms of applicable objects, dependence on prior knowledge, physical authenticity, temporal resolution, and reconstruction accuracy:
| Aspect | Revealer SMART Technology | Extended MVS |
|---|---|---|
| Applicable Objects | Does not rely on texture features; suitable for non-rigid objects such as fluids, flames, and particle swarms (translucent, no surface texture) | Only supports non-rigid objects with rich surface textures (e.g., soft tissues); completely ineffective for fluids and smoke |
| Prior Model | No prior model required; only calibration of the imaging system is needed | Relies on the assumption of motion smoothness |
| Physical Authenticity | High-speed cameras directly capture real physical transient processes | Limited to reconstructable regions |
| Temporal Resolution | Resolves microsecond-level dynamics through high-speed cameras | Quasi-static; does not support millisecond-level deformation |
| Reconstruction Accuracy | Voxel-level resolution, up to sub-millimeter level | Achieves pixel-level accuracy only in local areas with rich texture features |
In summary, the non-rigid 3D reconstruction technology incorporating synchronous multiplicative algebra proposed by HF Agile Device Co.,Ltd is suitable for 3D reconstruction tasks of high-speed, high-dynamic non-rigid objects that are textureless and transparent.
In a typical application case of reconstructing the 3D shape of high-speed water jets (a textureless, transparent non-rigid object), engineers at HF Agile Device Co.,Ltd used 4 Revealer high-speed cameras to synchronously capture capillary waves on the water column surface, initial breakage points, and droplet splashing. By applying the SMART technology, they successfully reconstructed the 3D structure of the main water column and the droplets generated by the breakage of its edges.
This article introduces a non-rigid 3D reconstruction method based on the Synchronous Multiplicative Algebra Reconstruction Technique (SMART). By transforming the problem into an inversion problem in voxel space and adopting MLOS initialization and SMART iterative optimization, it overcomes the limitations of the extended MVS technical path in reconstructing high-speed, dynamic non-rigid objects without textures (such as fluids, smoke, and arc plasmas) or those that are translucent. This technology holds significant application value in fields such as fluid mechanics research, combustion diagnosis, environmental smoke diffusion analysis, and high-speed industrial monitoring.
