Operator Reference

HALCON Operator Reference / 3D Reconstruction / Multi-View Stereo

• HDevelop
• .NET
• Python
• C++
• C

ClassesClasses | | Operators

reconstruct_points_stereoT_reconstruct_points_stereoReconstructPointsStereoReconstructPointsStereoreconstruct_points_stereo (Operator)

reconstruct_points_stereoT_reconstruct_points_stereoReconstructPointsStereoReconstructPointsStereoreconstruct_points_stereo — Reconstruct 3D points from calibrated multi-view stereo images.

Signature

reconstruct_points_stereo( : : StereoModelID, Row, Column, CovIP, CameraIdx, PointIdx : X, Y, Z, CovWP, PointIdxOut)

Herror T_reconstruct_points_stereo(const Htuple StereoModelID, const Htuple Row, const Htuple Column, const Htuple CovIP, const Htuple CameraIdx, const Htuple PointIdx, Htuple* X, Htuple* Y, Htuple* Z, Htuple* CovWP, Htuple* PointIdxOut)

void ReconstructPointsStereo(const HTuple& StereoModelID, const HTuple& Row, const HTuple& Column, const HTuple& CovIP, const HTuple& CameraIdx, const HTuple& PointIdx, HTuple* X, HTuple* Y, HTuple* Z, HTuple* CovWP, HTuple* PointIdxOut)

void HStereoModel::ReconstructPointsStereo(const HTuple& Row, const HTuple& Column, const HTuple& CovIP, Hlong CameraIdx, Hlong PointIdx, HTuple* X, HTuple* Y, HTuple* Z, HTuple* CovWP, HTuple* PointIdxOut) const

void HStereoModel::ReconstructPointsStereo(double Row, double Column, const HTuple& CovIP, Hlong CameraIdx, Hlong PointIdx, double* X, double* Y, double* Z, double* CovWP, Hlong* PointIdxOut) const

static void HOperatorSet.ReconstructPointsStereo(HTuple stereoModelID, HTuple row, HTuple column, HTuple covIP, HTuple cameraIdx, HTuple pointIdx, out HTuple x, out HTuple y, out HTuple z, out HTuple covWP, out HTuple pointIdxOut)

void HStereoModel.ReconstructPointsStereo(HTuple row, HTuple column, HTuple covIP, int cameraIdx, int pointIdx, out HTuple x, out HTuple y, out HTuple z, out HTuple covWP, out HTuple pointIdxOut)

void HStereoModel.ReconstructPointsStereo(double row, double column, HTuple covIP, int cameraIdx, int pointIdx, out double x, out double y, out double z, out double covWP, out int pointIdxOut)

def reconstruct_points_stereo(stereo_model_id: HHandle, row: MaybeSequence[Union[float, int]], column: MaybeSequence[Union[float, int]], cov_ip: Sequence[Union[float, int]], camera_idx: int, point_idx: int) -> Tuple[Sequence[float], Sequence[float], Sequence[float], Sequence[float], Sequence[int]]

def reconstruct_points_stereo_s(stereo_model_id: HHandle, row: MaybeSequence[Union[float, int]], column: MaybeSequence[Union[float, int]], cov_ip: Sequence[Union[float, int]], camera_idx: int, point_idx: int) -> Tuple[float, float, float, float, int]

Description

The operator reconstruct_points_stereoreconstruct_points_stereoReconstructPointsStereoReconstructPointsStereoreconstruct_points_stereo reconstructs 3D points from point correspondences found in the images of a calibrated multi-view stereo setup. The calibration information for the images is provided in the camera setup model that is associated with the stereo model StereoModelIDStereoModelIDStereoModelIDstereoModelIDstereo_model_id during its creation (see create_stereo_modelcreate_stereo_modelCreateStereoModelCreateStereoModelcreate_stereo_model). Note that the stereo model type must be 'points_3d'"points_3d""points_3d""points_3d""points_3d", otherwise the operator will return an error.

The point correspondences must be passed in the parameters RowRowRowrowrow, ColumnColumnColumncolumncolumn, CameraIdxCameraIdxCameraIdxcameraIdxcamera_idx, and PointIdxPointIdxPointIdxpointIdxpoint_idx in form of tuples of the same length. Each set (Row[I],Column[I],CameraIdx[I],PointIdx[I]) represents the image coordinates (RowRowRowrowrow, ColumnColumnColumncolumncolumn) of the 3D point (PointIdxPointIdxPointIdxpointIdxpoint_idx) in the image of a certain camera (CameraIdxCameraIdxCameraIdxcameraIdxcamera_idx).

The reconstructed 3D point coordinates are returned in the tuples XXXxx, YYYyy, and ZZZzz, relative to the coordinate system of the camera setup model (see create_camera_setup_modelcreate_camera_setup_modelCreateCameraSetupModelCreateCameraSetupModelcreate_camera_setup_model). The tuple PointIdxOutPointIdxOutPointIdxOutpointIdxOutpoint_idx_out contains the corresponding point indices.

The reconstruction algorithm works as follows: First, it identifies point correspondences for a given 3D point by collecting all sets with the same PointIdxPointIdxPointIdxpointIdxpoint_idx. Then, it uses the RowRowRowrowrow, ColumnColumnColumncolumncolumn, and CameraIdxCameraIdxCameraIdxcameraIdxcamera_idx information from the collected sets to project lines of sight from each camera through the corresponding image point [Row,Column]. If there are at least 2 lines of sight for the point PointIdxPointIdxPointIdxpointIdxpoint_idx, they are intersected and the result is stored as the set (X[J],Y[J],Z[J],PointIdxOut[J]). The intersection is performed with a least-squares algorithm, without taking into account potentially invalid lines of sight (e.g., if an image point was falsely specified as corresponding to a certain 3D point).

To compute the covariance matrices for the reconstructed 3D points, statistical information about the extracted image coordinates, i.e., the covariance matrices of the image points (see , e.g., points_foerstnerpoints_foerstnerPointsFoerstnerPointsFoerstnerpoints_foerstner) are needed as input and must be passed in the parameter CovIPCovIPCovIPcovIPcov_ip. Otherwise, if no covariance matrices for the 3D points are needed or no covariance matrices for the image points are available, an empty tuple can be passed in CovIPCovIPCovIPcovIPcov_ip. Then no covariance matrix for the reconstructed 3D points is computed.

The covariance matrix of an image point is:

The covariance matrices are symmetric 2x2 matrices, whose entries in the main diagonal represent the variances of the image point in row-direction and column-direction, respectively. For each image point, a covariance matrix must be passed in CovIPCovIPCovIPcovIPcov_ip in form of a tuple with 4 elements:

Thus, |CovIP|=4*|Row| and CovIP[I*4:I*4+3] is the covariance matrix for the I-th image point.

The computed covariance matrix for a successfully reconstructed 3D point is represented by a symmetric 3x3 matrix:

The diagonal entries represent the variances of the reconstructed 3D point in x-, y-, and z-direction. The computed matrices are returned in the parameter CovWPCovWPCovWPcovWPcov_wp in form of tuples each with 9 elements:

Thus, |CovWP|=9*|X| and CovWP[J*9:J*9+8] is the covariance matrix for the J-th 3D point. Note that if the camera setup associated with the stereo model contains the covariance matrices for the camera parameters, these covariance matrices are considered in the computation of CovWPCovWPCovWPcovWPcov_wp too.

If the stereo model has a valid bounding box set (see set_stereo_model_paramset_stereo_model_paramSetStereoModelParamSetStereoModelParamset_stereo_model_param), the resulting points are clipped to this bounding box, i.e., points outside it are not returned. If the bounding box associated with the stereo model is invalid, it is ignored and all points that could be reconstructed are returned.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

StereoModelIDStereoModelIDStereoModelIDstereoModelIDstereo_model_id (input_control)  stereo_model → HStereoModel, HTupleHHandleHTupleHtuple (handle) (IntPtr) (HHandle) (handle)

Handle of the stereo model.

RowRowRowrowrow (input_control)  number(-array) → HTupleMaybeSequence[Union[float, int]]HTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Row coordinates of the detected points.

ColumnColumnColumncolumncolumn (input_control)  number(-array) → HTupleMaybeSequence[Union[float, int]]HTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Column coordinates of the detected points.

CovIPCovIPCovIPcovIPcov_ip (input_control)  number-array → HTupleSequence[Union[float, int]]HTupleHtuple (real / integer) (double / int / long) (double / Hlong) (double / Hlong)

Covariance matrices of the detected points.

Default: []

CameraIdxCameraIdxCameraIdxcameraIdxcamera_idx (input_control)  number → HTupleintHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Indices of the observing cameras.

Suggested values: 0, 1, 2

PointIdxPointIdxPointIdxpointIdxpoint_idx (input_control)  number → HTupleintHTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Indices of the observed world points.

Suggested values: 0, 1, 2

XXXxx (output_control)  real(-array) → HTupleSequence[float]HTupleHtuple (real) (double) (double) (double)

X coordinates of the reconstructed 3D points.

YYYyy (output_control)  number(-array) → HTupleSequence[float]HTupleHtuple (real) (double) (double) (double)

Y coordinates of the reconstructed 3D points.

ZZZzz (output_control)  number(-array) → HTupleSequence[float]HTupleHtuple (real) (double) (double) (double)

Z coordinates of the reconstructed 3D points.

CovWPCovWPCovWPcovWPcov_wp (output_control)  number(-array) → HTupleSequence[float]HTupleHtuple (real) (double) (double) (double)

Covariance matrices of the reconstructed 3D points.

PointIdxOutPointIdxOutPointIdxOutpointIdxOutpoint_idx_out (output_control)  number(-array) → HTupleSequence[int]HTupleHtuple (integer) (int / long) (Hlong) (Hlong)

Indices of the reconstructed 3D points.

Alternatives

reconstruct_surface_stereoreconstruct_surface_stereoReconstructSurfaceStereoReconstructSurfaceStereoreconstruct_surface_stereo, intersect_lines_of_sightintersect_lines_of_sightIntersectLinesOfSightIntersectLinesOfSightintersect_lines_of_sight

Module

3D Metrology

---

ClassesClasses | | Operators