camera_calibrationCameraCalibrationCameraCalibrationcamera_calibrationT_camera_calibration🔗

Short description🔗

camera_calibrationCameraCalibrationCameraCalibrationcamera_calibrationT_camera_calibration — Determine all camera parameters by a simultaneous minimization process.

Signature🔗

camera_calibration( number NX, number NY, number NZ, number NRow, number NCol, campar StartCamParam, pose NStartPose, string EstimateParams, out campar CameraParam, out pose NFinalPose, out real Errors )

Description🔗

camera_calibrationCameraCalibration performs the calibration of a single camera. For this, known 3D model points (with coordinates NXNXnx, NYNYny, NZNZnz) are projected into the image and the sum of the squared distances between the projected 3D-coordinates and their corresponding image point coordinates (NRowNRownrow, NColNColncol) is minimized.

As initial values for the minimization process the external (NStartPoseNStartPosenstart_pose) and internal (StartCamParamstartCamParamstart_cam_param) camera parameters are used. Thereby NStartPoseNStartPosenstart_pose is an ordered tuple with all initial values for the external camera parameters given in the form \(\mvPoseVar{{ccs}}{wcs}\), where ccs denotes the camera coordinate system and wcs the world coordinate system (see Transformations / Poses and “Solution Guide III-C - 3D Vision”). Individual camera parameters can be explicitly included or excluded from the minimization with EstimateParamsestimateParamsestimate_params. For a detailed description of the available camera models, the different sets of internal camera parameters, and general requirements for the setup, see Calibration.

For a successful calibration, at least one calibration object with accurately known metric properties is needed, e.g., a HALCON calibration plate. Before calling camera_calibrationCameraCalibration, take a series of images of the calibration object in different orientations and make sure that the whole field of view or measurement volume is covered. The success of the calibration highly depends on the quality of the calibration object and the images. So you might want to exercise special diligence during the acquisition of the calibration images. See the section ``How to take a set of suitable images?’’ in Calibration for further details.

After a successful calibration, camera_calibrationCameraCalibration returns the optimized internal (CameraParamcameraParamcamera_param) and external (NFinalPoseNFinalPosenfinal_pose \(\mvPoseVar{{ccs}}{wcs}\)) camera parameters of the camera. Additionally, the root mean square error (RMSE) of the back projection of the optimization is returned in Errorserrorserrors (in pixels). This error gives a general indication whether the optimization was successful.

Preparation of the calibration process🔗

How to extract the calibration marks in the images?

If a HALCON calibration plate is used, you can use the operator find_calib_objectFindCalibObject to determine the coordinates of the calibration marks in each image and to compute a rough estimate for the external camera parameters. Using HALCON calibration plates with rectangularly arranged marks (see gen_caltabGenCaltab), a combination of the two operators find_caltabFindCaltab and find_marks_and_poseFindMarksAndPose will have the same effect. In both cases, the hereby obtained values can directly be used as initial values for the external camera parameters (NStartPoseNStartPosenstart_pose).

Obviously, images in which the segmentation of the calibration plate (find_caltabFindCaltab) has failed or the calibration marks have not been determined successfully by find_marks_and_poseFindMarksAndPose or find_calib_objectFindCalibObject should not be used.
How do you get the required initial values for thecalibration?

If you use a HALCON calibration plate, the input parameters NXNXnx, NYNYny, and NZNZnz are stored in the description file of the calibration plate. You can easily access them by calling the operator caltab_pointsCaltabPoints. Initial values for the internal camera parameters (StartCamParamstartCamParamstart_cam_param) can be obtained from the specifications of the used camera. Further information can be found in Calibration. Initial values for the poses of the calibration plate and the coordinates of the calibration marks NRowNRownrow and NColNColncol can be calculated using the operator find_calib_objectFindCalibObject. The tuple NStartPoseNStartPosenstart_pose is set by the concatenation of all these poses.
Which camera parameters are estimated?

The input parameter EstimateParamsestimateParamsestimate_params is used to select which camera parameters to estimate. Usually, this parameter is set to 'all'"all", i.e., all 6 external camera parameters (translation and rotation) and all internal camera parameters are determined. If the internal camera parameters already have been determined (e.g., by a previous call to camera_calibrationCameraCalibration), it is often desired to only determine the pose of the world coordinate system in camera coordinates (i.e., the external camera parameters). In this case, EstimateParamsestimateParamsestimate_params can be set to 'pose'"pose". This has the same effect as EstimateParamsestimateParamsestimate_params = ['alpha', 'beta', 'gamma', 'transx', 'transy', 'transz']. Otherwise, EstimateParamsestimateParamsestimate_params contains a tuple of strings that indicates the combination of parameters to estimate. In addition, parameters can be excluded from estimation by using the prefix ~. For example, the values [‘pose’,’~transx’] have the same effect as ['alpha', 'beta', 'gamma', 'transy', 'transz']. As a different example, [‘all’,’~focus’] determines all internal and external parameters except the focus. The prefix ~ can be used with all parameter values except 'all'"all".
Which limitations exist for the determination of thecamera parameters?

For additional information about general limitations when determining camera parameters, please see the section “Further Limitations Related to Specific Camera Types” in the chapter Calibration.
What is the order within the individual parameters?

The length of the tuple NStartPoseNStartPosenstart_pose depends on the number of calibration images, e.g., using 15 images leads to a length of the tuple NStartPoseNStartPosenstart_pose equal to \(15 \cdot 7 = 105\) (15 times the 7 external camera parameters). The first 7 values correspond to the pose of the calibration plate in the first image, the next 7 values to the pose in the second image, etc.

This fixed number of calibration images must be considered within the tuples with the coordinates of the 3D model marks and the extracted 2D marks. If 15 images are used, the length of the tuples NRowNRownrow and NColNColncol is 15 times the length of the tuples with the coordinates of the 3D model marks (NXNXnx, NYNYny, and NZNZnz). If every image consists 49 marks, the length of the tuples NRowNRownrow and NColNColncol is \(15 \cdot 49 = 735\), while the length of the tuples NXNXnx, NYNYny, and NZNZnz is 49. The order of the values in NRowNRownrow and NColNColncol is “image after image”, i.e., using 49 marks the first 3D model point corresponds to the 1st, 50th, 99th, 148th, 197th, 246th, etc. extracted 2D mark.
What is the meaning of the output parameters?

If the camera calibration process has finished successfully, the output parameters CameraParamcameraParamcamera_param and NFinalPoseNFinalPosenfinal_pose contain the adjusted values for the internal and external camera parameters. The length of the tuple NFinalPoseNFinalPosenfinal_pose corresponds to the length of the tuple NStartPoseNStartPosenstart_pose.

The representation types of NFinalPoseNFinalPosenfinal_pose correspond to the representation type of the first tuple of NStartPoseNStartPosenstart_pose (see create_poseCreatePose). You can convert the representation type by convert_pose_typeConvertPoseType.

As an additional parameter, the root mean square error (RMSE) (Errorserrorserrors) of the back projection of the optimization is returned. This parameter reflects the accuracy of the calibration. The error value (root mean square error of the position) is measured in pixels. If only a single camera is calibrated, an Error in the order of 0.1 pixel (the typical detection error by extraction of the coordinates of the projected calibration markers) is an indication that the optimization fits the observation data well. If Errorserrorserrors strongly differs from 0.1 pixels, the calibration did not perform well. Reasons for this might be, e.g., a poor image quality, an insufficient number of calibration images, or an inaccurate calibration plate.
Do I have to use a planar calibration object?

No. The operator camera_calibrationCameraCalibration is designed in a way that the input tuples NXNXnx, NYNYny, NZNZnz, NRowNRownrow, and NColNColncol can contain any 3D/2D correspondences. The order of the single parameters is explained in the paragraph ``What is the order within the individual parameters?’‘.

Thus, it makes no difference how the required 3D model marks and the corresponding 2D marks are determined. On the one hand, it is possible to use a 3D calibration object, on the other hand, you also can use any characteristic points (e.g., natural landmarks) with known position in the world. By setting EstimateParamsestimateParamsestimate_params to 'pose'"pose", it is thus possible to compute the pose of an object in camera coordinates! For this, at least three 3D/2D-correspondences are necessary as input. NStartPoseNStartPosenstart_pose can, e.g., be generated directly as shown in the program example for create_poseCreatePose.

Attention🔗

The minimization process of the calibration depends on the initial values of the internal (StartCamParamstartCamParamstart_cam_param) and external (NStartPoseNStartPosenstart_pose) camera parameters. The computed average errors Errorserrorserrors give an impression of the accuracy of the calibration. The errors (deviations in x- and y-coordinates) are measured in pixels.

For line scan cameras, it is possible to set the start value for the internal camera parameter Sy to the value 0.0. In this case, it is not possible to determine the position of the principal point in y-direction. Therefore, EstimateParamsestimateParamsestimate_params must contain the term ‘~cy’. The effective distance of the principle point from the sensor line is then always \(p_{v}\) = \(S_{y} \cdot C_{y} = 0.0\). Further information can be found in the section ``Further Limitations Related to Specific Camera Types’’ of Calibration.

Execution information🔗

Execution information

Multithreading type: reentrant (runs in parallel with non-exclusive operators).
Multithreading scope: global (may be called from any thread).
Processed without parallelization.

Parameters🔗

NXNXnx (input_control) number-array → (real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Ordered tuple with all x coordinates of the calibration marks (in meters).

NYNYny (input_control) number-array → (real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Ordered tuple with all y coordinates of the calibration marks (in meters).

Number of elements: NY == NX

NZNZnz (input_control) number-array → (real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Ordered tuple with all z coordinates of the calibration marks (in meters).

Number of elements: NZ == NX

NRowNRownrow (input_control) number-array → (real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Ordered tuple with all row coordinates of the extracted calibration marks (in pixels).

NColNColncol (input_control) number-array → (real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Ordered tuple with all column coordinates of the extracted calibration marks (in pixels).

Number of elements: NCol == NRow

StartCamParamstartCamParamstart_cam_param (input_control) campar → (real / integer / string)HTuple (double / Hlong / HString)Sequence[Union[float, int, str]]Htuple (double / Hlong / char*)

Initial values for the internal camera parameters.

NStartPoseNStartPosenstart_pose (input_control) pose(-array) → (real / integer)HTuple (double / Hlong)HPose, HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Ordered tuple with all initial values for the external camera parameters.

Number of elements: NStartPose == 7*NRow/NX

EstimateParamsestimateParamsestimate_params (input_control) string-array → (string)HTuple (HString)HTuple (string)Sequence[str]Htuple (char*)

Camera parameters to be estimated.

Default: 'all'"all"
List of values: 'all', 'alpha', 'beta', 'camera', 'cx', 'cy', 'focus', 'gamma', 'image_plane_dist', 'k1', 'k2', 'k3', 'kappa', 'magnification', 'poly', 'poly_tan_2', 'pose', 'sx', 'sy', 'tilt', 'transx', 'transy', 'transz', 'vx', 'vy', 'vz'

CameraParamcameraParamcamera_param (output_control) campar → (real / integer / string)HTuple (double / Hlong / HString)Sequence[Union[float, int, str]]Htuple (double / Hlong / char*)

Internal camera parameters.

NFinalPoseNFinalPosenfinal_pose (output_control) pose(-array) → (real / integer)HTuple (double / Hlong)HPose, HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Ordered tuple with all external camera parameters.

Number of elements: NFinalPose == 7*NRow/NX

Errorserrorserrors (output_control) real(-array) → (real)HTuple (double)HTuple (double)Sequence[float]Htuple (double)

Average error distance in pixels.

Example🔗

(HDevelop)

* Read calibration images.
read_image(Image1, 'calib/grid_space.cal.k.000')
read_image(Image2, 'calib/grid_space.cal.k.001')
read_image(Image3, 'calib/grid_space.cal.k.002')
*  Find calibration pattern.
find_caltab(Image1, CalPlate1, 'caltab_big.descr', 3, 112, 5)
find_caltab(Image2, CalPlate2, 'caltab_big.descr', 3, 112, 5)
find_caltab(Image3, CalPlate3, 'caltab_big.descr', 3, 112, 5)
*  Find calibration marks and start poses.
StartCamPar := ['area_scan_division', 0.008, 0.0, 0.000011, 0.000011, \
                384, 288, 768, 576]
find_marks_and_pose(Image1, CalPlate1, 'caltab_big.descr', StartCamPar, \
                    128, 10, 18, 0.9, 15.0, 100.0, RCoord1, CCoord1, \
                    StartPose1)
find_marks_and_pose(Image2, CalPlate2, 'caltab_big.descr', StartCamPar, \
                    128, 10, 18, 0.9, 15.0, 100.0, RCoord2, CCoord2, \
                    StartPose2)
find_marks_and_pose(Image3, CalPlate3, 'caltab_big.descr', StartCamPar, \
                    128, 10, 18, 0.9, 15.0, 100.0, RCoord3, CCoord3, \
                    StartPose3)
*  Read 3D positions of calibration marks.
caltab_points('caltab_big.descr', NX, NY, NZ)
*  Camera calibration.
camera_calibration(NX, NY, NZ, [RCoord1, RCoord2, RCoord3], \
                   [CCoord1, CCoord2, CCoord3], StartCamPar, \
                   [StartPose1, StartPose2, StartPose3], 'all', \
                   CameraParam, NFinalPose, Errors)
*  Write internal camera parameters to file.
write_cam_par(CameraParam, 'campar.dat')

(C++)

HTuple StartCamPar, NX, NY, NZ\;
HTuple RCoord1, CCoord1, StartPose1\;
HTuple RCoord2, CCoord2, StartPose2\;
HTuple RCoord3, CCoord3, StartPose3\;
HTuple StartPoses, RCoords, CCoords\;
HTuple CameraParam, NFinalPose, Errors\;
// Read calibration images.
HImage Image1("calib/grid_space.cal.k.000")\;
HImage Image2("calib/grid_space.cal.k.001")\;
HImage Image3("calib/grid_space.cal.k.002")\;
// Find calibration pattern.
HRegion CalPlate1 = Image1.FindCaltab("caltab_big.descr", 3, 112, 5)\;
HRegion CalPlate2 = Image2.FindCaltab("caltab_big.descr", 3, 112, 5)\;
HRegion CalPlate3 = Image3.FindCaltab("caltab_big.descr", 3, 112, 5)\;
// Find calibration marks and start poses.
StartCamPar[8] = 576\;                  // ImageHeight
StartCamPar[7] = 768\;                  // ImageWidth
StartCamPar[6] = 288\;                  // Cy
StartCamPar[5] = 384\;                  // Cx
StartCamPar[4] = 0.000011\;             // Sy
StartCamPar[3] = 0.000011\;             // Sx
StartCamPar[2] = 0.0\;                  // Kappa
StartCamPar[1] = 0.008\;                // Focus
StartCamPar[0] = "area_scan_division"\; // CameraType
RCoord1 = Image1.FindMarksAndPose(CalPlate1, "caltab_big.descr", StartCamPar,
                                  128, 10, &CCoord1, &StartPose1)\;
RCoord2 = Image2.FindMarksAndPose(CalPlate2, "caltab_big.descr", StartCamPar,
                                  128, 10, &CCoord2, &StartPose2)\;
RCoord3 = Image3.FindMarksAndPose(CalPlate3, "caltab_big.descr", StartCamPar,
                                  128, 10, &CCoord3, &StartPose3)\;
// Read 3D positions of calibration marks.
caltab_points("caltab_big.descr", &NX, &NY, &NZ)\;
// Camera calibration.
StartPoses = (StartPose1.Append(StartPose2)).Append(StartPose3)\;
RCoords = (RCoord1.Append(RCoord2)).Append(RCoord3)\;
CCoords = (CCoord1.Append(CCoord2)).Append(CCoord3)\;
camera_calibration(NX, NY, NZ, RCoords, CCoords, StartCamPar, StartPoses,
                   "all", &CameraParam, &NFinalPose, &Errors)\;
// Write internal camera parameters to file.
write_cam_par(CameraParam, "campar.dat")\;

Result🔗

camera_calibrationCameraCalibration returns 2 (H_MSG_TRUE) if all parameter values are correct and the desired camera parameters have been determined by the minimization algorithm. If necessary, an exception is raised.

Combinations with other operators🔗

Combinations

Possible predecessors

find_marks_and_poseFindMarksAndPose, caltab_pointsCaltabPoints, read_cam_parReadCamPar

Possible successors

write_poseWritePose, pose_to_hom_mat3dPoseToHomMat3d, disp_caltabDispCaltab, sim_caltabSimCaltab

Alternatives

calibrate_camerasCalibrateCameras

Module🔗

Calibration