Skip to content

adjust_mosaic_imagesadjust_mosaic_imagesAdjustMosaicImagesAdjustMosaicImagesadjust_mosaic_imagesT_adjust_mosaic_imagesπŸ”—

Short descriptionπŸ”—

adjust_mosaic_imagesadjust_mosaic_imagesAdjustMosaicImagesAdjustMosaicImagesadjust_mosaic_imagesT_adjust_mosaic_images β€” Apply an automatic color correction to panorama images.

SignatureπŸ”—

adjust_mosaic_images( image Images, out image CorrectedImages, integer From, integer To, integer ReferenceImage, real HomMatrices2D, string EstimationMethod, string EstimateParameters, string OECFModel )adjust_mosaic_images( Images : CorrectedImages : From, To, ReferenceImage, HomMatrices2D, EstimationMethod, EstimateParameters, OECFModel : )void AdjustMosaicImages( const HObject& Images, HObject* CorrectedImages, const HTuple& From, const HTuple& To, const HTuple& ReferenceImage, const HTuple& HomMatrices2D, const HTuple& EstimationMethod, const HTuple& EstimateParameters, const HTuple& OECFModel )static void HOperatorSet.AdjustMosaicImages( HObject images, out HObject correctedImages, HTuple from, HTuple to, HTuple referenceImage, HTuple homMatrices2D, HTuple estimationMethod, HTuple estimateParameters, HTuple OECFModel )def adjust_mosaic_images( images: HObject, from_val: Sequence[int], to: Sequence[int], reference_image: int, hom_matrices_2d: Sequence[float], estimation_method: str, estimate_parameters: MaybeSequence[str], oecfmodel: str ) -> HObject

Herror T_adjust_mosaic_images( const Hobject Images, Hobject* CorrectedImages, const Htuple From, const Htuple To, const Htuple ReferenceImage, const Htuple HomMatrices2D, const Htuple EstimationMethod, const Htuple EstimateParameters, const Htuple OECFModel )

HImage HImage::AdjustMosaicImages( const HTuple& From, const HTuple& To, Hlong ReferenceImage, const HTuple& HomMatrices2D, const HString& EstimationMethod, const HTuple& EstimateParameters, const HString& OECFModel ) const

HImage HImage::AdjustMosaicImages( const HTuple& From, const HTuple& To, Hlong ReferenceImage, const HTuple& HomMatrices2D, const HString& EstimationMethod, const HString& EstimateParameters, const HString& OECFModel ) const

HImage HImage::AdjustMosaicImages( const HTuple& From, const HTuple& To, Hlong ReferenceImage, const HTuple& HomMatrices2D, const char* EstimationMethod, const char* EstimateParameters, const char* OECFModel ) const

HImage HImage::AdjustMosaicImages( const HTuple& From, const HTuple& To, Hlong ReferenceImage, const HTuple& HomMatrices2D, const wchar_t* EstimationMethod, const wchar_t* EstimateParameters, const wchar_t* OECFModel ) const (Windows only)

HImage HImage.AdjustMosaicImages( HTuple from, HTuple to, int referenceImage, HTuple homMatrices2D, string estimationMethod, HTuple estimateParameters, string OECFModel )

HImage HImage.AdjustMosaicImages( HTuple from, HTuple to, int referenceImage, HTuple homMatrices2D, string estimationMethod, string estimateParameters, string OECFModel )

DescriptionπŸ”—

adjust_mosaic_imagesAdjustMosaicImages performs the radiometric adjustment of images in panoramas. The images to be corrected have to be passed in Imagesimagesimages, the corrected images will be returned in CorrectedImagescorrectedImagescorrected_images.

The parameters Fromfromfrom and Tototo must contain the source and destination indices of all image pairs in the panorama. The projective 3x3-matrix of each image pair has to be passed in HomMatrices2DhomMatrices2Dhom_matrices_2d. The image, which will be used as the reference for brightness and white balance, is selected with the parameter ReferenceImagereferenceImagereference_image.

This means, that one image specifies the β€œideal” brightness and white balance settings. All other images will be corrected such that their brightness and white balance match that of the reference image. In other words, the reference image will not be changed, but all other images will be.

EstimationMethodestimationMethodestimation_method is used for choosing whether a fast but less accurate, or a slower but more accurate determination method should be used. This is done by setting EstimationMethodestimationMethodestimation_method either to 'standard'"standard" or 'gold_standard'"gold_standard".

The methods based on 'standard'"standard" use only the average gray value difference of all images in the overlap area between each image pair. With 'gold_standard'"gold_standard" the gray value difference for each pixel in the overlap area is taken into account explicitly.

The error function that is minimized in all cases is computed as the summed square of the difference between the corresponding gray values.

The availability of the individual method is depending on the selected EstimateParametersestimateParametersestimate_parameters, which determines the model to be used for estimating the radiometric adjustment terms. It is always possible to determine the amount of vignetting in the images by selecting 'vignetting'"vignetting". However, if selected, EstimationMethodestimationMethodestimation_method must be set to 'gold_standard'"gold_standard". For the remainder of the radiometric adjustment three different options are available:

Estimation of vignetting in images is based on the commonly known

\[\begin{eqnarray*} \cos^4 Alpha\end{eqnarray*}\]

approach. This approach assumes that vignetting does not exist in the center of the image and increases with the opening angle by the above equation.

  1. Image adjustment with the additive model. This should only be used to adjust images with very small differences in exposure or white balance. To choose this method, EstimateParametersestimateParametersestimate_parameters must be set to 'add_gray'"add_gray". This model can be selected either exclusively and only with EstimationMethodestimationMethodestimation_method = 'standard'"standard" or in combination with EstimateParametersestimateParametersestimate_parameters = 'vignetting'"vignetting" and only with EstimationMethodestimationMethodestimation_method = 'gold_standard'"gold_standard".

This model is based on the assumption, that the gray value differences between the images can be corrected by adding an individual value to each image accept the reference image. Basically, the modification to every image can be expressed as a call of scale_imageScaleImage:

Β 
scale_image(Image_n,CorrectedImage_n,1.0,Add_n)

where Add_n is the correction term for this image.

  1. Image adjustment with the linear model. In this model, images are expected to be taken with a camera using a linear transfer function. The adjustment terms are consequently represented as multiplication factors. To select this model, EstimateParametersestimateParametersestimate_parameters must be set to 'mult_gray'"mult_gray". It can be called with EstimationMethodestimationMethodestimation_method = 'standard'"standard" or EstimationMethodestimationMethodestimation_method = 'gold_standard'"gold_standard". A combined call with EstimateParametersestimateParametersestimate_parameters = 'vignetting'"vignetting" is also possible, EstimationMethodestimationMethodestimation_method must be set to 'gold_standard'"gold_standard" in that case.

This model is based on the assumption that the gray value differences between the images can be corrected by multiplying the gray values in each image by a factor. Basically, the modification to every image can again be expressed as a call of scale_imageScaleImage:

Β 
scale_image(Image_n,CorrectedImage_n,Mult_n,0)

where Mult_n is the correction term for this image.

  1. Image adjustment with the calibrated model. In this model, images are assumed to be taken with a camera using a nonlinear transfer function. A function of the OECF class selected with OECFModelOECFModeloecfmodel is used to approximate the actually used OECF in the process of image acquisition. As with the linear model, the correction terms are represented as multiplication factors. This model can be selected by choosing EstimateParametersestimateParametersestimate_parameters = ['mult_gray', 'response']["mult_gray", "response"] and must be called with EstimationMethodestimationMethodestimation_method = 'gold_standard'"gold_standard". It is possible to determine the amount of vignetting as well in this case by choosing EstimateParametersestimateParametersestimate_parameters = 'vignetting'"vignetting".

This model is similar to the linear model. However, in this case the camera may have a nonlinear response. This means that before the gray values of the images can be multiplied by their respective correction factor, the gray values must be backprojected to a linear response. To do so, the camera’s response must be determined. Since the response usually does not change over an image sequence, this parameter is assumed to be constant throughout the whole image sequence.

Any kind of function could be considered to be used as an OECF. As in the operator radiometric_self_calibrationRadiometricSelfCalibration, a polynomial fitting might be used, but for typical images in a mosaicking application this would not work very well. The reason for this is that polynomial fitting has too many parameters that need to be determined. Instead, only simpler types of response functions can be estimated. Currently, only so-called Laguerre-functions are available.

The response of a Laguerre-type OECF is determined by only one parameter called Phi. In a first step, the whole gray value spectrum (in case of 8bit images the values 0 to 255) is converted to floating point numbers in the interval [0:1]. Then, the OECF backprojection is calculated based on this and the resulting gray values are once again converted to the original interval.

The inverse transform of the gray values back to linear values based on a Laguerre-type OECF is described by the following equation:

\[\begin{eqnarray*} I_l=I_nl+\frac{2}{\pi} \cdot \arctan(\frac{Phi \cdot \sin(\pi \cdot I_nl)} {1-Phi \cdot \cos(\pi \cdot I_nl)}) \end{eqnarray*}\]

with I_l the linear gray value and I_nl the (nonlinear) gray value.

The parameter OECFModelOECFModeloecfmodel is only used if the calibrated model has been chosen. Otherwise, any input for OECFModelOECFModeloecfmodel will be ignored.

The parameter EstimateParametersestimateParametersestimate_parameters can also be used to influence the performance and memory consumption of the operator. With 'no_cache'"no_cache" the internal caching mechanism can be disabled. This switch only has and influence if EstimationMethodestimationMethodestimation_method is set to 'gold_standard'"gold_standard". Otherwise this switch will be ignored. When disabling the internal caching, the operator uses far less memory, but therefore calculates the corresponding gray value pairs in each iteration of the minimization algorithm again. Therefore, disabling caching is only advisable if all physical memory is used up at some point of the calculation and the operating system starts using swap space.

A second option to influence the performance is using subsampling. When setting EstimateParametersestimateParametersestimate_parameters to 'subsampling_2'"subsampling_2", images are internally zoomed down by a factor of 2. Despite the suggested value list, not only factors of 2 and 4 are available, but any integer number might be specified by appending it to subsampling_ in EstimateParametersestimateParametersestimate_parameters. With this, the amount of image data is tremendously reduced, which leads to a much faster computation of the internal minimization. In fact, using moderate subsampling might even lead to better results since it also decreases the influence of slightly misaligned pixels. Although subsampling also influences the minimization if EstimationMethodestimationMethodestimation_method is set to 'standard'"standard", it is mostly useful for 'gold_standard'"gold_standard".

Some more general remarks on using adjust_mosaic_imagesAdjustMosaicImages in applications:

  • Estimation of vignetting will only work well if significant vignetting is visible in the images. Otherwise, the operator may lead to erratic results.

  • Estimation of the response is rather slow because the problem is quite complex. Therefore, it is advisable not to determine the response in time critical applications. Apart from this, the response can only be determined correctly if there are relatively large brightness differences between the images.

  • It is not possible to correct saturation. If there are saturated areas in an image, they will remain saturated.

  • adjust_mosaic_imagesAdjustMosaicImages can only be used to correct different brightness in images, which is caused by different exposure (shutter time, aperture) or different light intensity. It cannot be used to correct brightness differences based on inhomogeneous illumination within each image.

Execution informationπŸ”—

Execution information
  • Multithreading type: reentrant (runs in parallel with non-exclusive operators).

  • Multithreading scope: global (may be called from any thread).

  • Processed without parallelization.

ParametersπŸ”—

Imagesimagesimages (input_object) (multichannel-)image-array β†’ object (byte)object (byte)HObject (byte)HImage (byte)HObject (byte)Hobject (byte)

Input images.

CorrectedImagescorrectedImagescorrected_images (output_object) (multichannel-)image-array β†’ object (byte)object (byte)HObject (byte)HImage (byte)HObject (byte)Hobject * (byte)

Output images.

Fromfromfrom (input_control) integer-array β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)Sequence[int]Htuple (Hlong)

List of source images.

Tototo (input_control) integer-array β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)Sequence[int]Htuple (Hlong)

List of destination images.

ReferenceImagereferenceImagereference_image (input_control) integer β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Reference image.

HomMatrices2DhomMatrices2Dhom_matrices_2d (input_control) real-array β†’ (real)(real)HTuple (double)HTuple (double)Sequence[float]Htuple (double)

Projective matrices.

EstimationMethodestimationMethodestimation_method (input_control) string β†’ (string)(string)HTuple (HString)HTuple (string)strHtuple (char*)

Estimation algorithm for the correction.

Default: 'standard'"standard"
List of values: 'gold_standard', 'standard'"gold_standard", "standard"

EstimateParametersestimateParametersestimate_parameters (input_control) string(-array) β†’ (string)(string)HTuple (HString)HTuple (string)MaybeSequence[str]Htuple (char*)

Parameters to be estimated.

Default: ['mult_gray']["mult_gray"]
Suggested values: 'add_gray', 'mult_gray', 'response', 'vignetting', 'subsampling_2', 'subsampling_4', 'no_cache'"add_gray", "mult_gray", "response", "vignetting", "subsampling_2", "subsampling_4", "no_cache"

OECFModelOECFModeloecfmodel (input_control) string β†’ (string)(string)HTuple (HString)HTuple (string)strHtuple (char*)

Model of OECF to be used.

Default: ['laguerre']["laguerre"]
List of values: 'laguerre'"laguerre"

ExampleπŸ”—

(HDevelop)


* For the input data to stationary_camera_self_calibration, please
* refer to the example for stationary_camera_self_calibration.
stationary_camera_self_calibration (4, 640, 480, 1, From, To, \
                                    HomMatrices2D, Rows1, Cols1, \
                                    Rows2, Cols2, NumMatches, \
                                    'gold_standard', \
                                    ['focus','principal_point'], \
                                    'true', CameraMatrix, Kappa, \
                                    RotationMatrices, X, Y, Z, Error)
adjust_mosaic_images(Images,CorrectedImages,From,To,1,HomMatrices2D, \
                     'gold_standard',['mult_gray','response'],'laguerre')

ResultπŸ”—

If the parameters are valid, the operator adjust_mosaic_imagesAdjustMosaicImages returns the value 2 (H_MSG_TRUE). If necessary an exception is raised.

Combinations with other operatorsπŸ”—

Combinations

Possible predecessors

stationary_camera_self_calibrationStationaryCameraSelfCalibration

Possible successors

gen_spherical_mosaicGenSphericalMosaic

ReferencesπŸ”—

David Hasler, Sabine SΓΌsstrunk: Mapping colour in image stitching applications. Journal of Visual Communication and Image Representation, 15(1):65-90, 2004.

ModuleπŸ”—

Foundation