Understanding the RMS value
The problem solved by the optimizer can be seen as a curve fitting problem: given a curve model (e.g. y=a*X+b) find the parameters (a and b) that make the curve fit at best to a series of data points.
In the context of panorama stitching, the model is the equation of projection of a 3D scene point to a 2D picture pixel and the parameters are the calibration unknowns and the orientation of each image.
Steps of a successful optimization
The input of the optimizer is in two parts:
- The model of the problem. In the case of image stitching, this is essentially the type of lens used (standard or fisheye). In this exemplification, we'll replace it by a simpler one: the 2D line defined by y=a*x+b
- The data. The data is a set of points that are supposed to be generated by the model. In the case of image stitching, they are the matched control points coming from the detector.
The goal of the optimizer is to estimate the parameters of the model (a and b) from the data points.
There are actually two types of data points. Inliers: they are good points but never perfectly accurate. Outliers: they are points completely wrong (mismatched control points from different parts of the panorama)
Two common mistakes
Decreasing the RMS by cleaning too much
We resume from step 2) with a lower bad points cleaning threshold.
Decreasing the RMS by adding more unknowns
Let us imagine one want to stitch 3 pictures from the same camera with the same settings.
In automatic mode, the optimizer will try to estimate one global calibration for the 3 pictures.
But, the optimizer can also be manually asked to find the calibration of each picture independently.
In our example, it would be like trying to fit at best the data points with 3 pieces of line instead of one unique line.
By increasing unnecessarily the number of unknowns to be estimated like this, one will always get a lower final RMS, but the result will very often be worse.
In this documentation page you can find a stitching result with and without unknown sharing.
Both are computed without point clean. The RMS without sharing is better than with (2.33 vs. 2.43) but the result with sharing is far better.
- The RMS is a measure of error between a point and the current estimation, NOT the ground truth which is unknown
- It is always possible to decrease the global RMS by cleaning more points or adding unnecessary unknowns, but this does not always lead to a better result