Hands-On Stereo Camcalib
This article will show you how to use camcalib to calibrate the intrinsic and extrinsics of a stereo camera pair.
To calibrate a stereo camera pair, you must complete the following sequence.
Acquire a calibration board. You can find more details on how to create your board right here.
Record synchronized image pairs of your calibration board
Load the data into camcalib
Configure and run the intrinsic calibration
Run the extrinsic calibration
Save results and report
We assume that steps 1 and 2 are completed and will start the example at step 3.
Load synchronized image pairs
Camcalib supports many input data formats. In this example, we will use the image directory approach for simplicity. The structure of the image folder is as follows.
Dataset ├── cam1 │ ├── 001.png │ ├── 002.png │ ├── ... │ └── 027.png ├── cam2 │ ├── 001.png │ ├── 002.png │ ├── ... │ └── 027.png
Note that the filenames in cam1 and cam2 are the same. Images with the same filename tell camcalib that they were shot simultaneously. Simultaneous image tuples are not required for intrinsic calibration but are essential for the extrinsic calibration process.
Why camera synchronization?
Most stereo cameras record images synchronously. If, however, you can not guarantee that your stereo camera records images synchronously, you must somehow freeze all motion of the calibration pattern until all cameras have taken their snapshots. For accurate calibration, all cameras must observe the same state.
Example: the error metric for camera calibration is the reprojection error measured in pixels. Good results are below 0.2 pixels. Consider a camera with 1000x1000 pixels resolution and a 90° wide FOV. At a working distance of 1m, a reprojection error of 0.2 pixels corresponds to a length of 200µm. If one of your cameras takes a picture 1ms later than the others, a calibration board moving at 1m/s during calibration will cause your reprojection error to jump to >1pixel.
We load our dataset using the Load Data button and select our dataset from the Dataset folder.
Metric board baseline
Camcalibs extrinsic calibration requires a visible length gauge. To define the length gauge, you must measure the board shape parameters with a ruler or calipers. See the examples for ChArUco and AprilTag configurations below.
ChArUco and AprilTag configuration parameters
ChArUco board measures are simple to determine. Measure the lengths of one checkerboard field and one marker. For more accurate measurements, compute the square size by measuring the length of the checkerboard pattern from one end to the other and divide it by the number of fields along the measured distance.
AprilTag patterns also require two length measures. However, the tag spacing parameter is the fraction of the tag space (b) and tag size (a).
Check out this article for more detail on creating your calibration pattern.
Before we start the intrinsic calibration, we must configure our calibration board. The board we use here is a ChArUco board with
0.055m square size
0.033m marker size
and a C6x6 marker type (6x6 pixel aruco markers)
If you are using camcalib for the first time, you will have to set up your board first. Camcalib will remember your settings so that you can use your configured board again in a later calibration session (red box 4 in the previous screenshot).
Note that the square and marker size parameters are unit-free. We have used 0.055 and 0.033 and interpreted them as meters. Alternatively, we could have set the square and marker sizes 2.1654 and 1.2992 if we wanted our results in inches. You may interpret these numbers in any other length metric you like. Remember that your calibration results will then also be in the unit you selected.
Detect board features
Now, we are ready to detect the checkerboard corners in all camera images by clicking the Detect Features button below the board settings.
Note: we use ChArUco and AprilTag boards as their chessboard corners are uniquely identifiable – even when the board is only partially visible. Identifiable partial board observations at the image edges yield more feature observations up to the image edges. More observed features at the image edges allow us to increase the accuracy of the camera's distortion parameters at the edges of the images and increase your calibration accuracy significantly.
We are now ready to start the intrinsic calibration.
Select Intrinsic from the calibration settings,
choose intrinsic camera model KanalaBrandt (other options are Pinhole, PinholeRadTan, and DoubleSphere),
make sure to enable the Optimize Object Points feature that takes account of fabrication errors in the calibration board itself and further improves calibration accuracy,
and start the calibration with the Calibrate button
You may observe the camcalib optimizers progress as the projection error shrinks on the logarithmic plot. The Log console will also print helpful information – such as the achieved intrinsic projection errors.
Note the reprojection error of cam1 in the above screenshot. We achieved a reprojection error of 0.053 pixels. From the intrinsic calibration results (w=1280, fx=2282.3921), we also can conclude that the camera system has a horizontal FOV of 31.3°, which allows us to convert the 0.053 pixels to a meaningful angular error of 22.6µrad or 0.0013°. Check out our article on what the reprojection error means for you if you are interested in more details.
If all you are interested in is intrinsic calibration, you could stop here and save your results. Continue reading to see the extrinsic calibration process in action.
Now that we have completed the intrinsic calibration, we can continue with the extrinsic calibration of our camera pair. The extrinsic calibration uses knowledge of the calibration board's physical dimensions and observed feature points to calculate all cameras' 3D positions and orientations.
With camcalib, extrinsic calibration is done with the click of a single button Calibrate. By default, Optimize Object Points is enabled and, as with the intrinsic calibration, will improve our results.
After you hit the calibrate button, you will see the optimizer, progress bar, and log output kick into action. Once the calibration is complete, you can review the estimated board errors. They are given in the same unit as you configured your calibration board. The results tab will also show you the intrinsic and extrinsic calibration results.
Again note the achieved extrinsic reprojection error of 0.068 pixels. The extrinsic reprojection error encompasses the intrinsic reprojection errors of both cameras as well as the extrinsic reprojection error. This is the overall error you can expect from your system using the camcalib results. Note, however, that your application building on camcalib will introduce its own errors in addition to the extrinsic reprojection error.
Save the results
To use camcalib results in your own application, click Save Result and choose a location for the YAML file. If you need a detailed report to document the results, use the Generate Report button and save the file in an appropriate place.
We placed the result files of our calibration example here if you want to review the contents in more detail.
Review the results
Let's review calibration_results.yaml in more detail. The YAML file, in essence, contains a tree structure with the sensor names (or serial numbers if you named the input data that way) placed in the root of the structure. Each sensor (cam1 and cam2) has two child nodes: extrinsics and intrinsic. Extrinsics will always contain a 3D position in the form of a translation vector and a 3D orientation in the form of an axis-angle vector.
sensors: cam1: extrinsics: ... intrinsics: ... cam2: extrinsics: ... intrinsics: ...
The intrinsics node is a variable structure depending on the sensor type and configuration you selected at the beginning of the intrinsic calibration step. The intrinsic node will contain all the model parameters needed to describe the chosen calibration model. In our example, we see the parameters cx, cy, fx, fy, and image_size parameters that describe the pinhole camera matrix and the KnnalaBrandt distortion parameters k1-k4. You can find more details on the model parameter options in our documentation.
sensors: cam1: extrinsics: axis_angle: - 0.0 - 0.0 - 0.0 translation: - 0.0 - 0.0 - 0.0 intrinsics: parameters: cx: 631.2595405625116 cy: 491.5258233725439 fx: 2282.3920867826905 fy: 2282.5710507986964 image_size: - 1280 - 1024 k1: 0.19408319800216575 k2: -0.0032982721692749804 k3: -0.018097059724866766 k4: -0.2839958241290381 type: KannalaBrandt cam2: extrinsics: axis_angle: - -0.0017098846275279435 - 0.1681972700321447 - 0.04623455960511724 translation: - -0.19068022336063817 - -0.0017535410879024739 - 0.004499676517813966 intrinsics: parameters: cx: 615.0347480681874 cy: 472.525057978465 fx: 2301.3314301312444 fy: 2301.140570503219 image_size: - 1280 - 1024 k1: 0.18254782022559218 k2: 0.4203174919560141 k3: -5.89114449668284 k4: 25.664870652441312 type: KannalaBrandt
We showed you, in detail, how to calibrate a stereo camera pair and understand camcalibs' result outputs. You should now also have an intuition about the meaning of the reprojection error and how to use it in estimating the errors you may expect from your applications when you use the camcalib results.