Hands-On Stereo Camcalib

This article will show you how to use camcalib to calibrate the intrinsic and extrinsics of a stereo camera pair.

Overview

To calibrate a stereo camera pair, you must complete the following sequence.

1. Acquire a calibration board. You can find more details on how to create your board right here.

2. Record synchronized image pairs of your calibration board

3. Load the data into camcalib

4. Configure and run the intrinsic calibration

5. Run the extrinsic calibration

6. Save results and report

We assume that steps 1 and 2 are completed and will start the example at step 3.

Load synchronized image pairs

Camcalib supports many input data formats. In this example, we will use the image directory approach for simplicity. The structure of the image folder is as follows.

Dataset
├── cam1
│   ├── 001.png
│   ├── 002.png
│   ├──     ...
│   └── 027.png
├── cam2
│   ├── 001.png
│   ├── 002.png
│   ├──     ...
│   └── 027.png

Note that the filenames 001.png, 002.png, ...,027.png in the folders cam1 and cam2 are the same. Images with the same filename tell camcalib that they were shot simultaneously. Simultaneous image tuples are not required for intrinsic calibration but are essential for the extrinsic calibration process.

Why camera synchronization?

Most stereo cameras record images synchronously. If, however, you can not guarantee that your stereo camera records images synchronously, you must somehow freeze all motion of the calibration pattern until all cameras have taken their snapshots. For accurate calibration, all cameras must observe the same state.

Example: the error metric for camera calibration is the reprojection error measured in pixels. Good results are below 0.2 pixels. Consider a camera with 1000x1000 pixels resolution and a 90° wide FOV. At a working distance of 1m, a reprojection error of 0.2 pixels corresponds to a length of 200µm. If one of your cameras takes a picture 1ms later than the others, a calibration board moving at 1m/s during calibration will cause your reprojection error to jump to >1pixel.

We load our dataset using the Load Data button and select our dataset from the Dataset folder.

Load data dialog window

If you get "Error: No data found"

If you try to load an image data folder but get the error "No data found", you likely have tried to load the wrong folder.

Ensure that your images are inside the folder with the camera's name (folders named cam1 and cam2 in our example) and that all camera-name folders are in a dataset folder. Camcalib expects you to load a dataset folder as depicted in the following screenshot:

If you are calibrating a single camera, follow the same guidelines. Camcalib expects a dataset folder with all sensors data in named subfolders.

Camcalib window after loading data

Metric board baseline

Camcalibs extrinsic calibration requires a visible length gauge. To define the length gauge, you must measure the board shape parameters with a ruler or calipers. See the examples for ChArUco and AprilTag configurations below.

ChArUco and AprilTag configuration parameters

ChArUco board measures are simple to determine. Measure the lengths of one checkerboard field and one marker. For more accurate measurements, compute the square size by measuring the length of the checkerboard pattern from one end to the other and divide it by the number of fields along the measured distance.

ChArUco board parameters

AprilTag patterns also require two length measures. However, the tag spacing parameter is the fraction of the tag space (b) and tag size (a).

AprilTag board parameters

Check out this article for more detail on creating your calibration pattern.

Configure calibration board and detect features dialog

Before we start the intrinsic calibration, we must configure our calibration board. The board we use here is a ChArUco board with

• 5 columns

• 7 rows

• 0.055m square size

• 0.033m marker size

• and a C6x6 marker type (6x6 pixel aruco markers)

If you are using camcalib for the first time, you will have to set up your board first. Camcalib will remember your settings so that you can use your configured board again in a later calibration session (red box 4 in the previous screenshot).

Note that the square and marker size parameters are unit-free. We have used 0.055 and 0.033 and interpreted them as meters. Alternatively, we could have set the square and marker sizes 2.1654 and 1.2992 if we wanted our results in inches. You may interpret these numbers in any other length metric you like. Remember that your calibration results will then also be in the unit you selected.

Detect board features

Now, we are ready to detect the checkerboard corners in all camera images by clicking the Detect Features button below the board settings.

Detected ChArUco board features

Note: we use ChArUco and AprilTag boards as their chessboard corners are uniquely identifiable – even when the board is only partially visible. Identifiable partial board observations at the image edges yield more feature observations up to the image edges. More observed features at the image edges allow us to increase the accuracy of the camera's distortion parameters at the edges of the images and increase your calibration accuracy significantly.

Intrinsic calibration

We are now ready to start the intrinsic calibration.

1. Select Intrinsic from the calibration settings,

2. choose intrinsic camera model KanalaBrandt (other options are Pinhole, PinholeRadTan, and DoubleSphere),

3. make sure to enable the Optimize Object Points feature that takes account of fabrication errors in the calibration board itself and further improves calibration accuracy,

4. and start the calibration with the Calibrate button

Intrinsic calibration in action

You may observe the camcalib optimizers progress as the projection error shrinks on the logarithmic plot. The Log console will also print helpful information – such as the achieved intrinsic projection errors.

Note the reprojection error of cam1 in the above screenshot. We achieved a reprojection error of 0.053 pixels. From the intrinsic calibration results (w=1280, fx=2282.3921), we also can conclude that the camera system has a horizontal FOV of 31.3°, which allows us to convert the 0.053 pixels to a meaningful angular error of 22.6µrad or 0.0013°. Check out our article on what the reprojection error means for you if you are interested in more details.

If all you are interested in is intrinsic calibration, you could stop here and save your results. Continue reading to see the extrinsic calibration process in action.

Extrinsic calibration

Now that we have completed the intrinsic calibration, we can continue with the extrinsic calibration of our camera pair. The extrinsic calibration uses knowledge of the calibration board's physical dimensions and observed feature points to calculate all cameras' 3D positions and orientations.

With camcalib, extrinsic calibration is done with the click of a single button Calibrate. By default, Optimize Object Points is enabled and, as with the intrinsic calibration, will improve our results.

Extrinsic calibration in action

After you hit the calibrate button, you will see the optimizer, progress bar, and log output kick into action. Once the calibration is complete, you can review the estimated board errors. They are given in the same unit as you configured your calibration board. The results tab will also show you the intrinsic and extrinsic calibration results.

Again note the achieved extrinsic reprojection error of 0.068 pixels. The extrinsic reprojection error encompasses the intrinsic reprojection errors of both cameras as well as the extrinsic reprojection error. This is the overall error you can expect from your system using the camcalib results. Note, however, that your application building on camcalib will introduce its own errors in addition to the extrinsic reprojection error.

Save the results

To use camcalib results in your own application, click Save Result and choose a location for the YAML file. If you need a detailed report to document the results, use the Generate Report button and save the file in an appropriate place.

Save results/report dialog windows

We placed the result files of our calibration example here if you want to review the contents in more detail.

calibration_report.pdf

Download PDF • 565KB

calibration_result.yaml.zip

Download ZIP • 649B

Review the results

Let's review calibration_results.yaml in more detail. The YAML file, in essence, contains a tree structure with the sensor names (or serial numbers if you named the input data that way) placed in the root of the structure. Each sensor (cam1 and cam2) has two child nodes: extrinsics and intrinsic. Extrinsics will always contain a 3D position in the form of a translation vector and a 3D orientation in the form of an axis-angle vector.

sensors:
  cam1:
    extrinsics: ...
    intrinsics: ...
  cam2:
    extrinsics: ...
    intrinsics: ...

The intrinsics node is a variable structure depending on the sensor type and configuration you selected at the beginning of the intrinsic calibration step. The intrinsic node will contain all the model parameters needed to describe the chosen calibration model. In our example, we see the parameters cx, cy, fx, fy, and image_size parameters that describe the pinhole camera matrix and the KnnalaBrandt distortion parameters k1-k4. You can find more details on the model parameter options in our documentation.

sensors:
  cam1:
    extrinsics:
      axis_angle:
      - 0.0
      - 0.0
      - 0.0
      translation:
      - 0.0
      - 0.0
      - 0.0
    intrinsics:
      parameters:
        cx: 631.2595405625116
        cy: 491.5258233725439
        fx: 2282.3920867826905
        fy: 2282.5710507986964
        image_size:
        - 1280
        - 1024
        k1: 0.19408319800216575
        k2: -0.0032982721692749804
        k3: -0.018097059724866766
        k4: -0.2839958241290381
      type: KannalaBrandt
  cam2:
    extrinsics:
      axis_angle:
      - -0.0017098846275279435
      - 0.1681972700321447
      - 0.04623455960511724
      translation:
      - -0.19068022336063817
      - -0.0017535410879024739
      - 0.004499676517813966
    intrinsics:
      parameters:
        cx: 615.0347480681874
        cy: 472.525057978465
        fx: 2301.3314301312444
        fy: 2301.140570503219
        image_size:
        - 1280
        - 1024
        k1: 0.18254782022559218
        k2: 0.4203174919560141
        k3: -5.89114449668284
        k4: 25.664870652441312
      type: KannalaBrandt

In conclusion

We showed you, in detail, how to calibrate a stereo camera pair and understand camcalibs' result outputs. You should now also have an intuition about the meaning of the reprojection error and how to use it in estimating the errors you may expect from your applications when you use the camcalib results.