Analysis and optimization of the stereo-system with a four-mirror adapter

Stereo imaging is becoming an important diagnostic tool in many practical applications throughout the industries. In the diagnostic procedure two views are used to provide in-depth information of the observing object. The conventional two-synchronized-camera systems used in most applications, pose difficulty to people new to the applications, especially when off-the-shelf cameras have to be used. In this article, several commonly used single camera stereo systems are reviewed and studied. The stereo system with a four-mirror adapter in particular is analyzed, whereafter an improvement is made to achieve the maximum field of view (FOV) of the stereo system with the four-mirror adapter. [DOI: 10.2971/jeos.2008.08033]


INTRODUCTION
With fast development in computer vision techniques, since the late 1970s, stereo imaging has been playing a significant role as a three dimensional diagnostic approach in practical applications.Stereo imaging simply uses two views obtained from different view angles to extract and recover the out-ofplane information of the subject through the three dimensional reconstruction of the subject.It has the advantage of being non-intrusive as most of the optical diagnostic methods do.In addition to that, the in-depth information of interest which is offered by three dimensional reconstruction of the stereo image pairs provides full-scale knowledge of the structure.Apart from three dimensional reconstruction, stereo imaging techniques have been practiced in various engineering applications as a vision-based method to measure parameters, such as particle motion and velocity in stereo particle image velocimetry (SPIV) [1]- [3], stereo particle tracking velocimetry (SPTV) [4]- [6] and stereo particle image displacement velocimetry (SPIDV) [7,8].
In stereo imaging, two images of the subject are taken at different view angles, whereafter corresponding features in the images are extracted, matched and processed to obtain indepth information about the subject via certain algorithms.One common means of obtaining stereo image pairs is to use two synchronized cameras.The two cameras are either fixed in lateral displacement known as translation systems or in angular displacement known as rotational systems [9].
In translation systems as indicated in Figure 1, the optical axes A 1 and A 2 of the two cameras are parallel to each other.Assuming the two cameras have exactly the same parameter setting the two images with the same magnification of d i /d o will be formed on the two co-plane image planes (in reality, there will be certain slight differences in image magnifications as unavoidable dissimilarities exist between the two cameras).Evident limitations of the translation system are the small "common area" and this area of interest, shown as the region in shadow, being off-axis of angle θ to the cameras Rotational systems attain a larger "common area" and this area is more centered in both cameras' FOV (Figure 2).However, since each object point is of different distances to the two lens plane, images of the same magnifications can not be obtained from the cameras.Moreover various image points bear defocus errors to different extent in the two images.The noneven magnifications and considerable defocus errors result in gradient intensity dissimilarities that may influence the correlation procedure in reconstruction.A pre-assumption made in the common correlation technique used in points matching, is that the gradient intensity of a pixel in the first image is close to its corresponding one in the second image [10].
Two-camera stereo systems in stereo imaging guarantee better spatial resolution and a comparatively wider common FOV than stereo optical systems of single cameras do.However slight differences of cameras' specifications such as focal length zoom level and exposure time, are unavoidable between two cameras and will thus bring difficulty in establishing correspondences between the two images [11].In addition, the synchronization mechanism between the two cameras could be expensive and complicated.In the 50s of last century, many manufacturers produced stereo camera sets of two lenses.One big issue with these camera sets is that 135 mm film was commonly employed to make stereo slides, which makes it impossible to be processed with digital image processing techniques.Nowadays these off-the-shelf cameras are difficult to obtain, especially for novice users.
More researches and implementations have turned to singlecamera stereo optical systems in which different external devices are incorporated to the camera.Various arrangements are used to capture two different views of the same subject.This study reviews some representative single-camera stereo systems.Then the optical principle and limitations of the fourmirror stereo system are thoroughly analyzed.Afterward an improvement to the four-mirror stereo system's performance is proposed; results of the simulation using optical software are also presented.

Stereo systems to watch still objects
Nishimoto and Shirai [12] proposed a configuration, as indicated in Figure 3, of a single camera with a glass plate being placed in front of the camera.The rotating axis of the glass plate is orthogonal to the camera's optical axis.Images are taken when the plate rotates to different positions a and b.This method has also been adopted by Gao and Ahuja in their recent study [13] to capture stereo pairs for three dimensional reconstruction.The device in principle is similar to two-camera stereo systems in lateral displacement.The optical axis shifts slightly according to different refraction paths through the

Glass Plate
CCD Camera a b FIG. 3 A single camera stereo system with a glass plate.

CCD Camera rotating mirror outer mirror outer mirror
FIG. 4 A single camera stereo system with two outer mirrors and a rotating mirror.
glass plate in different positions and remains parallel.However this system is only applicable to static subjects.
The problem of limited application to static subject also exists in the system described in Teoh and Zhang's work [14].The configuration is shown in Figure 4; two outer mirrors are fixed at 45 • relative to the lens plane.A third mirror is placed in front of the camera lenses and it rotates clockwise and anticlockwise by 90 • in sequence to get stereo image pairs.Rays go through the reflections on the side when the third mirror becomes parallel to the outer mirror on the defined side and form the respective imaging optical path.
In most practical situations, objects being observed and diagnosed are three dimensional and constantly in dynamic motion, such as flame.Thus two images are required to be taken simultaneously.

Complicated stereo systems
One mechanism that uses one camera to obtain two concurrent images was proposed by Gosthasby and Gruver [15].As displayed in Figure 5, two mirrors being placed symmetrically to the optical axis in front of the camera produces two virtual cameras in a rotational displacement.The whole arrangement is shown in Figure 6 whilst M represents a point in object field.It's noticeable that the optical axis of camera lenses should not be co-plane with the normals of the two mirror planes, so that the camera can be avoided appearing in the images.The disadvantage of the system is it is complicated to align the mirrors with the camera, and it would also be difficult to handle and relocate the system during the observation while keeping several parts relatively static in positions.
Based on the above single-camera stereo system using two plane mirrors, Nene and Nayar proposed three similarstructured stereo systems [16] using different shaped mirrors: elliptical, hyperbolic and parabolic as shown in Figure 7. O In the structure of two parabolic mirrors in Figure 7(c), the axes of two mirrors are parallel to the optical axis of the camera.These three systems of non-planar reflecting surfaces provide wider FOVs than the planar mirror system does.However, the calibration of the arrangement is complicated, particularly in systems of elliptical and hyperbolic mirrors since the optical centre of a zooming lens camera is difficult to locate.Additionally non-planar mirrors need high precision and are of high cost.

Compact stereo system
The following two systems are both applicable to objects in motion, and are easy to realize and compact to handle in practical experiments.
Lee et al. proposed a novel single-camera stereo system [18] using a biprism in front of the camera as shown in Figure 8.The camera lenses are simply substituted by a single piece of lens.The cross-section of the biprism is an isosceles triangle, the two side surfaces of equal length forming an angle α to the base plane.Rays from the object proceed through the biprism and are split into two optical paths by coming out of different side-surfaces.Two views of the subject via different optical diffraction paths are imaged by the camera.Assuming the material's index of refraction is n, the rays entering and leaving the prism cause double refractions, which change the directions of rays.The angle variation γ of the ray's direction as indicated in Figure 9 is then found as follows The equation reveals that the rays have nonlinear angle change owing to beam splitting though the biprism.In addition, rays have different displacement along y direction during the biprism, due to the different incident angles and the different optical paths for each entry point.Serious distortion exists in the images of the biprism stereo system for nonlinear angle variation and different lateral displacement and straight edges of the object appear curving in the picture.Another big problem with the biprism is the colour aberration.Both the distortion and the color aberration become more serious when the angle α increase further.
The single camera stereo system with the four-mirror stereo adapter [17] is relatively a more mature architecture for stereo imaging.This four-mirror stereo adapter has been extensively applied in stereo imaging technique or study [20,21]  are still available in the market from several manufacturers.Figure 10 demonstrates a real stereo adapter attached to the front of the camera lens.

Basic principle of stereo adapter of four mirrors.
A simple and an illustrative geometry of a four-mirror stereo adapter with the camera lenses is displayed in Figure 11.Similarly, the camera lenses are simply substituted by a single piece of lens.The stereo adapter directs the rays from the object to the image plane from two different view directions through the four mirrors.The two inner mirrors are fixed and tilted at 45 • relative to the lens plane, by which way they form an orthogonal angle.The outer mirrors are set almost parallel to the respective inner mirrors but with a discrepancy angle δ.
Supposing a ray incident on the outer mirror on the right side as indicated in Figure 12, it is then reflected by the outer and inner mirrors successively and eventually emerges into the  camera lenses with an angle variation α to its original direction.The angle variation α of the ray direction and the discrepancy angle δ has the following relationship This is valid for all rays entering the adapter and undergoing two reflections afterwards.

Optical Structure and FOV of the System
The four-mirror stereo adapter with the camera can be seen as a system of two virtual cameras, the positions of which are easily obtained using the law of reflection.Taking the group of mirrors on the right side as an example, again as shown in Figure 13, we define the intersecting corner of the two inner mirrors as point C, the optical centre of the real camera as point O, the right outer mirror as R O and the right inner mirror as R I .The distance from point C to mirror R O in horizontal direction is m, and the distance from C to O is n.
The virtual camera, of optical centre O , is symmetrical to the camera O regarding to the line along R I .Similarly the virtual camera O is obtained using the symmetry of the camera O about the line along R O According to Eq. ( 2), the angle α between optical axes of O and O is twice of the discrepancy angle δ between the outer and inner mirrors.
Distance l from O along its optical axis to R O and the distance f from O to optical axis of the camera O can be described by the equations Figure 14 shows the optical structure of the camera with two virtual cameras, k 1 and k 2 are the optical axes of two virtual cameras respectively.The baseline, equivalently known as the distance between optical centers of two virtual cameras, is 2 f Since two virtual cameras share the FOV of the camera and have equal sized images adjacent on the real image plane, each of camera contains half of the real camera's FOV.

FOV of the system and the nearest observable stereo point
We assume half angle of the real camera's FOV is w.l 1 and k 1 represent the edge rays along the FOV of the virtual camera O 1 while l 2 and k 2 represent the edge rays along the FOV of the other virtual camera O 2 .The overlapping of two virtual cameras' FOV, as indicated by the deep shadow region, is the common observing area of the stereo system.
Angle α between k i , (i = 1, 2) and optical axis of the real camera defines the half angle of the stereo system's FOV.Rays k 1 and ray k 2 intersect on the optical axis of the real camera at point P which indicates the nearest observable point along the optical axis that is observable.The distance e between P and the intersecting corner C of two inner mirrors is The region within the distance between P and C and surrounded by k 1 and k 2 is a blind area of the system.The object in this area can not be imaged by both virtual cameras.

DIFFERENT ARRANGEMENTS FOR DIFFERENT OBSERVING OBJECTS
The major problems encountered in the four-mirror stereo system are its limited FOV and the blind area surrounded by the nearest observable stereo point P and the two optical axes k 1 and k 2 of the virtual cameras.In this section several arrangements of the stereo-adapter with different discrepancy angle δ are proposed adapting to different situations.Similarly, we assume the half angle of the camera's FOV is w, accordingly each half of the system has a FOV of w

Arrangement of maximum fixed FOV
As shown in the Figure 15(a), lines l 1 and k 1 indicate the FOV of the left half of the stereo system and lines l 2 and k 2 indicate the FOV of the right half.In order to get a common view of maximum fixed FOV, lines l 1 and k 2 should be parallel and so are the lines l 2 and k 1 as shown in Figure 15(a).The FOV of the system is restricted by rays k 1 and k 2 to be w, and will not be truncated by rays l 1 and l 2 .The nearest observable stereo point in this arrangement is e = (c) Arrangement of restricted field depth.

Arrangement of fixed-width view
For a closer observable stereo point and a wider FOV near this observable stereo point, the system could be modified as in Figure 15(b), in which rays l 1 and l 2 are parallel to each other, and the angle between rays k 1 and k 2 is 2w.In this arrangement, the discrepancy angle δ between the outer and inner mirrors is δ= w/2.
The nearest observable stereo point to the intersecting corner of two inner mirrors is e = m tan 2w and the FOV nearby this nearest observable point is 2w.The view depth is also unlimited in this arrangement; however the width of the view is limited by rays l 1 and l 2 and is fixed to the length of baseline.

Arrangement of restricted field depth
When the discrepancy angle δ between the outer and inner mirrors increased from δ= w/2, the common view of the stereo system becomes a restricted area as shown in Figure 15(c).The distance between the nearest observable stereo point and the intersecting corner of two inner mirrors will be reduced; though the width of the view is smaller than the baseline.This arrangement can be adapted to the case where the observed object is of small dimension.

SIMULATION OF STEREO SYSTEM WITH AN ADAPTER OF FOUR MIRRORS
A four-mirror adapter according to the dimension of Pentax stereo-adapter (as shown in Figure 16), a comparatively commonly available adapter in the market, with the camera lenses are simulated in TracePro as shown in Figure 17 The surface properties of four mirror surfaces are set to be perfect mirror of 100% reflectance with no scattering.The discrepancy angle between the outer and inner mirrors is 3.5 • .According to Eqs.
(1) and ( 3), the half angle of the system's FOV is 7 • , and the approximate length between the nearest observable point and the corner of two inner mirrors is 198 mm.
In the simulation, three objects are placed along the optical axis of the system with different distances to the corner of two inner mirrors.The blue object is placed at the distance 400 mm which is sited in common viewing place; the red ob-  ject is placed at the distance 198 mm which is the approximate distance of the first observable point of the system.The green object is positioned at the distance 72.5 mm where belongs to the blind area of the system.
Figure 18 gives the ray tracing result, and Figure 19 presents the irradiance map consisting two clear blue images on two sides and a red object just observed at the middle point of the image.The green object which belongs to the blind area is invisible in the image.In the second simulation case, the discrepancy angle between outer and inner mirrors is changed to 10 • .The length between the nearest observable point and the corner of two inner mirrors is roughly computed to be 72.5 mm.The irradiance map of CCD sensor is shown in Figure 20, the images of the blue and the red objects move to the side, and the green object which once was invisible appears in the middle of both semiimages.

CONCLUSION
In the study, introduction was made about the twosynchronized-camera systems broadly used in researches and applications.Then different configurations of the stereo system with a single camera has been reviewed, several of which are restricted to the cases where the observed objects must be still.The four-mirror stereo optical system was particularly analyzed.The FOV of the stereo system is constrained by the discrepancy angle between the outer and inner mirrors and there is a blind area near the stereo adapter along the optical axis.Several arrangements have been proposed to improve the FOV according to different situations.Simulation of the system has been given to prove the result.The system is inexpensive and easy to operate in the experiments.Moreover in the system, the two images are taken at exactly the same time, which make its adoptable in observation of dynamic objects.

FIG. 5 A
FIG.5A stereo system proposed by Gosthasby and Gruver.

FIG. 11
FIG.11The geometry of the four-mirror stereo adapter with the camera.

FIG. 13
FIG.13Obtaining the virtual camera O on the right side.

FIG. 14
FIG.14Optical structure of two virtual cameras.

FIG. 17
FIG. 17 Simulation of the four-mirror adapter with the camera lenses.

FIG. 18 FIG. 20
FIG.18Ray tracing of the stereo system with a four-mirror adapter.