High-speed, high-accuracy 3D shape measurement based on binary color fringe defocused projection

A widely used method in high-speed 3D shape measurement, color-code fringe projection requires the projection of only one image. In traditional color-code method, the projected fringe is compounded by sinusoidal fringes, grayscale value distribution ranges from 0–255, and projection speed is limited to 120 frames/s. Consequently, measurement speed is restricted, and a nonlinear gamma of the projector exists, as well as high harmonics, which have a great inﬂuence on measurement accuracy. Binary color-code fringe defocused projection is proposed to solve the abovementioned problems. With the proposed method, projection speed can switch to tens of K frames/s because it only has two grayscale values (0 and 255). A standard sinusoidal color-code fringe can be generated by properly defocusing the binary color-code fringe, thereby overcoming the inﬂuence of nonlinear gamma and ultimately improving measurement accuracy. Experiment results verify the feasibility and superiority of the proposed method.


INTRODUCTION
The development of high-speed, high-accuracy threedimensional (3D) shape measurement has become a popular research topic because of the emergence of the trend of digitizing hardware technology (including computers, cameras, and projectors).3D shape measurement based on structured light projection has a widely application in industrial manufacturing, medical research, computer vision and so on, as the advantages of non-contact, full-field, easy to information processing [1].Major methods employed to measure dynamic objects include phase measuring profilometry (PMP) [2], Fourier transform profilometry (FTP) [3], and color-code fringe technology [4].PMP is widely used in 3D shape measurement because of its high-accuracy; however, this approach requires at least three images, and the coordinates of the object, once it moves, do not correspond, suggesting that pixel matching is needed for this method [5,6].Highspeed 3D shape measurement has developed rapidly with the development of digital light procession (DLP).Su [7] and Zhang [8] proposed binary fringe defocused projection, which has the following advantages: (1) this method requires only binary grayscale values (0 and 255) and can thus enhance projection speed from 120 frames/s to tens of k frames/s [9]; and (2) this method can overcome the nonlinear gamma of the projector as well as filter the high harmonics [10].Therefore, binary fringe defocused projection has significant advantages not only in measurement speed but also in measurement accuracy.Zhang realized facial expression capture, which can enhance measurement speed to 667 frames/s [11].The measurement speed of traditional sinusoidal method can reach only up to 40 Hz as the limitation of the projection speed.FTP and color-code methods are insensitive to vibration noises as both only need to project one image.They are widely used in 3D shape measurement of moving objects [12,13].However, the measurement accuracy of FTP is not high as it is sensitive to ambient light [4].Harding proposed a color-code moiré technique for high-speed 3D surface contour retrieval [14].Huang proposed a color-code digital fringe projection technique to achieve high-speed 3D shape measurement [15].However, color-code method has the following disadvantages: (1) the color code is difficult to identify because of the influence of color cross-talk and color texture [16], so many compensation algorithms have been proposed to solve these problems and improve measurement accuracy [15,17]; (2) the projection fringe of this method is compounded by sinusoidal fringes, the projection speed of the projector is limited to 120 frames/s, and the measurement speed is limited so that only slowly moving objects can be measured [18].
A binary color-code fringe defocused projection method is proposed to solve the abovementioned problems.The standard sinusoidal color-code fringe can be generated by properly defocusing the binary color-code fringe.Projection speed is greatly improved when projecting binary fringe.The proposed method can overcome the nonlinear gamma of the projector and filter the high harmonics.Measurement speed and accuracy can be dramatically improved compared with that of the traditional color-code fringe projection.Experiments confirmed the effectiveness of the proposed method.
Section 2 explains the principle of the proposed method.Section 3 shows the experimental results and discusses the proposed method.Section 4 concludes the advantages of the proposed method.

Measurement system
The measurement system is shown in Figure 1.The fringe patterns are projected onto the measured object.The images are  captured by the color CCD camera.The optical axis of the projector and that of the color CCD camera intersect at point O, which is the origin.Point E is the optical center of the projector and point C is the optical center of the color CCD camera, points E and C have the same distance which is noted l 0 from the reference plane.d is the distance between the color CCD camera and the projector.f 0 is the frequency of the fringe patterns on the reference plane.∆φ is the phase difference between the corresponding point on the object and reference plane.The height of the measured object is as follows [1]: The parameters l 0 , d and f 0 are obtained by calibration.The verticality and parallel of the system are calibrated by the method of reference [19].

Principle of the binary color-code fringe defocused projection
The RGB image is compounded by the three channels (Red, Green, and Blue).The traditional color-code fringe method is shown in Figure 2. The phase difference between two channels is 120 • .Three-step phase-shifting method is often used to obtain the phase.The intensity of each pixel (x, y) in the Red, Green, and Blue channels is as follows: I R (x, y), I G (x, y), and I B (x, y) are the intensities of the Red, Green, and Blue channels, respectively; I (x, y) is the average intensity; I (x, y) is the intensity modulation; and Φ(x, y) is the solved phase.
After each channel (Red, Green, Blue) of the color-code fringe is binarized according Eq. ( 5), the binary result was shown in Figures 3.
Then the results are compounded, and finally the binary colorcode fringe can be obtained, as shown in Figure 4.
FIG. 3 Each channel binarization result of the color-code fringe.
Zhang verified that a standard sinusoidal fringe can be obtained by properly defocusing a binary fringe [20].The twodimensional Fast Fourier Transformation (FFT) is used to determine the defocus degree of the binary fringe, the ideal defocus degree is obtained when only fundamental Fourier spectrum of the binary fringe exists.2.3 Color compensation method [21] In the traditional color structured light method, the misjudgment of fringe increases and directly affects the measurement accuracy because of the inter-channel crosstalk problem and because the camera are usually designed to have some overlaps with the spectra of the color channels.The color response model proposed by Caspi [22] is used in this paper to correct the crosstalk between color channels.As the crosstalk between color channels only affects by their wavelength λ, the model can be applied in defocus.Without considering the background lightthe response relationship between the pixel value captured by the CCD camera and the setting pixel value of projector is: where can be used to correct all pixels once it is measured.Here, an approximate measurement method is used to achieve channel coupling correction, where three pure RGB images are projected to the white reference plane and the three-channel coupling matrix A can be calculated from its response.The reflection of the white reference plane to the RGB image is approximately equal to 1, which means |K| = 1, and: The code value Subsequently, the inverse matrix A −1 is obtained.Finally, A −1 is multiplied by each pixel value [R G B] T of the image to correct the three-channel coupling and the influence by the threechannel coupling can be eliminated.
However, the reflectivity of the object is unlikely to be 1(|K| = 1) in actual measurement.The camera is avoided tuning to the maximum gray value in measurement, as the information is lost when the image is oversaturated.K can be determined for all pixels by acquiring an additional image of the scene under uniform full white illumination.
Finally, Substituting K into the Eq. ( 9), the three-channel coupling matrix of the camera A can be obtained.

Phase unwrapping
The Red, Green, and Blue channels of the image are separated, and the wrapped phase can be obtained by ϕ(x, y) is the wrapped phase which is the so-called modulo 2π at each pixel with value ranging from [−π, π].Many phase unwrapping algorithms have been presented to achieve the unwrapped phase φ(x, y), including discontinuity minimization algorithm [23], L p -norm algorithm [24], region growing algorithm [25], and quality-guided and least-squares algorithms [26].Here, Goldstein's branch-cut algorithm is used to achieve the unwrapped phase [27].The phase map can be converted to the depth map by a phase-to-height conversion algorithm based on triangulation and the height information of the object can be obtained by Eq. ( 1).

EXPERIMENT
Experiments have been arranged to further verify the effectiveness of the proposed method.The measurement system is set up by a Samsung projector (with a focal length of 12 mm) and a color CCD camera (Manta G-201).The capture speed can switch to 200 frames/s when the image resolution is 800 × 600.As projector speed is fast enough, the measurement speed of the experiment depends on the capture speed of the color CCD camera.The binary color-code fringe (64 cycles) is projected (by properly defocusing), and a standard sinusoidal color-code fringe is generated on the reference plane.The entire measurement process is performed in this defocused level.Three pure RGB images are projected onto the white reference plane.
The principle of color compensation method is used to realize color compensation, so that the influence of the three-channel coupling can be eliminated.The coupling matrix A is obtained as A counterclockwise rotation plaster face is used as the measured object (100 mm × 200 mm × 50 mm).Rotation speed is 60 r/min.A set of images modulated by the plaster face is captured by the color CCD camera with a resolution of 640 × 480.The capture speed of the CCD camera can switch to 200 frames/s in this resolution, therefore, 200 images are captured when the object rotates a week.Figure 6(a) is the defocusd binary color-code fringe.Figure 6(b) is a row of the R, G, B gray value of (a), which shows the inter-channel crosstalk exists.The matrix A −1 (A −1 is obtained by the principle of 2.3) multiplies the images captured by the CCD camera to correct inter-channel crosstalk.Figure 6(c) is the defocused color-code fringe after inter-channel crosstalk correction.Figure 6(d) is a row of the R, G, B gray value of (c), which shows the interchannel crosstalk can be corrected well.
The R, G, B channels of the image are separated, the wrapped phase ϕ(x, y) can be obtained by Eq. ( 11).The unwrapped phase can be obtained by the Goldsteins branch-cut phase unwrapping algorithms [27].After the phase difference is ∆φ(x, y) obtained, the 3D shape of the object can be reconstructed by Eq. ( 1).In order to shorten the calculation time as much as possible, GPU is used in our experiments, as its powerful parallel processing capability and high efficiency of data transmission capacity to achieve the 3D shape of object [28].GPU platform using the NVIDIA GeForce GTX 760, the CPU is i7-4770K, and the programming environment is based on the OpenGL Shading Language (GLSL).Figure 7 is the phase unwrapping process of the rotation plaster face captured by the CCD camera at 4.02 seconds.A standard step block is measured by the proposed method to verify the measurement accuracy, it is composed by three steps (the height of each steps are 15 mm, 30 mm, and 45 mm). Figure 9 is the standard step block modulated by the defocused binary color-code fringe.Figure 10(a) shows the 3D shape of the standard step block by using the proposed method.Figure 10(b) shows the 3D shape of the standard step block by using the traditional color-code method.Figure 11 shows the height distribution in the 72th row of the step block by using the proposed method, traditional method, and the actual height.Table 1 lists the absolute error and standard de- viation of the traditional and proposed method.The experimental results show that the proposed method achieves high accuracy 3D shape of the objects, as it can overcome the influence of the nonlinear Gamma and high harmonics.
The above experiment results demonstrated that the proposed method can dramatically improve measurement speed and accuracy, which further confirmed the feasibility and effectiveness of the proposed method.

CONCLUSIONS
A binary color-code fringe defocused projection method is proposed in this study to achieve 3D shape measurement.The method has the following advantages: 1. High projection speed : Traditional color-code fringe is compounded by sinusoidal fringes, its grayscale value distribution ranges from 0-255, and its projection speed is limited to 120 frames/s, which is unsuitable for measuring moving objects.The proposed method has a potential projection speed that can switch to tens of K frames/s as it only has two grayscale values (0 and 255), which greatly improves the projection speed making it feasibility to obtain real time processing and suitable for dynamic scene measurement.
2. High acquisition speed : The proposed method needs only one image to complete 3D reconstruction.Thus, image acquisition time is greatly reduced compared with that of the previous PMP methods.Image acquisition speed is affected by the receiving speed of the color CCD camera.The receiving FIG.11 The height distribution in the 72th row of the step block by using the proposed method, traditional method, and the actual height.
speed of the color CCD camera in this study can reach up to 200 frames/s when image resolution is 640 × 480.
3. Fast data processing speed : In order to shorten the calculation time as much as possible, the GPU is used in this paper, as its powerful parallel processing capability and high efficiency of data transmission capacity to achieve the 3D shape of object.Our experiment shows the 3D reconstruction is completed within 20ms when the image resolution is 640 × 480.

High measurement accuracy:
The proposed method can overcome the nonlinear Gamma of the projector, as well as filter out high harmonics.The color response model is used to eliminate the influence of the three-channel coupling.Therefore, the proposed method can achieve high-accuracy measurement.

Figure 5 (
a) shows the traditional color-code fringe.Figure 5(b) is the gray scale of the Red channel of (a).

Figure 5 (
c) is the two-dimensional Fast Fourier Transformation (FFT) of the corresponding Red channel.Figure 5(d) is the color-code fringe after properly defocusing the binary color-code fringe.Figure 5(e) is the gray scale of the Red channel of (d).
Figure 5(f) is the two-dimensional FFT of the corresponding Red channel.Figure 5(b) shows the nonlinear gamma and Figure 5(c) shows the high harmonics exist in traditional color-code fringe, which have a great influence on the measurement accuracy.However, this problem can be solved well after properly defocusing the binary color-code fringe, as shown in Figures 5(e) and (f).

FIG. 5
FIG.5Comparison charts between traditional color-code and binary color-code.
the normalized pixel value of the three pure images into the above equation, then the matrix A can be obtained as:


for all light planes.In this case, denote the camera readings at a certain pixel by inverse matrix A −1 is obtained as A −1 =   0.6816 −0.1157 0.0078 −0.0468 0.8019 −0.1886 0.0042 −0.4541 0.0910  

FIG. 6
FIG. 6 (a) the defocusd binary color-code fringe; (b) a row of the R, G, B gray value of (a); (c) the defocused color-code fringe after inter-channel crosstalk correction; and (d) a row of the R, G, B gray value of (c).
Figure 7(a) is the wrapped phase of the plaster face; Figure 7(b) is the unwrapped phase of the plaster face; and Figure 7(c) is the 3D reconstruction result of the plaster face.The entire 3D reconstruction is completed within 20 ms.

Figure 8 (FIG. 7
Figure8(a) is the three images taken from a set of images,

FIG. 8
FIG.8(a) Three images of the counterclockwise rotation plaster face; and (b) 3D shape of (a) using the proposed method.

FIG. 10
FIG.10(a) 3D shape of the standard step block by using the proposed method; and (b) 3D shape of the standard step block by using the traditional color-code method.