Controllable 3D display system based on frontal projection lenticular screen

A novel auto-stereoscopic three-dimensional (3D) projection display system based on the frontal projection lenticular screen is demonstrated. It can provide high real 3D experiences and the freedom of interaction. In the demonstrated system, the content can be changed and the dense of viewing points can be freely adjusted according to the viewers’ demand. The high dense viewing points can provide smooth motion parallax and larger image depth without blurry. The basic principle of stereoscopic display is described ﬁrstly. Then, design architectures including hardware and software are demonstrated. The system consists of a frontal projection lenticular screen, an optimally designed projector-array and a set of multi-channel image processors. The parameters of the frontal projection lenticular screen are based on the demand of viewing such as the viewing distance and the width of view zones. Each projector is arranged on an adjustable platform. The set of multi-channel image processors are made up of six PCs. One of them is used as the main controller, the other ﬁve client PCs can process 30 channel signals and transmit them to the projector-array. Then a natural 3D scene will be perceived based on the frontal projection lenticular screen with more than 1.5 m image depth in real time. The control section is presented in detail, including parallax adjustment, system synchronization, distortion correction, etc. Experimental results demonstrate the effectiveness of this novel controllable 3D display system. [DOI:


INTRODUCTION
Three-dimensional (3D) display has found many applications in different fields, such as medical science, military, education, architecture design, commercial, virtual reality, etc.It has attracted more and more attention.Several 3D display techniques were demonstrated in different experiments [1]− [4].To provide depth cues to viewers, 2-view stereo with one view for each eye is used in the traditional 3D display technology.A kind of special eye glasses is used to filter the corresponding view [4], which is very inconvenient for viewers.Autostereoscopic 3D display which uses a flat display panel and a lenticular sheet or a parallax barrier can provide the viewer N images of different parallax [2,3], but the resolution of each single view is inversely proportional to N. For example, when an image with the resolution 1920 × 1080 is displayed on the flat panel to achieve the auto-stereoscopic display of 9 views, the resolution of each parallax image is 640 × 360.
To avoid the inconvenience of wearing eyeglasses and resolution reduction, a large size 3D display system based on a holographic functional screen and a projector array composed of 64 micro-projectors [5,6] was presented recently, which could demonstrate not only 3D image but also natural 3D video.However, the display parameters and the contents in the 3D display system could not be easily controlled.Here, we present a novel controllable auto-stereoscopic 3D projection display system based on a frontal projection lenticular screen, which also saves nearly half the space of the previous rear-projection display system since the viewers and the projectors are on the same side of the screen.The system can demonstrate 30 different parallax images without any resolution reduction [7], the result is similar to the system based on the holographic functional screen.It can also provide high real 3D experiences and freedom of interaction.The parameters of the frontal projection lenticular screen are based on the demand of viewing such as the viewing distance and the width of view zones.In the demonstrated system, the content can be changed and the density of viewing points can be freely adjusted according to the viewers' demand.The high dense viewing points can provide smooth motion parallax and larger image depth without blurry [8,9].
Experimental results show the effectiveness of the novel controllable 3D display system.A natural 3D scene can be perceived based on the frontal projection lenticular screen with a depth of more than 1.5 m in real time.Viewers can get realtime, large-size, full color, high resolution and high brightness 3D images and videos on the same horizontal line with the projectors.

PRINCIPLE OF THE CONTROLLABLE 3D PROJECTION DISPLAY SYSTEM
The controllable auto-stereoscopic 3D display system primarily consists of one server PC, five client PCs, one 30-projectorarray and a lenticular screen.The projector array and the frontal projection lenticular screen are shown in Figure 1. 30 projectors are arranged in a horizontal array.These projectors project different parallax images together to generate the stereoscopic effect.The distance between the adjacent projectors is E. The distance between two adjacent viewing points is E. T represents the width of all the projectors, which is equal to 30E.The width of the viewing group is T. The pitch is P. The focal length of lenticular sheet is denoted by f.L is the projection and observation distance.
As shown in Figure 1, different lines denote projection light from projectors 1, 2, , 30.The 30 projected parallax images pass though the lenticular sheet and form different image stripes 1, 2, , 30 respectively.These stripes are diffused through the lenticular sheet and they will form different viewing points in the 3D space.Then the projected parallax images will give a synthetic 3D image to the viewer.According to the geometric analysis and Figure 1, the following relations can be obtained, From the relations, it can be concluded that the width of viewing group T is determined by L, f and P. When there are N projectors to project parallax images, the distance between every two adjacent projectors E should be equal to T/N.

Projector-array
The projector-array that consists of 30 projectors is arranged as shown in Figure 2.Each projector is placed on an adjustable platform.E and L (Figure 1) are set according to the actual requirement.

Screen
The pitch and the focal length of the lenticular sheet is denoted by P and f (Figure 1).They are set based on the field of view (FOV) of the lenticular screen [10].The lenticular structure contributes to the anisotropic effect of the screen as it reflects the incoming light towards the same direction in each view zone.When the projectors project different parallax images or videos at the same time, viewers can gain the high-quality stereoscopic effect at Position 1, 2... (Figure 1) In fact, based on the FOV of the lenticular screen [10], we can get clear scenes from several view zones of our system which are shown in Figure 3.Each of the view zones has the same display width, the width can be calculated from the Eq.(1).

Central control section
Different 3D images and videos have different contents and parallaxestherefore viewers will perceive different visual experiences.In our system, according to the viewers demand, the contents shown can be changed and the dense of viewing points can be freely adjusted by the central control section.So we can provide high dense viewing points to form smooth motion parallax and larger image depth without blurry [8,9].

Hardware architecture
Figure 4 shows the control system in which the server PC controls the five client PCs to provide parallax images for the projectors respectively.Based on such hardware architecture we can demonstrate an auto stereoscopic 3D display of 30 viewing points.Ignoring the unavoidable pixel loss in the multi-view-alignment procedure, the resolution of the image from each channel is 1280 × 800 with a display-fresh rate of 60 Hz.

Software architecture
The software architecture in this system is illustrated in Figure 5.When transmitted to the control section, the information of input images is written into the interaction module by which we can control the parallax adjustment module and the distortion correction module.After processing by the two modules, the information will be output.As all the applications are based on the C/S model, the synchronization module is indispensable to realize robust communication between the server PC and the client PCs.The key technology in the control section is providing feedback in accordance with the actual quality of the output images, which will directly affect the interaction module for controlling other modules to achieve high real 3D experiences and freedom of interaction.

Parallax adjustment
In order to accomplish the system, we utilize the virtual reality scenes because of its advantages including high resolution, fewer errors and easy software integration.We can adjust the virtual viewing points (or virtual cameras) in the applied virtual scene according to the feedback of actual information.
Each projector displays an image of different parallax in our system, and in the meanwhile it represents a virtual viewing point whose location and direction is decided by three parameters in the virtual scene.The three parameters are shown in Figure 6.'Position' is the space coordinate of the virtual viewing point in the scene.'Target' is the space coordinate of the reference point, which is usually a point in the looking direction of the virtual viewing point.'Up' is a 3D vector which indicates the up-direction of the viewing point.In the virtual scene, we utilize the off-axis technology.Each viewing point is with the same up-direction obviously, while the 'Target' is determined by the horizontal position (Figure 6).Since different viewing points have different horizontal positions, the output images have different parallax.If each adjacent viewing points have the same distance (D), the distance between the first and any other viewing point is integer multiple of D.
For this reason, the parallax can be adjusted by changing the distance between adjacent viewing points (D) which is controlled by the software.When the distance between each viewing points is set inappropriately originating from the inaccurate arrangement of the projector array and parallax images, the projected image will be blurry and viewers may feel uncomfortable.High real image which has smooth motion parallax and large image depth without blurry is guaranteed when the parameters of the auto-stereoscopic 3D display system is suitable.cause visual fatigue.Since we create five independent threads for the modules in the control section to reduce the delay time in rendering, the synchronization module is essential to maintain these threads working synchronously.In our system, the sever PC is used not only to transfer images and videos but also to control the client PCs to process the importing contents correctly at the right time as shown in Figure 8.In this way, the system saves time and space.

Distortion correction
Generally, a projector display an image in a certain angle (not 0) onto the projection surface, so the 3D image usually suffers from distortion, which in a general way means that the shape of the projected image is not rectangular but trapeze form.In our system, with more than one projector projecting images in different angles, distortion correction is really necessary.The projected parallax images should be corrected not only to eliminate vertical parallax but also to ensure that all the images fit the lenticular screen accurately.The transformation from one projective plane to another which maps straight lines to straight lines, namely, homography [11], is widely used.The transformation steps are given as in Figure 9.We firstly adjust the projectors to maximize the overlap region of these quadrilaterals (Figure 9) until the screen is completely covered by them (such as Parallax image1 and Parallax image2 in Figure 9), then the screen is regarded as the "benchmark-rectangle" of all the perspective-warped images.Finally, in order to correct the distortion accurately, we use some horizontal and vertical straight lines in each image by controlling the interaction module.As long as these straight lines are aligned, all the images are completely overlapped without vertical parallax and trapezoidal distortion.Figure 10 shows the parallax images without correction.Figure 11 shows the result with the corrected distortion.The parameters of the corrected images are stored in the control section and the lines can be removed.The stored parameters can be used to correct other images distortion quickly.More experimental results are described in the next section.

Parameters and result of display
The lenticular projection screen is 1.9 × 1.2 square meters.30 parallax images are projected on the screen to form autostereoscopic 3D display.In our design, the resolution of each parallax images is 1280 × 800.The synthetic image resolution is 30 × 800 × 600.All the parameters of the projection system are listed in

ANALYSIS
A photometer is used to measure the luminance distribution.To measure luminance distribution of one viewing point, the corresponding projector should be turned on with fullwrite image and others are turned off.Moving the photometer along the horizontal direction will get the luminance distribution of one projector in the space.The luminance distribution of a viewing point along the horizontal direction at the optimum viewing distance is shown in Figure 13.It shows that the horizontal distance between adjacent peaks is equal to the width of viewing group, which is 30 cm.The width of the peak luminance is equal to the width of viewing point approximately, which is 1 cm.Therefore, there is little crosstalk between the views.The projectors contents are uniform on the observation plane.Continuous parallax images will be observed in the range of 300 mm.
Figure 14 shows the spatial relationship between system parameters and image depth.D p represents the parallax of images, which is determined by D (Eq. ( 1)).Z out is the image depth.According to the geometric relationship, we can get the Eq. ( 3), The image depth is determined by L, D p and E. To provide more comfortable visual experience and larger image depth, we need to adjust the three parameters.L and E are given in Table 2.According to experimental results, D p is set to 6 mm.If D p is too large, the images will be blurry and viewers may feel uncomfortable.If it is small, the image depth will be short and the 3D immersive experience will be decreased.Therefore, the calculated Z out is 1.5 m.This result is in accord with our actual viewing experience.

CONCLUSIONS
An auto-stereoscopic 3D projection display system based on a frontal projection lenticular screen is demonstrated, and high real 3D experiences and freedom of interaction is successfully realized.Compared with traditional projection display, it is easier to control the parameters of the viewing stereoscopic images and saves space.In the system, the contents can be easily changed and the dense of viewing points can be freely adjusted according to the viewers' demand.By measuring the luminance distribution of one projector, it can be concluded that this frontal projection method can provide the viewers continuous parallax images.According to the spatial geometric relationships, we verify the image depth.A large-size, real- time, full color, high resolution and high brightness 3D display is achieved.

FIG. 5
FIG.5Modules of the software of central control section.

FIG. 13
FIG.13The normalized luminance distribution of a viewing point along the horizontal direction at the optimum viewing distance.

FIG. 14
FIG.14The spatial relationship between system parameters and images depth.

Table 1 .
There are 30 viewing points in the range

TABLE 1
Parameters of the projection system. of 300 mm.Viewers can observe continuous parallax images in the view zones with a depth of more than 1.5 m in real time.The observation parameters are shown in Table2.The results

TABLE 2
Parameters of the observation.