Applications of Digital Holography: From Microscopy to 3D-Television

The paper gives an overview of the applications of digital holography based on the one hand on CCD-recording, computer storage, and numerical reconstruction of the wave ﬁelds, and on the other hand on numerical calculation of computer generated holograms (CGH) and the transfer of these CGHs to spatial light modulators for optical reconstruction of the wave ﬁelds. The ﬁrst mentioned type of digital holography ﬁnds applications in digital holographic microscopy, particle analysis, and interferometric form and deformation measurement, while the second type constitutes the basis for holographic 3D TV. The space-bandwidth-problem occuring in this context is addressed and ﬁrst partial solutions are presented.


INTRODUCTION
According to Huang [1] digital holography (DH) is "the computer generation and reconstruction of holograms".However, in the last decades the term "digital holography" was generally used in a less rigorous way, describing only parts of the holographic chain from hologram generation to the reconstruction of the wave fields.While holography was presented by Gabor at the end of the 40's [2], we had to wait more than ten years until the invention of the laser made it possible to obtain impressive optical reconstructions from holograms in the visible [3].But then immediately research began for calculating holograms of objects which were not physically present but existed only as numerical models in computer.The printed calculated holograms have been photographically reduced in size and the desired wave fields were reconstructed optically.This whole topic occasionally got the name digital holography but nowadays it is widely known as the generation of computer generated holograms.
In 1967 Goodman and Lawrence [4] suggested the electronic capture of a hologram, its digitization, and the numerical reconstruction of the complex wave field.Yaroslavskii and Merzlyakov scanned an optical hologram with an electromechanical rotating drum microdensitometer [5] and applied successfully the Fresnel transform to the scanned data.Processing times were in the range of hours.We had to wait until the mid of the 90's, when there were CCD-arrays with reasonable resolution and pixel numbers as well as powerful computers to perform hologram capture and numerical reconstruction in seconds [6].This approach had a strong influence on holographic and holographic interferometric metrology.It was also called digital holography and at present it is the topic mainly connected with this term.
A hot topic of actual research in holography is the use of spatial light modulators (SLM) as holograms for optical recon-struction of 3D wave fields.This in future will enable variable holograms and thus the optical reconstruction of virtual moving 3D scenes.Again we have digital holography, since the holograms fed to the SLMs are coming in digital form from computer.
In the next sections the mentioned topics subsumed under the term digital holography will be presented in more detail with an emphasis on the particular difficulties and how they are solved as well as on the most important applications of these techniques.

HOLOGRAPHY AND DIGITAL HOLOGRAPHY
Holography is a method to record and to reconstruct whole wave fields with amplitude and phase.Recording is done by taking the object wave E O scattered or reflected by the object or the scene and superposing a mutually coherent reference wave E R .The resulting intensity distribution I originally was captured by high resolution photographic emulsion, however in digital holography it is recorded by a CCD-or CMOS-target: The intensity I results from interference between object and reference wave fields, so the spatial frequency at any point H in the hologram depends on the angle θ between both waves at this point.The distance p between two consecutive interference fringes forming the hologram at H is with λ the wavelength of the used coherent light.If the hologram is digitally recorded, we must obey the sampling theo-rem requiring more than two pixels for this p p > 2∆ξ, with ∆ξ the pixel pitch of the CCD-or CMOS-target.Thus given a pixel pitch ∆ξ we obtain an upper limit for the angle which for a pixel pitch ∆ξ = 7 µm and a wavelength λ = 0.5 µm is about 2 o .So we can record small objects, objects far away from the target, or an optically reduced image of a larger object [7,8].
The optical reconstruction of the wave field is performed by illumination of the hologram with the reference wave The resulting three terms on the right-hand side of Eq. ( 5) are the reference wave multiplied with a pure intensity acting as a dc-term, a disturbed conjugate image of the object wave and the desired object wave.The two last-mentioned terms are the so-called twin images.If we reconstruct optically we see a virtual image of the object at the place it was during recording with all 3D-impressions like depth of field or parallax.If the hologram is stored in digital form in computer, the described process is performed numerically by multiplying the hologram data with the complex conjugate of a model of the reference wave and then calculating the complex field E reconstr.in the image plane by propagation from the hologram plane into the image plane [9].For display we calculate the intensity I reconstr.and the phase φ reconstr.pointwisely by If a digital hologram is to be used for optical reconstruction of the wave field, the data have to be fed to an SLM.In this case we have to recognize the space-bandwidth-product: Each hologram in the reconstruction stage acts as a diffraction grating obeying ∆ξ sin θ = λn, (8) with ∆ξ the grating constant, which here is the pixel pitch of the SLM; λ is the wavelength used for reconstruction, in our case in the visible range; θ is the observation angle and n is the diffraction order.In holographic reconstructions normally only the order n = 1 is of interest.A reasonable hologram screen should have a size of at least 0.5 m in horizontal as well as in vertical direction.Let us assume a center wavelength of λ = 0.5 • 10 −6 m and an observation angle of ±30 o , then the above equation leads to a pixel pitch of ∆ξ = 10 −6 m or a bandwidth of 10 6 pixels/m.This results in 0.5 • 10 6 pixels in one direction and in 0.25 • 10 12 pixels with regard to both directions.

CALCULATING THE PROPAGATION OF WAVE FIELDS
As already seen, the numerical reconstruction of the wave fields from digital holograms consists of propagating the field from the hologram plane into the image plane.On the other hand when generating a hologram by computer we have to propagate the field from the object into the hologram plane.The propagation of optical wave fields generally is described by the diffraction integral, which under mild assumptions [9] can be written where U(ξ, η, z = 0) is the complex field amplitude in the plane where the diffracting aperture is assumed and E(x, y, z) is the plane in distance z, k = 2π/λ, and r = (x − ξ) 2 + (y − η) 2 + z 2 .For large distances z π(ξ 2 + η 2 )/λ for all ξ, η the Fraunhofer approximation can be applied, converting Eq. ( 9) into a pure Fourier transform with ν = x/(zλ), µ = y/(zλ) and A = A(ν, µ, z, λ) a complex factor not dependent on U(ξ, η).
In the near field, z 3 [(ξ − x) 2 + (η − y) 2 )]/λ for all ξ, η, x, y, we can use the Fresnel approximation which differs from the Fraunhofer approximation only by the chirp factor.
Examining Eq. ( 9) together with the definition of r we recognize that Eq. ( 9) is a convolution E = z iλ U g with which is the impulse response of free space propagation.Now the convolution theorem can be applied which allows the employment of the effective FFT algorithm.
Taking directly the transfer function instead of the impulse response Eq. ( 12) saves one of the three Fourier transforms in Eq. (13).Again the square root can be approximated leading to the Fresnel transform.Finally for numerical computation the continuous infinite integrals have to be transferred to discrete finite sums [9].
By modification of the impulse response a shifting and scaling of the propagated field can be obtained, which may be of interest in numerical reconstruction for achieving reconstructed objects filling the whole frame [9] or in computer-generation of holograms for tiling a large hologram from several high resolution partial holograms [10].
The U(ξ, η) in the hologram plane normally is the hologram I(ξ, η) of Eq. ( 1) multiplied with the numerical model of the reference wave.Since the hologram I is real we will obtain the twin images which mutually disturb or must be spatially separated, thus wasting at least half of the available pixels.On the other hand if U(ξ, η) is a complex field, we get a single complex field after propagation.Intensity and phase of the complex hologram are obtained by phase shifting methods.Hereby several holograms with shifted phase are captured Here a, b and φ are the unknowns, with a and b the additive and the multiplicative noise, φ Rj the phase shift and J at least the number of unknowns.φ is the desired phase.From at set of J equations, intensity and phase at each point can be calculated [9], leading to the complex field U.

DIGITAL CAPTURE AND NUMERICAL RECONSTRUCTION OF HOLOGRAMS
High resolution CCD-or CMOS-arrays allow the capture of digital holograms, storage of the hologram data in computer memory, and reconstruction of the recorded object or scene by numerical propagation.Now having access to the discrete intensity as well as phase data, this approach is successfully employed in various holographic measurement methods, e. g. microscopy, particle analysis, or interferometric form and deformation measurement.

Digital Holographic Microscopy
The sampling theorem is not violated, if the object to be captured by digital holography remains small, thus digital holographic microscopy (DHM) is an attractive technique to measure intensity and phase distributions of e. g. microelectromechanical systems or biological specimen.Due to the 3D nature of holographically reconstructed wave fields we can observe the field focused to different depths.A typical arrangement for DHM is shown in Figure 1.
Variations of this arrangement are common, so the microscope objective magnifying the object field but limiting the viewing area can be removed, an inclination of the mirror in the reference arm can separate the reconstructed virtual and real images, or a piezo-shifting of the reference mirror enables phase shifting [11].If the reference arm is omitted at all we have the in-line or Gabor configuration which can be employed for transparent or semitransparent specimen.A typical arrangement for in-line DHM with samples in different depths is given in Figure 2 [12].
A promising recent development is the combination of DHM with trapping and manipulation of micro-particles or microorganisms like cells by optical tweezers.In this technique laser beams generate the forces acting on a particle, while these forces arise from the momentum change that a particle experiences when being deflected by photons [13].Classical optical tweezers are realized employing single-beam traps or doublebeam traps.Modern holographic optical tweezers (HOT) use an SLM in which a Fourier hologram is inscribed.The laser beam, acting as reference wave to this hologram, is modulated by the SLM and after passing the microscope objective reconstructs any wave field within the specimen volume.Since it is possible to generate nearly arbitrary light fields, several independent traps can be produced, which can be moved in realtime in 3D [13].

Digital Holographic Particle Analysis
The Gabor in-line arrangement of DH is the choice for particle analysis.Using short laser pulses, it is possible to determine size, position, spatial distribution, velocity and related physical quantities of particle fields [9].The in-line geometry leads to small angles between the reference wave, that goes undisturbed between the particles, and the object wave scattered by the particles.So the sampling theorem inherently is fulfilled.A typical arrangement is given in Figure 3, where the particles can be imagined to be sprayed from a nozzle.
The lateral positions of the particles as well as their lateral dimensions are easily derived from their locations in the reconstructed plane, but the depth dimension is a problem, since the particles remain focused over an extended range.But if the collimated laser beam crosses the particle field several times before hitting the CCD-target, the particles are simultaneously recorded from different angles [14].Then their position and approximate form can be determined by a tomography-like algorithm from the multiple reconstructions.The reconstructions are performed with different distances d of the particle field from the target.Since the dimensions of the reconstructed field depend on this distance, if we use the direct Fresnel transform, the finite discrete form of Eq. (11), in this application we must employ the convolution approach, which maintains the lateral dimensions for any reconstruction distance [9]. Figure 4 shows a collimated beam passing three times the shaded area, which represents a section through the particle field.The deconvoluted light path is depicted in Figure 5.The velocities of the particles can be obtained by using double pulse exposures or laser pulses of defined duration leading to particle tracks whose lengths are proportional to the speed.

Digital Holographic Interferometry
In the former optical holographic interferometry the interference patterns have been generated by double exposure, realtime, or stroboscopic methods.From the holographic interference patterns the interference phase distributions have been determined by fringe counting, fringe tracking, skeletonizing, Fourier-transform evaluation, phase shifting, phase stepping, or heterodyne methods [9].Now in the digital case we have direct access to the phase distribution of e. g. a diffusely reflecting surface without going over the intensity.If we have the two numerically reconstructed wave fields b 1 (n, m) and b 2 (n, m) of a surface before and after deformation, the phase difference is calculated pointwisely by with values between −π and π.Since it is generally known which field belongs to the state before, and which one to the state after the deformation, there is no sign ambiguity in ∆φ as would be the case when evaluating an intensity distribution.In this way the three interference phase distributions of Figure 7 are generated.They belong to an aluminum plate clamped at all four sides and which (a) was pressed from behind, (b) was shifted laterally, and (c) was simultaneously pressed and shifted.Digital holographic interferometry not only is used for deformation measurement in experimental stress analysis or nondestructive testing, but also for form determination, vibration analysis, or measurement of refractive index variations and has found numerous applications in metrology [9].

DISPLAY OF DIGITAL HOLOGRAMS
The advent of powerful SLMs enabled the optical reconstruction of wave fields from digital hologram, wether these have been digitally recorded or numerically generated.This made possible the transmission of whole wave fields for metrologic purposes [15] and is the base of future holographic 3D TV [16].

Computer Generation of Holograms
Soon after the presentation of holography by Gabor the numerical generation of holograms has been tackled [17,18,19].
In the beginning computer generated Fourier holograms have been printed and reduced photographically, afterwards the desired wave fields have been optically reconstructed.The calculation basically consists in a superposition of the complex diffraction patterns belonging to all points of the sampled object surface.As already seen the diffraction pattern of a flat object can be calculated by a Fourier transform.So the object can be segmented into equidistant layers and the diffraction patterns of the points of each layer are effectively calculated by FFT.Finally all these patterns are superposed.Once the diffraction field in the hologram plane is calculated, there are several ways to generate the hologram.The detour phase holograms [18] and the referenceless on-axis complex holograms (ROACH) [20] are examples of complex holograms.Phase holograms are the kinoform [21] and the holograms which are iteratively calculated by a Gerchberg-Saxton-type algorithm [22].Intensity holograms may be binary or grayvalued.Since numerically generated, zero order and conjugated image can be avoided from the calculation [23].
Originally computer generated holograms were intended to be used in interferometric testing of optical components or as filters in optical data processing.Actual applications of computer generation of holograms are in optical tweezers [13] or in future holographic 3D television.

Approaches to Holographic 3D TV
3D television and 3D movies already enter the markets.Besides digital holography there are several approaches to display 3D scenes, namely stereoscopy [24], autostereoscopy [25], integral imaging [26], or volumetric display [27].In stereoscopy the dissimilar images seen by the two eyes are provided for observation where the separation of the images is by different wavelengths, by different polarizations or by different projection time instants [24].These methods require goggles while in autostereoscopy the two images are interlaced and by a lenticular lens plate or parallax barrier each eye is allowed to see only its dedicated image [25].In both methods we have the drawback of only horizontal parallax, so if the head is inclined the 3D effect breaks down.Furthermore we have a fixed view of the scene with no motion perspective.The problem of fixed viewer position in autostereoscopy can be overcome by costly eye tracking with fast recalculation.The multi-observer problem is generally unsolved.But the main drawback is the accomodation conflict: while the lenses of the eyes focus onto the screen, the eye's axes converge to the scene in 3D space.This discrepancy can lead to eye-fatigue and severe discomfort during prolonged watching.In integral imaging the image is combined of many small images captured by a lenslet array from a variety of directions, but still we have a reduced lateral resolution, a restricted viewing angle, and the image-depth range is limited [26].Volumetric methods use light-emitting, light scattering, or light relaying media in 3D space [27].But they suffer from the occlusion problem: Since the machine does not know the position of the observer, voxels in the foreground are generated the same as those in the background, which should be hidden.
Holography in principle reconstructs the whole complex wave field scattered by a 3D scene [28], so the full 3D effect with depth of field, full parallax, motion parallax as well as correct convergence and accomodation is displayed, Figure 8.
A holographic 3D TV system can be divided into three basic steps [?]: (1) recording of the 3D scene, (2) processing and transmission of the recorded data, (3) display and reconstruction of the optical wave fields, Figure 9.In an alldigital-holography approach, a digital hologram of the scene is recorded, the digital hologram data are transmitted and then fed to an SLM.This can be performed with small and medium scale scenes.Large 3D scenes should be recorded by non-holographic methods like stereoscopy, photogrammetry or fringe-projection.From the numerical 3D field gained in this way a computer generated hologram (CGH) is calculated, which is given to the SLM for holographic reconstruction of the wavefield.However, the straightforward realization of the digital holography concept in 3D TV is hindered by the necessary high space-bandwidth-product.This space-bandwidth-product in the range of 10 12 constitutes a severe problem, since there are no SLMs with as many pixels and such a small pixel pitch.SLMs of that kind are not expected to be offered in the foreseeable future.Furthermore there are no devices to record such holograms.On the other hand the computational effort to calculate the intensities or phases of CGHs for this amount of pixels is in a range to prevent real-time or near real-time operation with present computer power.Nevertheless there are several concepts and approaches to cope with these challenges.

The MIT Holo-Video-Display
Since the early 90's MIT has presented several holo-displays, the Mark I to III [29,30,31].By restriction to horizontal-onlyparallax and using about 1000 lines of 10 6 pixels each, a reduction of the space-bandwidth-product from 10 12 pixels to 10 9 is reached.Heart of the Mark I is an acousto-optic modulator (AOM) driven with variable frequency, that corresponds to the spatial frequency in the hologram.So the AOM acts as an SLM with an inscribed one-dimensional hologram.However the spatial frequency in the AOM is not stationary but travels through the crystal.This would generate a motion of the reconstructed wavefield.Now this motion is compensated by a mechanical deflection of the field, performed here by a spinning polygonal mirror, Figure 10.While the lines of the image are holographically reconstructed by the AOM, the vertical deflection is achieved by a much slower galvanometric mirror scanner [29].The later Mark II used an 18-channel AOM allowing the simultaneous output of 18 scan lines and replaced the large horizontal polygonal mirror by a linear array of small galvanometric scanners [30].The 3D images of this display had a 30 o view angle and stretched over a volume of 150 mm × 75 mm × 150 mm.
Because a further adding of more AOM channels and more galvo-scanners would increase the size and cost in a nonmanageable way, in the development of the Mark III the AOM and the horizontal scanning was replaced by a guided-wave acousto-optic device [31].This is a surface-acoustic wave (SAW) device that also contains an optical waveguide.Bandwidths in the GHz-range can be obtained by such devices, which makes them promising for holographic applications.

Image Plane Holographic Video Display System
The extremely high computational amount of Fresnelholography can be reduced by using an image plane hologram, where the imaged object points are located close to the hologram plane [32].In this case each single pixel of the hologram is only influenced by a few points of the scene lying in its direct neighborhood.Thus a reduction of the calculation amount to 1/2000 of that for Fresnel holography has been obtained.A further advantage of image plane holograms is the small chromatic aberration caused by wavelength differences when compared to Fresnel holography.This allows the reconstruction even with white light, at least if the depth of the scene is not too large.Yamaguchi et al [32] used parts of a conventional video projector with three SLMs of LCoS-type (Liquid Crystal on Silicon) and wavelengths of 460 nm, 540 nm, and 633 nm to produce full-color full-parallax 3D images.Due to the present pixel pitch of the LCoS the viewing angle was restricted to about 2.6 o .

The Concept of Sub-Holograms
Another way to reduce the computational amount is the concept of the so-called sub-holograms (SH) [33].A SH consists of all pixels corresponding to the object to be reconstructed and the instantaneous eye-position of the observer, which is determined by an eye-tracker.The eye-position defines a virtual window (VW) of a size of about 10 mm, Figure 11.The whole SH is the superposition of all SHs belonging to this VW and all object points.Only in the SH the hologram pixels need to be calculated.
The calculation of the CGH is performed by two methods: First, the visible object surface is discretized into discrete points.For each point the SH, which is a depth-dependent zone-plate, is calculated.All SHs are positioned and superposed in the SLM-plane.Second, the object is sliced into discrete plane layers parallel to the SLM.Each layer is Fresneltransformed into the VW.In the plane of the VW all Fresneltransforms are summed, resulting in the wavefront in the VW corresponding to the whole visible object.This resulting wavefront then is Fourier-transformed into the SLM-plane thus generating the digital hologram to be fed to the SLM.

The Active Tiling Concept
In the active tiling concept holograms of high information bandwidth are generated by a combination of electrically addressable SLMs (EASLM) and optically addressable SLMs (OASLM) [34].The EASLMs are DMDs (Digital Micromirror Device) which are fast -their temporal bandwidth is about 1.3 Gbit/s -, but they do not fulfil the space-bandwidthproduct requirements for holographic displays, since they exhibit pixel numbers of 10 6 to 2 × 10 6 with a pixel pitch between 10 µm and 20 µm.On the other hand there are OASLMs with information capacities of more than 3 Gbits corresponding to an area of e. g. (300 mm) 2 and 100 line-pairs/mm.If they are updated with video rate 30 Hz, we get a bandwidth of more than 100 Gbit/s.In active tiling a binary CGH is written onto an EASLM, while the OASLM is divided into a number of rectangles.By suitable optics the hologram is multiplied and imaged onto the OASLM so that each rectangle of the OASLM receives a replication of the hologram [34], Figure 12.The OASLM is a bistable liquid crystal layer, which only stores a hologram in a defined area if it is simultaneously optically and electrically activated.Sequentially all rectangles of the OASLM get their parts of the total CGH.Thus the information is stored in the OASLM and is refreshed with video frequency.For reconstruction the OASLM is illuminated with coherent radiation, so that the 3D scene is reconstructed in the Fourier plane of the display optics.
Recent developments are the generation of volatile holograms by superposing a reference beam and an SLM-generated object-beam in liquid-crystal films [35] or in a photorefractive polymer [36] where the desired wave field is simultaneously reconstructed by a probe beam of different wavelength.

CONCLUSIONS
The digital recording of holograms, their storage in computer memory, and the numerical reconstruction of the wavefields give access to numerical values of intensity and phase.Intensities are of interest in microscopy or in particle analysis, while the phase is used in microscopy of phase objects or in the interferometric measurement of form and deformation of opaque objects as well as refractive index variations of transparent media.This type of digital holography depends on sensors with a high number of pixels and a small pixel pitch.It is state of the art in holographic metrology since the existence of such CCD-arrays.
On the other hand the feasibility of optical reconstruction of wave fields from digital holograms fed to SLMs is proven in laboratory experiments.If we assume a progress of the spacebandwidth-product of the SLMs comparable to that in the field of the CCD-arrays and a further increase in computer power as in the last decades, then holographic 3D TV will be reality in not so far future.

Figure 6 (
Figure 6(a) shows the recorded digital hologram and Fig-

FIG. 7
FIG. 7 Interference phase distributions of aluminum plate a) pressed from back side, b) shifted laterally, c) pressed and shifted