Coded access optical sensor (CAOS) imager

High spatial resolution, low inter-pixel crosstalk, high signal-to-noise ratio (SNR), adequate application dependent speed, economical and energy efﬁcient design are common goals sought after for optical image sensors. In optical microscopy, overcoming the diffraction limit in spatial resolution has been achieved using materials chemistry, optimal wavelengths, precision optics and nanomotion-mechanics for pixel-by-pixel scanning. Imagers based on pixelated imaging devices such as CCD/CMOS sensors avoid pixel-by-pixel scanning as all sensor pixels operate in parallel, but these imagers are fundamentally limited by inter-pixel crosstalk, in particular with interspersed bright and dim light zones. In this paper, we propose an agile pixel imager sensor design platform called Coded Access Optical Sensor (CAOS) that can greatly alleviate the mentioned fundamental limitations, empowering smart optical imaging for particular environments. Speciﬁcally, this novel CAOS imager engages an application dependent electronically programmable agile pixel platform using hybrid space-time-frequency coded multiple-access of the sampled optical irradiance map. We demonstrate the foundational working principles of the ﬁrst experimental electronically programmable CAOS imager using hybrid time-frequency multiple access sampling of a known high contrast laser beam irradiance test map, with the CAOS instrument based on a Texas Instruments (TI) Digital Micromirror Device (DMD). This CAOS instrument provides imaging data that exhibits 77 dB electrical SNR and the measured laser beam image irradiance speciﬁcations closely match (i.e., within 0.75% error) the laser manufacturer provided beam image irradiance radius numbers. The proposed CAOS imager can be deployed in many scientiﬁc and non-scientiﬁc applications where pixel agility via electronic programmability can pull out desired features in an irradiance map subject to the CAOS imaging operation. [DOI: http://


INTRODUCTION
Imaging of Electromagnetic (EM) radiation is of fundamental importance to medical, manufacturing, astronomy, aerospace, arts, entertainment, forensics, education, research, and defense sectors.For large scale deployment and impact from optical imagers, it is not only highly desirable to have the features mentioned earlier, but also important to realize portability, low energy consumption, and security of operations.Over the last 65 years, there have been tremendous technological advances in optical image sensor design including optoelectronic photo-detection CCD and CMOS sensor chips and their custom precision interface optics, as well as advances in coding and imaging techniques like Stimulated Emission Depletion (STED) using fluorescence to achieve spatial resolutions beyond the diffraction limit [1]- [11].Nevertheless, these and other prior imager designs approached the optical imaging sensor challenge from a mainly fixed space-time-frequency pixel view point that to a large extent have increased system complexity and limitations, particularly when relying on fluorescence materials, wavelength sensitive coherent sources, detectors, and optics, as well as pixel-scale sensitive mechanical alignments and iterative image processing and recovery methods.
Today, the airwaves are filled with trillions of human-made EM signals emanating from radiating static and mobile platforms ranging from very large to ultra-small such as cell phones, computers, airplanes, earth orbiting satellites, and ground-based radars.Each of these EM electronic devices has the remarkable capability to sift through trillions of different EM signals simultaneously incident on the device and to extract the unique EM signal directed specifically to the device.For example, each cell phone has its specific phone number that acts as its unique identifier code which is deployed in the sifting process.This amazing feat of very weak EM signal detection buried in wideband EM noise (created by all the unwanted EM signals) is achieved using highly sensitive Radio Frequency (RF) electronics with powerful on-chip encoding and decoding via analog and Digital Signal Processing (DSP) within energy efficient and compact high speed hardware [12,13].
Earlier we have proposed and extensively demonstrated an agile pixel DMD-based imager that functions basically as the proposed CAOS imager, but with the agile pixels programmed in a limited SNR operations starring mode which acquires irradiance (i.e., intensity) data without any timefrequency modulation [14]- [20].With present-day RF/optical communications and a robust fully programmable smart image sensor design in mind, proposed is the CAOS imager design that has the potential to deliver high SNR images by uniquely exploiting the time-frequency signaling domain of agile pixels-based spatially distributed optical radiation that is received by one easy to align optical antenna with a single RF output [21].Specifically, the proposed CAOS imager uses image optical irradiance agile pixel position coding via time-frequency modulation codes implemented by a programmable Two Dimensional (2-D) Optical Array Device (OAD) interfaced with a single point optical receive antenna called a point Photo-Detector (PD).Each agile pixel position with its corresponding irradiance value in the multi-agile pixels optical irradiance map has its unique time-frequency code, thereby unleashing the extreme sensitivity and processing power of electrical domain analog and digital signal postprocessing implemented by modern electronic chip technologies [22].
The paper starts by providing the design and working principles of the CAOS imager, including the key features and capabilities of the imager.An experimental CAOS imager demonstration is implemented based on using a visible band DMD as the OAD.The experimental CAOS system imaging quality is tested and compared using a commercial system laser beam with the laser manufacturer providing the accurate Gaussian laser beam irradiance map specifications [23].In addition, the demonstrated CAOS imager SNR performance significant improvement is compared with our earlier demonstrated limited SNR agile pixel DMD-based starring mode imager [19,20].The paper concludes with a summary of the presented key CAOS imager design and experimental results.

PROPOSED CAOS IMAGER DESIGN
Figure 1 shows a possible implementation of the CAOS imager that can operate in two modes of imaging, i.e., passive mode and active mode.The passive mode of CAOS imager operation occurs when irradiance to be imaged comes from an external radiation source and falls on the OAD agile pixelsplane.The active mode of operation occurs when the OAD acts as an internal pixelated radiation source that illuminates the object undergoing imaging that for near-field imaging is placed at or adjacent to the OAD agile pixels-plane.The OAD is an electronically programmable device that allows one to select the location, shape, and size of any given n th agile pixel plus its time-frequency modulation characteristics to suit the particular imaging scenario.The pixel is called an agile pixel as its agility comes both in the spatial domain and timefrequency coding domain and can be electronically optimized for a given imaging scenario.The number of agile pixels N deployed is also application dependent.As an example, Figure 1 shows N = 3 pixels each having a unique code c n (t) defined by its time-frequency modulation characteristics, i.e., the irradiance I 1 due to agile pixel 1 (i.e., n = 1) is modulated at a higher temporal frequency (shown as a digital on/off time sequence) than the irradiance I 2 due to agile pixel 2, which in turn is modulated at a higher temporal frequency than the ir-radiance I 3 due to agile pixel 3. Lens S1 (or concave mirror optic) is positioned to collect the individually coded optical irradiance signals on the OAD and deliver them to the point PD for optical-to-electrical signal conversion.Depending on the light's coherence properties as well as the physical aperture sizes of the OAD, S1, PD, and agile pixel set, S1 can simply act as an imaging optic between the OAD and PD planes or operate in a focus mode or a combination of the two.It is important to note that when operating with coherent light, the PD aperture being large enough does a fringe averaging operation that washes out any optical interference effects, leaving the photo-current to represent the desired sum of the agile pixel irradiance values.Note that in some cases, optical signal amplification can be deployed before the light reaches the point PD.The light collection optic S1 is not required to have an exceptional optical quality so it can spatially resolve a pixel size on the OAD.In fact, if the OAD and PD have similar active area cross-sections, they can be placed directly adjacent to each other, making an ultra-compact lens-less imager design.The PD generated electrical signal can be written as: where K is a constant depending on various photo-detection factors including the PD quantum efficiency.For the n th agile pixel on the OAD, x n , y n are the pixel central coordinates, I n is the pixel irradiance, and c n (t) is the pixel time-frequency code.The PD generated electrical signal containing the temporally coded pixel irradiance data set (e.g., 3 pixels for Figure 1) next undergoes electrical amplification.This amplification operation can be stand-alone, i.e., electrically incoherent where simply the PD generated signal is electrically amplified by a fixed factor G A giving an amplified signal i A (t) = G A i(t).Another option is to implement electrically coherent amplification where the time-base reference frequency signal r(t) used to generate the different pixel temporal codes for the OAD also provides an electrical phase-reference for the electrical mixing operation inside a phase-locked amplifier to produce an output signal i C (t) = G A r(t)i(t), similar to some extent to a super-heterodyne radio receiver.In both cases, the amplified signal undergoes RF filtering to block out the undesired frequency spectrum and retain the high gain signal containing the coded pixel irradiance values.
Next the amplified and filtered electrical signal is fed to a signal processor module where it undergoes time/frequency domain signal transform analysis such as Fourier spectrum analysis or correlation analysis that is used to recover each agile pixel irradiance value given the known identifier code for the given agile pixel.Take the simple case where pure frequency codes are deployed for the agile pixel coding, so c n (t) = cos(2π f n t) and incoherent electrical detection is used in the imager.In this case, simple time domain spectrum analysis or Fourier Transform (FT) in the f Hz frequency space can be conducted on the processed PD signal i A (t) with the signal processor output signal S( f ) given by: Assuming single sideband spectrum analysis and G being a fixed spectrum analysis gain factor, one can write Eq. ( 2) as: In Eq. ( 3), one is assuming that a single frequency spectral code appears as a Delta function of zero 3-dB bandwidth in the f Hz frequency space.In reality, each finite time duration real code signal has some finite 3-dB spectral bandwidth in Hz.The key for very low inter-pixel crosstalk in the Eq. ( 3) CAOS operation is to choose code frequencies not closely positioned in the instantaneous frequency space for the N simultaneously sampled agile pixels.As shown by the Eq. ( 3) FT spectral output S( f ) produced by the signal processor, a complete irradiance map at the time-frequency coded spatial locations on the OAD is generated by the CAOS system.In practice, the number of agile pixels processed simultaneously is application dependent as well as CAOS deployed hardware and software dependent.For the case of more complex codes c n (t), time domain correlation processing can be implemented to decode the sampled pixel irradiances values.In fact, to unleash the full power of high speed DSP, the PD provided signal can undergo immediate digitization with all signal postprocessing operations handled in the all-digital space.
To further elaborate on the signal flow workings of the Complete electronic programmability gives the CAOS imager powerful attributes both as a smart spatial sampler of irradiance maps and also for electronic processing to enable high performance encoding and decoding of the agile pixel irradiance map.Much like wireless and wired communication networks, the agile pixel can operate in different programmable time-frequency coding modes like Frequency Division Multiple Access (FDMA), Code-Division Multiple Access (CDMA), and Time Division Multiple Access (TDMA) [22], [24]- [26].
CDMA and FDMA will produce spread spectrum RF signals from the PD while TDMA is the staring-mode operation of the CAOS imager, one agile pixel at a time producing a DC signal.
For full impact of the CAOS imager, agile pixel codes should include CDMA, FDMA or mixed CDMA-FDMA codes that produce not only PD signals on a broad RF spectrum but also engage sophisticated analog, digital, and hybrid information coding techniques to provide isolation (e.g., minimum crosscorrelation) and robustness amongst time-frequency codes used for OAD pixel coding [27].As pointed out earlier, coherent high sensitivity detection in the electrical domain can be used for decoding of the agile pixels.For example, high SNR correlation operations can be run in parallel using the code set to decode the pixel irradiances.These correlation operations can be implemented in physical hardware that forms a bank of 1-D time signal correlators or in a software-based DSP system or even a hybrid of the two approaches.
The CAOS platform instantaneously brings the following unique features when compared to prior-art imagers.(a) The agile pixel space-time-frequency characteristics can be programmed to suit the imaging scenario with adaptive control deployed as needed.(b) Staring mode PDs such as CCD/CMOS sensors naturally produce a photo-detection electrical noise spectrum that is dominant around the DC and lower frequency components [28].The CAOS imager produces its photo-detected signal at a temporal frequency band that is away from the noisy part of the PD output near DC, thus creating a higher SNR signal for decoding signal processing.All electronics are fundamentally subjected to 1/frequency( f ) or 1/ f noise.By having the output signal frequency band for the PD in the CAOS imager away from DC, the 1/ f noise in the signal processing electronics chain is also lower as f can be from many tens of Hertz to many GHz [29].In addition, it has been known for many years that when using a single photo-detector for spatial light capture, simultaneous detection of light from different sampled spatial zones versus single zone light capture can lead to improved SNR of the time modulated photo-detected output signal [30].where sub-critical data must be extracted from the image scene.

EXPERIMENT
For a first demonstration of the basics of the CAOS imager, the 2-D irradiance map of a CW visible laser beam is chosen for imaging as it is well specified by the commercial laser system manufacturer and also showcases both extreme brightness zones and dim light zones.The CAOS imager is set-up in the lab as shown in Figure 3.A TI DMD is deployed as the OAD.Components used in the present experiment are: TI DLP LightCrafter (LCr) evaluation board with pico DMD chip micro-mirror binary tilt states of +12 • and -12 Melles-Griot single-mode Gaussian laser beam is used for imager testing where the data sheet indicates the beam characteristics [23].
Because of the slow data transfer speed and significant memory limitations of this low cost TI LCr board built for pico-projectors, the full image of the incident laser beam irradiance is generated by operating the CAOS imager agile pixel in a hybrid FDMA plus TDMA mode by scanning 3 agile pixels in a raster format line scan.For example, agile pixels 1, 2 and 3 modulate at frequency codes f 1 , f 2 and f 3 Hz.After the irradiance data is recorded, agile pixels 4, 5 and 6 are also set to modulate at f 1 , f 2 and f 3 Hz and the subsequent irradiance data is recorded, and so on.Furthermore, in-coherent stand-alone electrical detection of the PD acquired electrical signal data is used to generate the post-processing signal that contains the coded pixel irradiance values.The PD electrical signal is digitized using the ADC of the µC and fed to the PC where it is subject to DSP via a Fast Fourier-Transform (FFT) operation in MATLAB.As mathematically indicated in the earlier section, the agile pixel irradiance values are proportional to the FFT spectral peak values and are obtained directly from the frequency domain of the FFT data.Specifically, the imager operation 2-D scan parameters are: 2160 agile pixels time modulating via FDMA coding in sets of three with each set modulating for 6.24 seconds followed by a 14 seconds delay before the next set modulates to ensure synchronisation between PC, µC and LCr DMD board.The size of each agile pixel is 6×6 micromirrors and the sampling rate of the ADC is set to 2.4 kHz to acquire the PD signal.A total of 720 (2160/3) text files having coded irradiance map data are generated from the PD signal ADC data.A program in MATLAB which takes 5 seconds to run, implements the FFT algorithm on the 720 files and extracts the relevant electrical spectral peak values which represent the normalized optical irradiances of the agile pixel sampled laser beam map.Using this decoded data, the 2-D irradiance map is plotted and an inbuilt 2-D Gaussian fitting function in MATLAB is used to estimate the 1/e 2 beam radius.Figure 4  FFT frequency spectral peak overlaps between the 3 coded agile pixels (i.e., when one assumes that each FFT spectral peak resembles a narrow spectral Delta function with no frequency content outside its main spectral lobe), one can consider that there is very low optical crosstalk between the 3 agile pixel decoded irradiances, a fundamental feature of the CAOS imager.In reality, when operating in this FDMA only mode, one must choose the coding frequencies to have minimal instantaneous spectrum overlap and with additional filtering, one can pluck spectral/optical irradiance data with essentially minimal inter-pixel crosstalk.In comparison, using Gaussian laser beam propagation theory and the laser manufacturer data sheet 325 µm minimum beam waist radius and 67.5 cm beam travel distance to DMD, one gets w x = w y = 529.9µm.This manufacturer specified beam radius value is within 0.75% error (i.e., 100% × (529.9−526)/529.9) of the CAOS imager Figure 5(d) data provided 2-D Gaussian fitting measurement [23].A 1-D comparison of the central cross-section of the measured laser beam image with the expected Gaussian beam envelope using manufacturer's values is presented in Figure 5(e).The marker dots show normalized raw (i.e., unprocessed) CAOS imager acquired experimental irradiance data while the dashed curve, which matches well with the dots, is the theoretically expected 1-D Gaussian normalized irradiance envelope at the same propagation distance.

CONCLUSION
In conclusion, presented is the CAOS imager agile pixel platform for optical imager design that engages an electronically programmable OAD to enable agile pixel coded multiple access of an optical irradiance map subjected to the imaging operation.The CAOS imager can electronically adapt to the specific imaging application and its specific needs to sift out desired image information, including under extreme lighting conditions.As a proof-of-concept demonstration, a CAOS imager is demonstrated using a low cost DMD as the OAD with a laser test beam map providing the extreme contrast imaging zone.Image data acquired for the laser beam map shows an excellent 77 dB electrical SNR with a 0.75% spatial beam measurement error, illustrating the power of the CAOS platform where time/frequency domain signal processing of photodetected data can deliver excellent SNR properties.Compared to our previous starring-mode agile pixel DMD imager, the CAOS imager has demonstrated a near 55 dB or over 5 orders of magnitude improvement in electrical SNR.The CAOS platform can also be configured to engage polarization diversity, optical field modulation/encoding and coherent optical detection with DSP decoding to deliver advanced sensing capabilities.In addition, CAOS enables direct image content manipulations using both optical and electronic information processing.It is important to note that in order to realize the full potential of the CAOS imager, dedicated high speed OAD, PD, and chip-scale control and signal processing hardware and software must be developed and optimized.Given the maturity and pace of development of today's electronics, laser array, nano-photonic, nano-mechanical and optical modulation and display technologies including high speed parallel drive optoelectronic devices and pipelined processor architectures, this goal is feasible and can lead to creating critical impact via the CAOS agile pixel space-time-frequency programmable smartness platform matched for certain imaging environments.
Figure 1 CAOS imager implementation, Figure 2 shows a signal flow chart where shown are 3 different sizes, shapes, and positions of the agile pixels labelled as 1, 2, and 3 that are coded with codes c 1 , c 2 , and c 3 , respectively.All the time-frequency coded optical signals engage simultaneously on the PD.This optical analogy is similar to the cell phone scenario where numerous EM signals incident on the RF antenna is equivalent to the many optical agile pixels in the irradiance map incident simultaneously on the point PD.Decoding of agile pixel position based irradiance values is implemented by using the PD generated temporally varying electrical signal and subjecting it to high speed analog and digital electronics-based one Dimensional (1-D) coherent (i.e., electrical phased locked) or incoherent signal processing.With the agile pixel-based irradiance values recovered from what looks like a chaotic RF signal, computer-based non-iterative image (2-D) processing and reconstruction techniques are used to stitch together the 2-D optical irradiance map observed by the CAOS Imager.Do note that the selected sizes, shapes, and locations of the agile pixels within a given sampling time slot can also be optimized to extract the desired image features with maximum SNR based on application specific intelligence.

3 FIG. 4
FIG. 4 (a) Coded 1-D time signal produced by the PD for three agile pixels on the DMD chip.(b) FFT signal processing decoding of the three micromirrors agile pixel irradiances I 1 , I 2 , and I 3 that are proportional to the normalized spectral values computed by the FFT operation.
(a) shows the coded 1-D time signal produced by the PD and recorded by the ADC for three agile pixels at the DMD chip plane coded with on/off modulations of f 1 = 80.1 Hz, f 2 = 133.4Hz, and f 3 = 200.2Hz resulting from the two-tilt state nature of the DMD chip micromirrors.Figure 4(b) shows the FFT signal processing decoding of the three agile pixel irradiances I 1 , I 2 , and I 3 at the DMD chip plane that were coded with the FDMA-TDMA coding.Because in theory, there are no

FIG. 5
FIG. 5 a) 2-D irradiance of 2160 Agile pixels data from the experimental CAOS imager using 60 1-D line scans across DMD chip.b) (Left) The optical irradiance distribution representation of a laser spot incident on an example 6 pixels (in a column) by 9 pixels (in a row) DMD chip.(Right)The total pixel-by-pixel optical irradiance on the chip is represented in a single 1-D plot, which includes the optical irradiance contributions of each row of the DMD chip.c) Zoomed version of Figure 5(a) irradiance data to estimate the average null or noise floor in the imager.d) The fully decoded 2-D irradiance map produced by CAOS Imager showing raw (i.e., unprocessed) intensity values, normalized to 1. e) Comparison of central cross-section data of the CAOS imager acquired laser beam image (dots) versus laser manufacturer provided theoretically expected Gaussian beam envelope (dashed line).

Figure 5 (
a) shows the 2-D irradiance of 2160 agile pixels data (60×36 = 2160) from the experimental CAOS imager using 60 1-D line scans across DMD chip with each line scan consisting of 36 agile pixels.Specifically, the Figure 5(a) plot shows the normalized optical intensity versus distance in mm covered during the 1-D line-by-line scanning process using 3 simultaneous and side-by-side placed agile pixels moving along each scan line.Figure 5(a) is explained using Figure 5(b).

Figure 5
(b)   on the left side shows the optical irradiance distribution incident on a simulated DMD chip having a micromirror grid of 6 rows and 9 columns.

Figure 5 (
b) on the right side shows the pixel-by-pixel irradiance map over the simulated chip beginning with the irradiance contribution from row 1 and onwards.In Figure5(a), the inter-pixel pitch in the line scan direction is 64.8 microns (6×micromirror diagonal of 10.8 microns).Note that in the line scan plot, the maximum value of the coded electrical spectral component obtained via the FFT corresponding to a particular agile pixel in the scan is normalized and given a value of 1. Figure5(c) shows a zoom of the FFT acquired decoded irradiance data to estimate the average null or noise floor in the imager giving an estimated normalized average minimum noise floor of 0.00014267 leading to an imager electrical SNR of 77 dB.This is equivalent to a 38.5 dB optical irradiance SNR with optical SNR = 10 log(Max FFT spectral value/FFT spectral noise floor value) = 10 log(1/0.00014267).Imager electrical SNR is two times the optical SNR.Note that previously we have proposed and demonstrated a DMD-based staring mode agile pixel optical imager that demonstrated an optical SNR of 10.7 dB, resulting in an image electrical SNR of 21.4 dB[19,20].Our present results point to the greatly improved SNR of the CAOS imager, even though coherent detection is not deployed in the electronics of the decoder to achieve image decoding.Figure5(d) shows the full decoded 2-D irradiance map produced by the CAOS imager using the FFT decoding data.Using the acquired CAOS imager-based irradiance 2-D spatial beam map and 2-D Gaussian fitting in MATLAB, obtained are 1/e 2 laser beam waists radii of w x = 526.0µm and w y = 530.1 µm. 3 FIG.1CAOS imager implementation (side view).