Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Off-axis image plane hologram compression in holographic tomography – metrological assessment

Open Access Open Access

Abstract

In this paper, we present a novel study on the impact of lossy data compression on the metrological properties of holographic tomography reconstruction of the refractive index (RI). We use a spatial bandwidth-optimized compression procedure that leverages the properties of image plane off-axis holograms and standardized compression codecs, both widely applied in research and industry. The compression procedure is tested at multiple bitrates, for four different objects and against three reconstruction algorithms. The metrological evaluation is primarily done by comparison to the reconstruction from original data using the root-mean-squared error (RMSE). We show that due to differences between objects and different noise sensitivities of the reconstruction algorithms, the rate-distortion behaviour varies, but in most cases allows for the compression below 1 bit per pixel, while maintaining an RI RMSE less than 10−4.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Holographic tomography (HT) [1,2] is a quantitative label-free imaging technique that reconstructs the 3D refractive index (RI) of a transmissive or quasi-transmissive object from several holographic projections [3]. The label free nature makes it attractive for biological analysis, as specimens like cells and tissues are inherently semi-transparent. RI distribution and its changes are related to the biophysical features of the specimens and are utilized in such fields as cytometry, cell biology, biotechnology, infectious disease, hematology, and cancer research [4]. HT techniques are also a potential choice for industrial metrological applications like quality control of optical micro-components such as fiber optics, waveguides or microlenses [5].

In contrast to digital holographic microscopy (DHM), in tomography, several holographic projections of the object are required for its 3D reconstruction. These measurements are typically obtained by scanning with the object illumination beam (limited angle tomography [3,6]), rotating the object itself (full angle tomography [7]), or a combination of both approaches [8]. Each tomographic measurement requires capturing from a few tens up to a few hundreds of holograms for a single volumetric measurement. Often a tomographic capture is repeated in time (time-lapse scenario) to investigate temporal changes of the sample [9,10] or in space to cover large field of view (data stitching scenario) [11,12].

The resulting large volume of data and its storage are a major barrier of entry to practical adoption of holographic techniques, which can be mitigated by developing efficient data compression solutions. The compression solutions applicable for HT can be broadly classified into two categories: the end user applications and for storing the captured camera measurements. For end user applications, the final 3D refractive index distribution reconstructed from the holographic projections needs to be efficiently compressed to enable random access [13] so that parts of the encoded reconstruction volume can be rapidly decoded from a storage medium without having to process/reconstruct the entire volume. In the second case, compression solutions encoding the object wavefield can be utilized immediately after obtaining the camera measurements, as the state of the art reconstruction algorithms are computationally complex and typically applied after a large number of hologram projections are already available. Furthermore, this type of compression is suitable for archival purposes as it preserves the original complex valued object wavefield, permitting future changes in the processing pipeline used by reconstruction algorithms. In this work, we focus on the latter approach, that is to compress the complex valued wavefield encoding the information present in the first order term of captured off-axis hologram. In previous works [1416], we had introduced lossless and lossy compression solutions utilizing existing image/video codecs applicable for image plane off-axis holograms (IP-OAH) used in DHM. For IP-OAH, the bandwidth of the first order term that encodes the received wavefield is limited to a well defined region in the Fourier domain [17]. This fact was utilized to develop spatial bandwidth optimized compression solutions.

Apart from the minimal and well defined spatial bandwidth, IP-OAHs allow for desirable properties of the optical system (compact design, easy modality switching), which makes it popular for commercial HT systems. These setups are similar to their DHM counterparts, but utilize multiple consecutive oblique illumination beam incidences instead of the single normal illumination beam incidence used in DHM. In this work we propose an appropriately modified lossy IP-OAH compression solution accounting for the oblique illumination in HT setup.

For the first time, we analyzed the influence of the input data compression on the final accuracy of tomographic reconstruction. The selected compression codecs are tested from the metrological point of view. What is more, we have tested the metrological sensitivity of different tomographic reconstruction algorithms on data compression and determined the acceptable compression conditions. The tomographic reconstruction methods include the direct inversion method (DI) [2,6], the Gerchberg-Papoulis (GP) algorithm with non-negativity constrain (NNC) [18,19] and finally the GP algorithm with object support constrain (TVIC) [20,21]. These algorithms are based on Rytov approximation, typically used for biological specimens.

This paper is organized as follows. In section 2., we present the instrumentation and methods used to capture the holograms and we assess the compression performance. This includes the description of the optical system (Sec. 2.1), presentation of biological test objects (Sec. 2.2), brief characterization of tested reconstruction algorithms (Sec. 2.3), detailed description of the compression methodology (Sec. 2.4), as well as basic information about tested codecs (Sec. 2.5) and performance metrics (Sec. 2.6). In section 3., we discuss the results of the conducted experiments, starting with the basic analysis of tested reconstruction algorithms’ noise sensitivity (Sec. 3.1), followed by the benchmarks of tested codecs used in the proposed compression scheme (Sec. 3.2). Finally the work is concluded in section 4.

2. Instrumentation and methods

The overall pipeline for the metrological assessment of compression influence on tomographic reconstruction is visualized in Fig. 1 with the details of each step described below. The oblique illumination beam ($u_{i\phi }$) is partially scattered by the sample (S), producing the scattered beam ($u_{s\phi }$, visualised in red). $u_{i\phi }$ and $u_{s\phi }$ are combined with the reference beam ($u_{r}$) producing the holographic pattern that is captured by the camera (CAM). After a set of holograms is captured the data is used to produce two reconstructions – one with the raw data, the other with the data after compression. Our assessment methodology is based on comparison of the volumes reconstructed from compressed holograms with the reference volume reconstructed from the raw holograms.

 figure: Fig. 1.

Fig. 1. The general scheme of data capture and processing pipeline towards metrological assessment and comparison of tomographic results. The “compression” and “decompression” blocks refer to the procedures discussed in Sec. 2.4 and Fig. 3.

Download Full Size | PDF

2.1 Optical system

In this work, a modified Mach-Zehnder setup with the object illumination beam scanning module and imaging module is used to capture the off-axis holograms (see. Figure 2). The HT system works in the limited angle optical diffraction tomography (LA-ODT) configuration [22], which is the most often used one for biological applications, including commercial HT systems [2225]. In this configuration the object (located at Petri dish or a microscopic slide) is stationary, while the object illuminating beam is changing its direction.

 figure: Fig. 2.

Fig. 2. The experimental setup and the selected micro-objects (X-Y cross section of DI-reconstructions): a) Schematic of the used HT setup, b) HaCaT cells, c) SHSY-5Y cell (163% zoom), d) Skin epithelial cell, e) 3D printed cell phantom (163% zoom). The optical elements in a) are: LASER – laser light source, C – collimator lens, M – mirror, GM – galvanometric mirror, BF – beam formation module, S – sample, MO – microscope objective, AP – MOs aperture plane, TL – tube lens, S’ – image plane, CAM – camera. Red colored beam indicates a light scattered by the object ($u_{s\phi }$). Cross section through the AP shows how the illuminating beam $u_{i\phi }$ (green) and scattered beam $u_{s\phi }$ (red) are distributed with oblique illumination by the moving GM.

Download Full Size | PDF

In our system a series of holograms are captured as the galvanometric mirror (GM) tilts the illumination beam, thereby capturing the object from different perspectives. After the retrieval and demodulation of the +1 order term by the carrier frequency of the reference wave $u_{r}$, the received field $u_{\mathbf {\phi }}$ associated with illumination beam angle $\mathbf {\phi } = (\phi _x, \phi _y)$ is obtained. The retrieved complex-valued field can be written as a superposition of two fields as shown in Eq. (1)

$$u_{\phi}(\mathbf{x}) = u_{i\phi}(\mathbf{x}) + u_{s\phi}(\mathbf{x}).$$

The scattered field $u_{s\phi }$ refers to the scattering contribution of the object and it can not be accessed directly – it is obtained from both $u_{\phi }$ and $u_{i\phi }$. The $u_{i\phi }$ denotes the incident field that would propagate through the mounting medium if no object was present, so that it characterizes the illumination beam including the illumination system aberrations. Typically, $u_{i\phi }$ is obtained by capturing holographic projections before introducing the object – one projection for each illumination direction denoted by $\phi$. The same illumination directions are then used to capture the holographic projections of the object $u_{\phi }$, such that along with the previously captured $u_{i\phi }$ the $u_{s\phi }$ can be calculated using single scattering approximations such as Rytov of Born approximations (see Sec. 2.3).

The setup that was used to capture the projections was comprised of an He-Ne laser with $\lambda = {632.8}\,{\mathrm {nm}}$ and a microscope objective (MO) with $\mathrm {NA} = 1.3$. Although the nominal Zeiss MO magnification was 40x, due to the non-standard tube lens (TL, $f_{TL} = {200}\,{\mathrm {mm}}$), the MO-TL module provided magnification of approximately $\mathrm {M} = 48.48$. The holograms were captured with a camera with pixel pitch $\mathrm {pp} = {3.45} \,{{\mathrm{\mu} \mathrm{m}}}$ and they were cropped to $(R_x, R_y) = (894, 894)$ px, resulting in approximate FoV of $63.6 \times {63.6}\,{{\mathrm{\mu} \mathrm{m}}}^2$. The bit depth of the captured holograms was 8 bits, hence the original bitrate was 8 bits per pixel (see Sec. 2.6).

2.2 Samples

We used four test objects, namely the HaCaT cells, SHSY-5Y cell, skin epithelial cell and 3D printed cell phantom [26]. The samples used in this study are presented in Fig. 2 b)-e). The objects vary in complexity, RI contrast, the amount of high frequency details, as well as the area that individual samples occupy within the field of view. The measurements have been chosen such that the variety of parameters were tested – we used measurements with different NA, different number of projections and different illumination angle. The summary of the parameters related to the sample measurement is given in Table 1. Note that the cell phantom used different mounting medium compared to other objects; however, the $n_m$ varied nonetheless due to corrections accounting for environmental temperature. Also the magnification varied slightly due to minor changes in system adjustment.

Tables Icon

Table 1. The measurement parameters for each object used in this work.

2.3 Reconstruction

In our work, we test the proposed compression scheme against three different tomographic reconstruction methods, namely: the non-iterative Direct Inversion (DI) method [2,6], and two iterative methods: Gerchberg-Papoulis (GP) with the non-negativity constraint (NNC) [18,19] and GP with total variation iterative constraint (TVIC) [20,21]. This choice is dictated by the fact that these methods (with or without custom modifications) are nowadays most often used in research and commercial HT systems [4]. Below we provide short descriptions of the applied tomographic reconstruction methods:

Direct inversion (also known as Wolf transform) is a reconstruction method that utilizes Fourier Diffraction Theorem that assumes either the Born approximation or Rytov approximation. In this work, we utilize the latter, as it has no restrictions on a sample thickness and thus is more suitable for analysis of biological specimens. DI allows the reconstruction of 3D scattering potential from multiple complex-valued projections $u_{s\phi }(\mathbf {x})$. With the a priori knowledge of the mounting medium RI, the scattering potential is converted to 3D distribution of RI of the sample. [2,6]

Gerchberg-Papoulis (GP) is a general iterative extrapolation method that we utilize in conjunction with DI algorithm. In GP iterations, some chosen constraints are applied in the spatial domain, while the measured data is replenished in the frequency domain. Two constraints that we use are:

Non-negativity constraint (NNC) that prevents the reconstructed scattering potential to contain any negative-valued voxels that are typically present after the DI reconstruction. [18,19]

Total variation iterative constraint (TVIC) that is used to suppress the voxels that are not part of the object. The object’s 3D footprint, called the object support is estimated with a separate procedure – in case of this work, it is a binarized tomographic reconstruction obtained with a strong regularization with the total variation component. [20,21]

Although the GP-based methods generally provide better results, they are more sensitive to noise [27], hence the compression requirements for those algorithms are expected to be higher.

2.4 Compression methodology

In this section, we discuss the spatial bandwidth optimized compression methodology using traditional image/video codecs, introduced previously in the context of the image plane off-axis DHM [1416], here applied for data captured in holographic tomography. In both microscopic and tomographic setups, the +1 order term is band-limited by the numerical aperture of the microscope objective. However, the differences between the techniques are significant – for DHM the peak Fourier component encoding the object transmittance function is obtained at the center of the pupil function of the microscope objective, while for LA-ODT there will be many captures with peak Fourier components displaced from the center of the pupil by $f_{\mathbf {\phi }} = (f_{\mathbf {\phi x}}, f_{\mathbf {\phi y}})$, corresponding to the illumination angle for a given projection. Furthermore, the reconstruction algorithms discussed in Section 2.3 for obtaining the 3D RI distribution of the object in LA-ODT are considerably different from the simple phase retrieval procedure in DHM. Due to those differences a separate analysis of the influence of bitrate of the various codecs on the RI reconstruction quality is required. The results of such analysis is presented in section 3.

In the spatial bandwidth optimized approach, the following steps are performed for encoding each projection (Fig. 3(a)):

  • • The +1 order term containing the received field $u_{\mathbf {\phi }}$ of a given projection is filtered using Fourier filtering corresponding to the band-limit of the IP-OAH setup. The band-limit in the discrete Fourier domain is given by an ellipse whose axis have length $\bigg (\frac {\mathrm {NA}\cdot \mathrm {pp}\cdot \mathrm {R_x}}{\lambda \mathrm {M}},\frac {\mathrm {NA}\cdot \mathrm {pp}\cdot \mathrm {R_y}}{\lambda \mathrm {M}}\bigg )$, where $\mathrm {NA},\mathrm {\lambda }$ and $\mathrm {M}$ represents the numerical aperture, wavelength and magnification of the microscope while $\mathrm {pp},\mathrm {R_x}$ and $\mathrm {R_y}$ represents the pixel pitch and resolution of the camera sensor. After this step, the information outside the band-limit of the +1 order term is removed.
  • • The +1 order term is now modulated by an appropriate frequency $f_{\mathbf {\phi }}$ such that the most dominant frequency is displaced to the zero frequency. This step is done to obtain a signal with predominantly low frequency components which can be compressed more efficiently by image/video codecs.
  • • The modulated term is now converted into the spatial domain by applying the inverse discrete Fourier transform within frequency window of size $\bigg (\frac {\mathrm {NA}\cdot \mathrm {pp}\cdot \mathrm {R_x}}{\lambda \mathrm {M}},\frac {\mathrm {NA}\cdot \mathrm {pp}\cdot \mathrm {R_y}}{\lambda \mathrm {M}}\bigg )$.
  • • The real and imaginary (R/I) components of the demodulated term in the spatial domain are quantized to either 8-bit or 16-bit unsigned integer format, depending on the codec that is being used.
  • • The quantized R/I components are compressed independently with the same target bitrate using image/video codecs.

 figure: Fig. 3.

Fig. 3. The pipeline for the hologram encoding (a) and decoding (b) procedures (referenced in Fig. 1). In the decoding pipeline the green arrow indicates the stage where the decoding procedure may shortcut to perform reconstruction, when the hologram representation is not required. The complex data is indicated by the $\mathbb {C}$ symbol. The spectrum representation is visualised as $\log _{10} \left ( \left | U_{\mathbf {\phi }} \right | \right )$, while the spatial domain is visualized as $\left | u_{\mathbf {\phi }} \right |^{2}$. Compared with the method proposed in the previous work [16] the only difference is the addition of the circular shift of most energetic Fourier component to the center of frequency domain.

Download Full Size | PDF

The decoding procedure is described below and shown in Fig. 3(b).

  • • The compressed streams of the R/I components are decoded into the appropriate bit depth (either 8 or 16-bit unsigned representation) using the corresponding image/video decoder.
  • • The signal is dequantized to obtain the lossy frequency-modulated projection in the spatial domain, which is required as input from the reconstruction algorithms.
  • • To obtain the lossy version $\widehat {u}_{\mathbf {\phi }}$ of the projection $u_{\mathbf {\phi }}$ encoded in the original hologram, the dequantized signal is circularly shifted to return the spectrum to the original frequencies. This signal may be passed to the reconstruction algorithm or may be further processed to create the synthetic version of the original hologram (steps below).
  • • The spectrum is zero-padded such that the center of the pupil region is equivalent to the carrier frequency of the original hologram.
  • • Inverse Fourier transform is calculated and the R component of the signal is multiplied by the factor of 2 in order to retrieve the real valued hologram.
The procedures presented above are used to process the holograms in the test path indicated in Fig. 1.

2.5 Codecs under test

For the sake of brevity, we only test two codecs - JPEG 2000 and HEVC, in the spatial bandwidth optimized configuration discussed in Section 2.4. For DHM, among a broad spectrum of codecs, JPEG 2000 and HEVC obtained the best metrological quality metrics [16] for static and timelapse compression respectively. Both these codecs are transform-based, implying they attempt to sparsify the signal by application of a suitably designed frequency transform operation.

JPEG 2000 JPEG 2000 [28], the successor to the popular but aging JPEG standard, produces smaller bitstreams than the latter for the same quality level particularly at mid to low bitrates. In addition, JPEG 2000 allows for a more scalable signalling of the bitstream for supporting bespoke requirements of different practical applications [29]. For lossy compression, JPEG 2000 applies the discrete wavelet transform with a Mallat style decomposition using the CDF (Cohen–Daubechies–Feauveau) 9/7 wavelet. The coefficients obtained after the wavelet transform are grouped into subbands and fed to the EBCOT (Embedded block coding with optimized truncation) stage, ultimately producing a layered bitstream that can support progressive decoding. The layers can be chosen such that each additional layer increases SNR (quality progressive) or the resolution (resolution progressive), where these layers can be further made to support spatial progressions as well [30].

We use the Kakadu [31] implementation (V8.0.5) of JPEG 2000 in this work, which allows a 16-bit data input.

HEVC H.265/HEVC [32] (High Efficiency Video Codec) was introduced in 2013 as the successor H.264/AVC (Advanced Video Codec), producing files that are on average 25-50% smaller than the latter for the same perceived visual quality. Notwithstanding the increased encoding complexity for HEVC compared to AVC, the decoding complexity is comparable, while also facilitating highly parallel processing hardware architectures. In these video codecs, both intra (within a frame) and inter (across frames) dependencies are utilized for compression. In HEVC, the encoder will initially divide the frame being encoded into coding tree units (CTU), where the CTU may be predicted from other units belonging to the same frame or neighbouring frames. Here, HEVC supports higher flexibility for the CTU block sizes (up to $64\times 64$) compared to AVC (up to $16 \times 16$). In addition, HEVC supports a nested recursive quadtree structure that can recursively partition the prediction residual from the CTU for application of the DCT/DST (discrete cosine/sine transform) [33].

In this work, the RExt (format range extension) reference software [34] designed for compressing data with high precision is used. For video coding, a GOP (group of pictures) size of 32 was used. RExt supports input data with bit-depth 16-bit but can only produce a bitstream where the coefficients can be at most 8 bits. For the sake of brevity the video coding mode will be denoted as Inter HEVC, while the mode not utilizing any temporal prediction method will be denoted as Intra HEVC.

2.6 Performance metrics

RMSE of final reconstruction This is the most important distortion metric that is used in this work. It evaluates the reconstructed volume from the metrological perspective. The threshold value that we use is $10^{-4}$ RI. It was chosen based on the visual assessment of reconstructed volumes, such that the distortion due to compression was not concentrated on the object’s features. The RMSE is evaluated according to equation:

$$\textrm{RMSE}=\sqrt{\sum\sum\sum \frac{\big(\textrm{RI}[x,y,z]-\widehat{\textrm{RI}}[x,y,z]\big)^{2}}{n}},$$
where $\widehat {\textrm {RI}}$ and $\textrm {RI}$ are respectively a distorted and reference reconstructions and $n$ is the number of voxels of that signal.

SNR This metric was used to evaluate the compression induced distortion in the intermediate processing steps, where the data is complex-valued.

$$\textrm{SNR}=10 \log_{10}\left(\frac{\sum \sum u[x,y]u^{*}[x, y]}{\sum\sum \left|u[x, y]-\widehat{u}[x, y]\right|^{2}}\right)$$

Bitrate This metric is used to assess how efficiently the data is stored in terms of memory. Bitrate is expressed as the number of bits required to store a single pixel of the data, averaged by the original number of pixels, regardless of how many pixels is used to store encoded data. The original bitrate of all datasets is 8 bits per pixel (bpp).

$$\textrm{Bitrate (bpp)}= \frac{\textrm{Number of bits in bitstream}}{\textrm{Number of pixels in camera sensor}}.$$

3. Results

In this section, the results of numerical experiments are presented. First, we characterize the sensitivity of the reconstruction algorithms by synthetically distorting the sinogram of the 3D printed cell phantom (Sec. 3.1). The distortion is measured at the stage of sinogram with the SNR metric and at the stage of RI with the RMSE metric, as discussed in Section 2.6. This allows us to find the target distortion level measured on the sinogram term being compressed, for meeting some target distortion measured from the RI reconstruction.

Finally, the experimental objective and subjective analysis of the proposed lossy compression methodology is presented (Sec. 3.2). The subjective analysis is undertaken by visual investigation of high frequency region of the 3D cell phantom at different levels of compression. For objective analysis, the distortion introduced by the codecs are measured in the final stage of processing, by comparing the RMSE of the RI obtained from the decoded measurements versus the threshold RMSE of $10^{-4}$. The smallest bitrate at which the threshold is met is reported for each codec, sample and reconstruction method.

3.1 Distortion analysis for the 3D printed cell phantom

The 3D printed cell phantom is utilized for comprehensively evaluating the distortion introduced by applying the proposed lossy compression. Before characterizing the compression-induced distortion, the influence of additive white Gaussian noise (AWGN) on the sinogram is investigated with a numerical experiment and shown in Fig. 4 for the various reconstruction methods. As expected from the GP-based reconstruction algorithms [27], we observe that the NNC and TVIC are more sensitive to AWGN when compared to DI, producing almost a magnitude of error increase in the reconstruction RMSE for the same level of AWGN on the sinogram. Slightly better performance of the TVIC algorithm may be explained by the stronger regularization by the object support. For lossy compression using the various codecs, a heightened sensitivity to noise for the iterative methods was again observed and is shown in Fig. 6.

 figure: Fig. 4.

Fig. 4. Reconstruction RMSE distortion in function of sinogram SNR.

Download Full Size | PDF

A visualization of the local reconstruction errors introduced by the various codecs tested for the phantom at different values of reconstruction RMSE is shown in Fig. 5. Overall, at around $10^{-4}$ RMSE, a relative absence of structural error in the reconstruction is seen, in comparison to the level of the random error. Therefore, in this work, we use a target RMSE of $10^{-4}$ for evaluating the data storage costs of the different codecs.

 figure: Fig. 5.

Fig. 5. The visualization of the errors regions with high spatial frequency information obtained for consecutive compression bitrates with the a) Inter HEVC, b) Intra HEVC and c) JPEG 2000 codec. Above the error maps, the global RMSE is indicated. The reference region is the USAF structure present in the cell phantom object. The cross-section is made within the depth of focus region.

Download Full Size | PDF

3.2 Codec benchmarks

The rate distortion plots for the reconstruction methods are shown in Fig. 6 for all presented objects (Sec. 2.2) and all tested reconstruction algorithms (Sec. 2.3). While the overall performance of the codecs track each other, for all operating points for the Intra HEVC mode, it consistently produced the smallest file sizes for the same target RMSE. The next best rate-distortion performer is JPEG 2000 followed by Inter HEVC. This indicates that the inter-coding (temporal) compression techniques utilized by the Inter HEVC profile may not be suitable for addressing the redundancy present in tomographic projections. Furthermore, even in the range extension (rEXT) profiles, the bitstream coefficients produced by HEVC can have a maximum bitdepth of 8. As we shall see later, this limits the applicability of the HEVC codec, if a very low target RMSE is required. Note that this is more a constraint imposed by operating parameters of the codec specification, rather than a limitation of the underlying techniques.

 figure: Fig. 6.

Fig. 6. Rate-distortion plots showing the degradation of the reconstructed volumes in terms of RMSE (vertical axes) versus bitrate (horizontal axes). The threshold RMSE value of $10^{-4}$ is indicated by dashed lines. Each column represents a given object – a) HaCaT, b) SHSY-5Y, c) Epithelial cell, d) Cell phantom.

Download Full Size | PDF

The comprehensive bitrate table for the various codecs, samples and reconstruction methods for achieving the target RMSE $10^{-4}$ is shown in Table 2. For achieving the metrological target for DI reconstruction, Intra HEVC was able to obtain lower bitrates than JPEG 2000 in the case of all objects, with the former producing files that were on average $5.2 \%$ smaller than the latter. However, for the HaCaT sample with the NNC and TVIC reconstruction methods, even at the highest quality setting it was not able to meet the metrological target RMSE. Comparing the reconstruction methods NNC and TVIC with each other, we see that TVIC is less sensitive to compression noise, requiring less bitrate to reach the RMSE $10^{-4}$ in all tested scenarios.

Tables Icon

Table 2. Bitrate required by various codecs and tomographic reconstruction methods for achieving RI RMSE less than $10^{-4}$. (-) indicates codec was not able to satisfy distortion target even in its highest quality mode.

4. Conclusions

In this work, we investigated the compression behaviour of HEVC and JPEG 2000 standards for limited angle holographic tomography utilizing the spatial bandwidth optimized compression technique introduced in [16], where compression is performed on the camera measurements captured by the IP-OAH setup.

For objective and subjective analysis, object RI reconstructions were obtained using three different reconstruction algorithms (DI, NNC and TVIC), at several compression quality levels and for a selection of biological/technical specimens. For all specimens, we observe that compared to DI, the iterative reconstruction methods are more sensitive to the noise introduced by lossy compression.

Among the codecs, the best compression performance was obtained by Intra HEVC (single frame compression) rather than the Inter HEVC (multiple frames compression). Additionally, it was seen that due to the limitations in the maximal bitdepth of the encoded bitstream in the HEVC specification, it could not support very low distortions. Lower distortions were available with the JPEG 2000 codec.

For majority of the specimens, we demonstrate that with spatial bandwidth optimized compression, it is possible to obtain bitrates less than 1 bpp while still maintaining a high fidelity in the RI reconstruction. However, it should be noted that the compression performance is object–dependent in general. We observe this in one case in our study where the rate-distortion performance deviates strongly from other cases.

Funding

European Social Fund (POWR.03.03.00-00-PN13/18); Narodowe Centrum Badań i Rozwoju (PL-TW/V/5/2018); European Regional Development Fund; Fundacja na rzecz Nauki Polskiej (TEAMTECH/2016-1/4).

Acknowledgments

The research leading to the described results was carried out within the program TEAM TECH/2016-1/4 of Foundation for Polish Science, co-financed by the European Union under the European Regional Development Fund, Polish-Taiwanese Joint Research Project (PL-TW/V/5/2018) financed by the National Centre for Research and Development (Poland) and “Projekt PROM – Międzynarodowa wymiana stypendialna doktorantów i kadry akademickiej” (POWR.03.03.00-00-PN13/18), financed by the European Social Fund.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. E. Wolf, “Three-dimensional structure determination of semi-transparent objects from holographic data,” Opt. Commun. 1(4), 153–156 (1969). [CrossRef]  

2. A. C. Kak, M. Slaney, and G. Wang, Principles of computerized tomographic imaging (Society of Industial and Applied Mathematics, 2001).

3. A. Kuś, W. Krauze, P. L. Makowski, and M. Kujawińska, “Holographic tomography: hardware and software solutions for 3d quantitative biomedical imaging,” ETRI Journal 41(1), 61–72 (2019). [CrossRef]  

4. V. Balasubramani, A. Kuś, H.-Y. Tu, C.-J. Cheng, M. Baczewska, W. Krauze, and M. Kujawińska, “Holographic tomography: techniques and biomedical applications,” Appl. Opt. 60(10), B65–B80 (2021). [CrossRef]  

5. K. Kim, J. Yoon, and Y. Park, “Large-scale optical diffraction tomography for inspection of optical plastic lenses,” Opt. Lett. 41(5), 934–937 (2016). [CrossRef]  

6. V. Lauer, “New approach to optical diffraction tomography yielding a vector equation of diffraction tomography and a novel tomographic microscope,” J. Microsc. 205(2), 165–176 (2002). [CrossRef]  

7. S. Vertu, J.-J. Delaunay, I. Yamada, and O. Haeberlé, “Diffraction microtomography with sample rotation: influence of a missing apple core in the recorded frequency space,” Open Phys. 7(1), 22–31 (2009). [CrossRef]  

8. S. Vertu, J. Flügge, J.-J. Delaunay, and O. Haeberlé, “Improved and isotropic resolution in tomographic diffractive microscopy combining sample and illumination rotation,” Open Phys. 9(4), 969–974 (2011). [CrossRef]  

9. C. M. Fang-Yen, W. Choi, Y. Sung, C. J. Holbrow, R. R. Dasari, and M. S. Feld, “Video-rate tomographic phase microscopy,” J. Biomed. Opt. 16(1), 011005 (2011). [CrossRef]  

10. G. Dardikman, M. Habaza, L. Waller, and N. T. Shaked, “Video-rate processing in tomographic phase microscopy of biological cells using cuda,” Opt. Express 24(11), 11839–11854 (2016). [CrossRef]  

11. A. J. Lee, H. Hugonnet, W. Park, and Y. Park, “Three-dimensional label-free imaging and quantification of migrating cells during wound healing,” Biomed. Opt. Express 11(12), 6812–6824 (2020). [CrossRef]  

12. H. Hugonnet, Y. W. Kim, M. Lee, S. Shin, R. H. Hruban, S.-M. Hong, and Y. Park, “Multiscale label-free volumetric holographic histopathology of thick-tissue slides with subcellular resolution,” Adv. Photonics 3(02), 026004 (2021). [CrossRef]  

13. T. Bruylants, A. Munteanu, A. Alecu, R. Deklerck, and P. Schelkens, Volumetric image compression with JPEG2000, in SPIE Newsroom, (2007), pp. 1–2. [CrossRef]  

14. P. Stepien, R. K. Muhamad, M. Kujawinska, and P. Schelkens, Hologram compression in quantitative phase imaging, in Quantitative Phase Imaging VI, vol. 11249 Y. Liu, G. Popescu, and Y. Park, eds., International Society for Optics and Photonics (SPIE, 2020), pp. 80–91.

15. R. K. Muhamad, P. Stepień, D. Blinder, P. Schelkens, and M. Kujawińska, Holographic data compression for holographic microscopy and tomography in biomedical applications, in Imaging and Applied Optics Congress (Optical Society of America, 2020), p. HTh5D.1.

16. P. Stepień, R. K. Muhamad, D. Blinder, P. Schelkens, and M. Kujawińska, “Spatial bandwidth-optimized compression of image plane off-axis holograms with image and video codecs,” Opt. Express 28(19), 27873–27892 (2020). [CrossRef]  

17. E. Sánchez-Ortiga, A. Doblas, G. Saavedra, M. Martínez-Corral, and J. Garcia-Sucerquia, “Off-axis digital holographic microscopy: practical design parameters for operating at diffraction limit,” Appl. Opt. 53(10), 2058–2066 (2014). [CrossRef]  

18. S. J. LaRoque, E. Y. Sidky, and X. Pan, “Accurate image reconstruction from few-view and limited-angle data in diffraction tomography,” JOSA A 25(7), 1772–1782 (2008). [CrossRef]  

19. K. Kim, H. Yoon, M. Diez-Silva, M. Dao, R. R. Dasari, and Y. Park, “High-resolution three-dimensional imaging of red blood cells parasitized by plasmodium falciparum and in situ hemozoin crystals using optical diffraction tomography,” J. Biomed. Opt. 19(01), 1 (2013). [CrossRef]  

20. W. Krauze, P. Makowski, M. Kujawińska, and A. Kuś, “Generalized total variation iterative constraint strategy in limited angle optical diffraction tomography,” Opt. Express 24(5), 4924–4936 (2016). [CrossRef]  

21. W. Krauze, “Optical diffraction tomography with finite object support for the minimization of missing cone artifacts,” Biomed. Opt. Express 11(4), 1919–1926 (2020). [CrossRef]  

22. M. Kujawińska, W. Krauze, M. Baczewska, A. Kuś, and M. Ziemczonok, Comparative study of laboratory and commercial limited-angle holographic tomography setups, in Quantitative Phase Imaging V, vol. 10887 (International Society for Optics and Photonics, 2019), p. 1088708.

23. D. Jin, R. Zhou, Z. Yaqoob, and P. T. So, “Tomographic phase microscopy: principles and applications in bioimaging,” JOSA B 34(5), B64–B77 (2017). [CrossRef]  

24. Nanolive SA, Nanolive, https://nanolive.ch (2020). [Online; accessed 25-August-2021].

25. Tomocube Inc., Tomocube, http://www.tomocube.com/ (2020). [Online; accessed 25-August-2021].

26. M. Ziemczonok, A. Kuś, P. Wasylczyk, and M. Kujawińska, “3d-printed biological cell phantom for testing 3d quantitative phase imaging systems,” Sci. Rep. 9(1), 18872 (2019). [CrossRef]  

27. J. W. Goodman, Introduction to Fourier optics (Roberts and Company Publishers, 2005).

28. A. Skodras, C. Christopoulos, and T. Ebrahimi, “The jpeg 2000 still image compression standard,” IEEE Signal Processing Magazine 18(5), 36–58 (2001). [CrossRef]  

29. P. Schelkens, A. Skodras, T. Ebrahimi, and M. Kriss, The JPEG 2000 Suite, Wiley-IS&T Series in Imaging Science and Technology (Wiley, 2009). Michael A. Kriss.

30. D. Taubman and M. Marcellin, JPEG2000 Image Compression Fundamentals, Standards and Practice (Springer Publishing Company, Incorporated, 2013).

31. D. Taubman, A. Naman, and R. Mathew, High throughput JPEG 2000 (HTJ2K): Algorithm, performance and potential, (2019).

32. G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (hevc) standard,” IEEE Transactions on Circuits and Systems for Video Technology 22(12), 1649–1668 (2012). [CrossRef]  

33. T. Nguyen, P. Helle, M. Winken, B. Bross, D. Marpe, H. Schwarz, and T. Wiegand, “Transform coding techniques in hevc,” IEEE Journal of Selected Topics in Signal Processing 7(6), 978–989 (2013). [CrossRef]  

34. D. Flynn, D. Marpe, M. Naccari, T. Nguyen, C. Rosewarne, K. Sharman, J. Sole, and J. Xu, “Overview of the range extensions for the hevc standard: Tools, profiles and performance,” IEEE Trans. Circuits Syst. Video Techn. 26, 1 (2015). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1. The general scheme of data capture and processing pipeline towards metrological assessment and comparison of tomographic results. The “compression” and “decompression” blocks refer to the procedures discussed in Sec. 2.4 and Fig. 3.
Fig. 2.
Fig. 2. The experimental setup and the selected micro-objects (X-Y cross section of DI-reconstructions): a) Schematic of the used HT setup, b) HaCaT cells, c) SHSY-5Y cell (163% zoom), d) Skin epithelial cell, e) 3D printed cell phantom (163% zoom). The optical elements in a) are: LASER – laser light source, C – collimator lens, M – mirror, GM – galvanometric mirror, BF – beam formation module, S – sample, MO – microscope objective, AP – MOs aperture plane, TL – tube lens, S’ – image plane, CAM – camera. Red colored beam indicates a light scattered by the object ($u_{s\phi }$). Cross section through the AP shows how the illuminating beam $u_{i\phi }$ (green) and scattered beam $u_{s\phi }$ (red) are distributed with oblique illumination by the moving GM.
Fig. 3.
Fig. 3. The pipeline for the hologram encoding (a) and decoding (b) procedures (referenced in Fig. 1). In the decoding pipeline the green arrow indicates the stage where the decoding procedure may shortcut to perform reconstruction, when the hologram representation is not required. The complex data is indicated by the $\mathbb {C}$ symbol. The spectrum representation is visualised as $\log _{10} \left ( \left | U_{\mathbf {\phi }} \right | \right )$, while the spatial domain is visualized as $\left | u_{\mathbf {\phi }} \right |^{2}$. Compared with the method proposed in the previous work [16] the only difference is the addition of the circular shift of most energetic Fourier component to the center of frequency domain.
Fig. 4.
Fig. 4. Reconstruction RMSE distortion in function of sinogram SNR.
Fig. 5.
Fig. 5. The visualization of the errors regions with high spatial frequency information obtained for consecutive compression bitrates with the a) Inter HEVC, b) Intra HEVC and c) JPEG 2000 codec. Above the error maps, the global RMSE is indicated. The reference region is the USAF structure present in the cell phantom object. The cross-section is made within the depth of focus region.
Fig. 6.
Fig. 6. Rate-distortion plots showing the degradation of the reconstructed volumes in terms of RMSE (vertical axes) versus bitrate (horizontal axes). The threshold RMSE value of $10^{-4}$ is indicated by dashed lines. Each column represents a given object – a) HaCaT, b) SHSY-5Y, c) Epithelial cell, d) Cell phantom.

Tables (2)

Tables Icon

Table 1. The measurement parameters for each object used in this work.

Tables Icon

Table 2. Bitrate required by various codecs and tomographic reconstruction methods for achieving RI RMSE less than 10 4 . (-) indicates codec was not able to satisfy distortion target even in its highest quality mode.

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

u ϕ ( x ) = u i ϕ ( x ) + u s ϕ ( x ) .
RMSE = ( RI [ x , y , z ] RI ^ [ x , y , z ] ) 2 n ,
SNR = 10 log 10 ( u [ x , y ] u [ x , y ] | u [ x , y ] u ^ [ x , y ] | 2 )
Bitrate (bpp) = Number of bits in bitstream Number of pixels in camera sensor .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.