Introduction

Hyperspectral imaging is a spectral imaging technique that combines spectroscopy and digital photography, thus yielding a spectrum at each pixel in the image of a scene. Due to the enhanced spectral resolution, hyperspectral imaging has been routinely used for many remote sensing applications such as monitoring agriculture and vegetation, and for detecting mineral and oil deposits1,2,3. Recently, hyperspectral imaging techniques have proliferated medical imaging4, including for cancer tissue detection5,6 and ophthalmology7,8. Furthermore, hyperspectral imaging is increasingly being used for industrial machine vision applications such as for food and process quality control9,10. As hyperspectral imagers are rapidly becoming more robust, compact and economical, this technology is poised to significantly revolutionize machine vision in many different settings.

One such setting is dynamic thin film interferometry11. Thin film interferometry is a technique that employs light interference to measure thickness of films that are usually a few microns thick. Thin film interferometry has been used since the early 1950’s to characterize static films12,13, and in the subsequent years the technique was adopted for studying dynamic thin liquid films that have a spatially and temporally evolving thickness profile14. This technique has since then become the conventional method for characterizing thin liquid films in research settings, primarily due to its simplicity and non-invasive nature. Some of the important areas that make use of this technique includes characterizing foam15,16 and emulsion17,18 stability, drop impacts19, tear films20,21,22 and liquid film coatings23. Despite the popularity of the technique in research settings, dynamic thin liquid film interferometry has not proliferated industrial or commercial settings.

The primary obstacle that prevents the wide spread commercial use of interferometry for characterizing dynamic thin films is the difficulty in automatically analyzing the interferograms to recover the underlying film thicknesses. The automatic analysis is complicated by the transcendental phase-periodic governing equations that non-uniquely relates the pixel intensities in interferograms to film thicknesses. This inherent complexity is amplified by the uncertainty introduced by the unavoidable imaging and background noise. As a result, the unconditional determination of the thickness of spatiotemporally evolving thin films from interferograms has remained a challenging research problem since the inception of this technique.

Figure 1
figure 1

Schematic of the compact experimental setup along with the details of hyperspectral camera used in this study. (a) The experimental arrangement used to record the hyperspectral interferograms. Here the camera, the lens and the light are vertically above the thin film. Further details are available in previously published works11. (b) Details of the fabry-perot filter array inside the snapshot hyperspectral camera. Each fabry-perot filter in the repeating filter array unit is numbered according to the ascending order of the peak wavelengths of the filters in that unit. (c) Five slices from the HSI cube showing the inteferograms obtained from a bubble in a silicone oil mixture along with two RGB composites generated by combining bands 1,8,16 and bands 4,12,16. The colormaps corresponding to the RGB composites are shown alongside. Such RGB composites are useful for visualizing hyperspectral interferograms, and in this case also qualitatively illustrates how hyperspectral imaging can overcome the non-uniqueness between film thickness and color associated with the traditional RGB interferometry.

Full size image

Researchers have traditionally side stepped this problem broadly by the following approaches. One of the approaches involves manually identifying the film thickness at a region utilizing a reference colormap11,24. This approach is quite robust, however it is quite slow and suffers from human subjectivity. Another common approach utilizes fringe counting from a known absolute reference thickness in the film25,26,27. This approach is quite fast but is not robust and requires assumptions on the spatial structure of the film. Yet another approach involves pre-calibrating the pixel intensities in interferograms to film thicknesses utilizing liquid films of known thickness profiles28,29. This approach is quite fast and robust, however, this technique is restricted to scenarios where such calibrations are possible.

In this manuscript we report an arrangement utilizing hyperspectral imaging for the unconditional and automated reconstruction of thickness profiles from thin liquid film interferograms. Approaches utilizing hyperspectral imaging (using the pushbroom like techniques) have previously been reported for characterizing static thin films30,31. However, these approaches are not suited for dynamic films, primarily due to the nature of image acquisition, and to the best of our knowledge have never been modified for use with dynamic films. Here, we describe a compact setup employing a snapshot hyperspectral imager and the related algorithms for automated determination of thickness profiles of thin films. In section “Methods”, we detail the theory, the experimental setup and the developed algorithms. In section “Results and discussion”, we establish the accuracy of this technique by comparing the absolute pixel-wise differences across the manually reconstructed and the automatically reconstructed thickness profiles. Finally, we conclude the paper by discussing the key advantages and unique characteristics of hyperspectral thin film interferometry.

Methods

Theory

The theory of thin film interference was formalized in the early 19th century by Fresnel, and has since been discussed by many researchers in the context of measuring the thickness of thin films such as for bubbles14, tear films32, and for surface profiling13. Here we will briefly develop a formulation relevant for a hyperspectral camera.

Consider a beam of light having intensity \(I_0(\lambda )\) incident on a thin liquid film of thickness d and refractive index \(n_2\). The film is bounded on top and bottom by media having refractive indices \(n_1\) and \(n_3\) respectively. Assuming normal incidence and non dispersive films, the reflected light intensity \(I(d,\lambda )\) emanating from the thin film can be written as,

$$\begin{aligned} \frac{ I(\lambda ,d)}{I_0(\lambda )}&= R_1 +R_2(1 - R_1)^2 + 2\sqrt{R_1R_2(1 - R_1)^2}\cos \left( \phi \right) \end{aligned}$$
(1)
$$\begin{aligned} \phi&= \frac{4\pi n_2 d}{\lambda } + \pi (\mathbb {1}(n_2>n_1)) + \pi (\mathbb {1}(n_3>n_2)). \end{aligned}$$
(2)

Here \(\lambda \) is the wavelength of light, \(\phi \) is the phase difference and \(\mathbb {1}\) is the indicator function that captures the phase phase shift of \(\pi \) radians that occurs when light passes in to a medium with a higher refractive index. \(R_1\) and \(R_2\) are the power (intensity) reflectivity coefficients obtained from the Fresnel equations evaluated for normal incidence, and are given by,

$$\begin{aligned} R_1 = \left( \frac{n_1 - n_2}{n_1 + n_2}\right) ^2 \end{aligned}$$
(3)
$$\begin{aligned} R_2 = \left( \frac{n_2 - n_3}{n_2 + n_3}\right) ^2. \end{aligned}$$
(4)

Finally, the intensity perceived by the ith channel of a pixel H in a hyperspectral camera as a function of the film thickness can be computed as,

$$\begin{aligned} H_i (d) = \int _{\lambda _0}^{\lambda _f} I(\lambda ,d) I_r(\lambda ) S_i(\lambda ) d\lambda . \end{aligned}$$
(5)

Here \(I_r(\lambda )\) is the spectral response of filters in the system, \(\lambda _0\) and \(\lambda _f\) are the smallest and largest wavelengths within the global band pass filter transmission window, and \(S_i(\lambda )\) is the spectral sensitivity of the \(i^{th}\) channel of a pixel. See Supplementary Materials for details on the assumption of normal incidence and an uncertainty analysis related to the refractive index.

Figure 2
figure 2

The spectral map and a block diagram showing the thickness reconstruction pipeline. (a) The spectral map obtained from Eq. 5 showing the theoretical spectral signature perceived by the different channels in the camera as a function of the film thickness. Here we have assumed a silicone oil film bounded by air on either side (refractive index triplet—[1 1.4 1]). The spectral response data of the camera is available in Supplementary Fig. S4. Note that there is a continuous variation of the spectrum as a function of the film thickness. (b) A block diagram summarizing the various steps involved in the thickness reconstruction pipeline.

Full size image

During an experiment (Fig. 1), a hyperspectral camera having h channels at every pixel will encode reflections from a thin film of thickness d as a h dimensional vector. Utilizing Eq.5, we can invert this h dimensional vector to recover the thickness of the thin film. In principle, when using a RGB camera this can be accomplished by first generating a color map that establishes a color to film thickness relationship (Fig. 1c), and subsequently using this map to assign thicknesses to colors in the interferogram. Unfortunately, due to the periodic nature of the cosine function, the generated color map for a RGB camera has nearly identical colors mapping to different thicknesses. Coupled with the imaging noise, the automated mapping of colors to interference patterns becomes infeasible (see Supplementary Materials). Hence in practice, a manual matching process is usually adopted, the details of which are available in Frostad et al.11. In the subsequent sections, we will detail the use of hyperspectral imaging for automatically reconstructing film thickness from interferograms utilizing a spectral map obtained from Eq. 5.

Experimental setup

The single bubble coalescence experiments used to validate the utility of hyperspectral imaging for thin film thickness measurements were performed using a modified Dynamic Fluid-Film Interferometer (DFI). The construction11 and the utility33,34,35,36 of the DFI has been previously discussed in a number of publications and in references therein.

For the current study, the DFI was modified to have a 16 channel snapshot HyperSpectral Imaging (HSI) camera (Model: MQ022HG-IM-SM4X4-VIS, Manufacturer: Ximea GmbH, Germany) having a maximum acquisition frame rate of 170 frames per second as its top camera (Fig. 1a). As the filter array inside the HSI camera has narrow spectral response bands (Supplementary Fig. S4), the dichroic triband filter utilized with the top light for the same purpose (reducing the FWHM of spectral bands of a RGB camera)11 was removed. The removal of the dichroic filter thus resulted in 120% increase in the luminous flux entering camera - improving the signal to noise ratio in the acquired hyperspectral interferograms. Further details of the setup including the make and model of the light are available in Frostad et al.11.

To benchmark the thin film measurement capability of the hyperspectral camera, single bubble experiments were also performed using RGB cameras (IDS UI 3060CP), commonly used for thin film interferometery11,34.

Figure 3
figure 3

The thickness reconstruction performance of the proposed snapshot hyperspectral interferometry technique. These hyperspectral interferograms were acquired over bubbles in silicone oil mixtures at 10 frames per second. (a) The RGB visualization of the hyperspectral interferogram (and its colormap) that is used to illustrate the reconstruction performance. (b) The manually reconstructed thickness profile11 (used here as the ground truth). (c) The corresponding automatically reconstructed thickness profile. (d) The automatically reconstructed thickness profile after optimization. (e) The absolute pixel wise height difference between the manual and the unoptimized thickness profile. Over 80% of the pixels differ by 100 nm or less. (f) The absolute pixel wise height difference between the manual and the optimized thickness profile. Over 90% of the pixels differ by 100 nm or less.

Full size image

Image processing

To recover the film thickness from the hyperspectral image, the following steps were executed utilizing Matlab. Initially, the raw images from the snapshot HSI camera were sliced and spliced appropriately to reconstruct the hyperspectral cubes. Subsequently, background subtraction was performed on the cubes, followed by cropping, intensity correction and normalization. Background subtraction is accomplished by imaging the measurement region in the absence of the thin film and subtracting this background spectrum from the measured spectrum. Intensity correction when necessary (for flat field corrections) is accomplished by imaging a surface of known reflectivity, and using the result to compute the correction mask for each spectral band. Intensity normalization is accomplished by normalizing the hyperspectral interferogram by the largest intensity in the interferogram. Subsequently, a k-Nearest neighbour search utilizing the cosine distance metric is performed between each pixel in the resulting HSI cube and the theoretical spectral map (Fig. 2a) generated from Eq.5. The thicknesses obtained as the first nearest neighbour in the k-Nearest neighbour search is used to construct the initial estimate of the thickness profile.

Finally, a spatial optimization algorithm is utilized to correct for the any incorrectly assigned points. The algorithm basically enforces the \(C^0\) spatial continuity in film thickness by replacing any incorrectly assigned thickness with an appropriate thickness from the k possible thickness values at that point. In practice, this is accomplished by identifying regions with unphysical gradients and utilizing the gradient descent technique to iteratively correct the thickness in these regions. All steps involved in the thickness reconstruction pipeline is summarized as a block diagram in Fig. 2b.

Figure 4
figure 4

(a) Robustness against noise as a function of number and type filters in a camera. The reconstruction accuracy is seen to increase with the number of channels in the camera. Further, a hypothetical camera with perfect (hat function) filters is seen to require the least number of channels for a high fidelity thickness reconstruction. (b) The cumulative pair wise distance distributions for the theoretical colormap (RGB camera) and spectralmap (HSI camera) entries. Here N is the number of contiguous thickness values in the theoretical maps starting at 0 nm and separated by 1 nm. The spectral entries corresponding to a hyperspectral camera are separated by a larger distance (r) as compared to that of a RGB camera.

Full size image

Results and discussion

Thickness reconstruction performance

The performance of the new camera system is established by reconstructing the dynamic film thickness of a bubble in a silicone oil mixture. The results before and after optimization are compared to the manually reconstructed thickness profiles11 in Fig. 3. The reconstructed thickness profile before optimization (Fig. 3c) broadly resembles the manually reconstructed profile (Fig. 3b). However, due to noise and spectral mixing in the interference data, there are regions in the reconstructed profile that have physically unrealistic gradients. These artifacts are removed utilizing the optimization routines, and the resultant thickness profile after optimization (Fig. 3d) is seen be very similar to the manually reconstructed profile. The mean film thickness (\(\iint T(x,y) dx dy /\iint dx dy \)), a very common metric used to report the thickness of thin films11,34,37, is very similar across the three cases with values of 358.9 nm, 344.4 nm and 347.9 nm for the manual, unoptimized and optimized thickness profiles respectively.

The absolute pixel-wise height difference between the manually reconstructed and the automatically reconstructed thickness profiles pre and post optimization are shown in Fig. 3e,f. Clearly, the reconstructions are very accurate with over 80% of the pixels differing by less than 100 nm for the unoptimized case, while over 90% of the pixels differ by less than 100 nm for the optimized case (see Supplementary Fig. S8 for more statistics). The remaining 10% of the regions have errors larger than 100 nm, and are most likely a result of inadequate background subtraction as well as due to the lack of pixel by pixel calibration information for the camera. As a reference, it is also worth noting that reconstructions using the 3 channel data available from a RGB camera has an unacceptable accuracy, with less than 20% of the pixels being classified within 100 nm (Supplementary Fig. S3).

Finally, a convenient advantage of the automated reconstruction is the ease of analyzing time sequential data to obtain the temporal evolution of the film thicknesses. Supplementary Video V1 shows one such reconstruction of a temporally evolving film thickness profile of a bubble in a silicone oil mixture alongside its corresponding raw interferogram.

Robustness against noise

Theoretically, the three channels in RGB interferograms are sufficient to disambiguate the underlying film thicknesses28. However in practice, the inherent noise in the acquired image data breaks this theoretical uniqueness between the RGB intensities and the film thickness; thus requiring optimization (even for static films)38 and/or calibration28 for thickness recovery. Unlike the RGB interferograms, hyperspectral interferograms are relatively more robust to noise.

To illustrate the robustness of hyperspectral interferograms against noise, we perform numerical experiments using a ramp thickness profile having a minimum thickness of 1 nm and a maximum thickness of 2,000 nm (Supplementary Fig. S1). This thickness profile is mapped to corresponding color interferograms using the color maps of the tested cameras. Subsequently, Gaussian noise (see Supplementary Text) is added to the images. Inset in Fig. 4a shows one such RGB image before and after addition of the noise (see Supplementary Fig. S6 for a block diagram detailing the pipeline for generating the test images). Reconstruction algorithms (without optimization) are then used to predict the thickness (T) at each pixel in the noisy images. T can be compared to the ground truth thickness (\(T^g\)) to quantify the reconstruction accuracy. We visualize the reconstruction accuracy in Fig. 4(a) by plotting the number of pixels having an error of less than 50 nm \(\left( \sum _i \mathbb {1}(|T^g_i - T_i|<50)\right) \) as a function of the number of channels in the camera.

From Fig. 4a we see that for an RGB camera only about 20% of the pixels are classified within acceptable error. However, for a HSI camera (utilizing all 16 channels) we are able to recover the thickness from 97% of the pixels within acceptable error; thus showing that HSI cameras are more robust to noise. For the tested HSI camera, we also obtain similar reconstruction accuracy using either the real or virtual filters (see Supplementary Figs. S4, S5 for the definitions and the spectral responses of the filters) on a HSI camera, thus suggesting any type of filter may be used in the reconstruction process. Further we also see that having an ideal light (uniform intensity across wavelengths) or ideal filters (responses given by hat functions) can enhance the reconstruction performance; with ideal filters having the greatest impact on reconstruction accuracy (see Supplementary Text for the rationale). Finally, its also interesting to note that the results indicate that a three channel camera (RGB camera) with ideal filters has a performance comparable to a regular HSI camera utilizing all 16 channels.

The increased robustness of HSI interferograms against noise is related to the higher dimensionality of the color co-ordinates that correspond to a given film thickness. As a result of the higher dimensionality, the pair wise Euclidean distances of HSI color co-ordinates are larger than the corresponding pair wise distances of color co-ordinates obtained from a RGB camera. This observation is quantified in Fig. 4b by plotting the cumulative pair wise distance distribution of the color co-ordinates (resolved at every nanometer) for a HSI and a RGB camera for two different film thickness ranges—[0, 2,000] and [0, 4,000] nm. From Fig. 4b, we clearly see that for a given thickness range, the curves corresponding to HSI cameras as compared to RGB cameras are shifted towards larger distances (r). Hence, as a consequence of the increased separation of HSI color co-ordinates, perturbations due to noise are less likely to result in a color co-ordinate becoming similar to another.

Absolute light intensity independent thickness reconstruction

Another advantage of using hyperspectral imaging is that the absolute intensity of light at any point in the interferogram may be neglected during thickness reconstruction. This can be shown to be true using two different arguments. Firstly, standard transformations (such as RGB to HSV) can isolate the intensity information into a single channel. Neglecting the intensity channel, still gives us (for the tested HSI camera) information from 15 channels to unambiguously (see Fig. 4) reconstruct the thickness. Since an RGB camera has only three channels, neglecting the intensity makes it impossible to unambiguously determine the thickness. Secondly, the cosine distance metric utilized in this paper to determine the film thicknesses completely ignores the intensity information. Despite ignoring the intensity information, we are able to recover the thickness profiles quite well (Fig. 3d), thus confirming that hyperspectral imaging enables thickness reconstruction without necessarily requiring information about the absolute light intensity.

A direct consequence of this result is that the reconstruction techniques described in this paper are robust against the natural spatial variation of incident light intensity (vignetting) over the interferogram. Hence in addition to obviating the need for accurately obtaining the absolute light intensities, the corrections for vignetting may also be conveniently avoided. Note that flat field corrections may still be required if there are spatial variations in pixel sensitivities in a camera.

Characteristics of hyperspectral thin film interferometry

Hyperspectral imaging when applied to thin film interferometry has some interesting characteristics. Firstly, the number of spectral classes is higher than in traditional hyperspectral imaging used for remote sensing or medical imaging. The number of spectral classes in thin film interferometry goes as \({\mathcal {O}}(T_{max}/T_{resolution})\), which is equal to 4,000 when trying to resolve film thickness of upto \( T_{max} = 4{,}000\;nm\) with a resolution of \( T_{resolution} = 1 \;nm\). Consequently, spectral matching routines used for thin film interferometry should have a high degree of specificity, and may also need optimization algorithms (as used in this paper) to completely reconstruct the thickness profile.

Secondly, the spectra vary gradually across spatially adjacent spectral classes. Unlike traditional hyperspectral imaging where there are no restrictions on spatially adjacent classes (and hence on the spectra), the spatial continuity of thin films restricts the spatially adjacent classes to correspond to contiguous film thicknesses. Hence, as a consequence of Fig. 2, there is a gradual variation of the spectrum across spatially adjacent classes. Consequently, spectral mixing (due to lack of sufficient camera resolution) does not pose difficulties during thickness reconstruction, thereby obviating any need for spectral unmixing techniques39.

Thirdly, as thin liquid films are dynamic, the spectral signatures change rapidly both in space and time. As a consequence, snapshot hyperspectral imaging (as opposed to techniques such as the pushbroom) is better suited for thin film interferometry.

Comparison to spectroscopic ellipsometry

We close the discussion by comparing the reported hyperspectral thin film interferometry technique, which in many ways is an improved form of spectroscopic reflectometry, to the commonly used spectroscopic ellipsometry. Ellipsometers utilize the change in the polarization state of light as it is reflected obliquely from a thin film to measure the film thickness. Ellipsometers are known for their sub-Angstrom precision, particularly when dealing with nanometric and sub-nanometric films40. Spectroscopic ellipsometers are a class of ellipsometers that employ a range of wavelengths to perform the measurement, resulting in an improved measurement range spanning a few Angstroms to tens of microns. In addition with the help of adequate models, they are also used to determine many optical and geometrical characteristics of thin films41.

The reported hyperspectral thin film interferometry technique currently has a measurement accuracy of about a 100 nm and can measure film that are between 100 nm and 4,000 nm. Despite having an inferior range and accuracy, hyperspectral interferometry complements spectroscopic ellipsometry by addressing the following two important limitations. Firstly, ellipsometers have a poor spatial resolution, often restricted to measuring the thickness at a single point. This makes ellipsometers unsuitable for characterizing dynamic liquid films that are evolving in both space and time. Secondly, measurements using ellipsometers require an oblique angle of incidence. This makes the imaging of thin films on curved geometries very challenging. The capability of the current hyperspectral technique to characterize dynamic liquid films over curved geometries makes this technique attractive to many technical fields including that of colloid and interface science where thin liquid films involving bubbles34,35,37, drops17,18 and the tear film20,21,22 are routinely studied.

Conclusions

In this paper we reported a compact setup employing snapshot hyperspectral imaging and the related algorithms for the automated determination of thickness profiles of dynamic thin liquid films. We showed that we can reconstruct dynamic thin film profiles to within 100 nm of those reconstructed manually. As manually reconstructed profiles themselves are only accurate upto 50 nm, the automatic reconstructed profiles are a faithful representation of the ground truth. We also demonstrated through numerical experiments that hyperspectral interferometry has two key advantages, namely, the increased robustness against imaging noise and the ability to neglect the absolute light intensity information during thickness reconstruction.

Future studies may be focused on improving both the hardware and software aspects presented in this work. Improvements in the spatial resolution, the sensor quantum efficiency, and the filter properties like transmissivity and FWHM of the hyperspectral imager will enhance the reconstruction performance of the system. Our results also suggest that improving the filters (especially the Q-factor) will have a significant impact on the robustness of the system. Finally improvements in image pre-processing (like background substraction) and in spectral matching algorithms (such as enhancing the specificity and robustness) will also aid in perfecting the proposed system.