Digital image correlation and tracking

Digital image correlation and tracking is an optical method that employs tracking and image registration techniques for accurate 2D and 3D measurements of changes in images. This method is often used to measure full-field displacement and strains, and it is widely applied in many areas of science and engineering. Compared to strain gauges and extensometers, digital image correlation methods provide finer details about deformation, due to the ability to provide both local and average data.

Overview

Digital image correlation (DIC) techniques have been increasing in popularity, especially in micro- and nano-scale mechanical testing applications due to their relative ease of implementation and use. Advances in computer technology and digital cameras have been the enabling technologies for this method and while white-light optics has been the predominant approach, DIC can be and has been extended to almost any imaging technology.

The concept of using cross-correlation to measure shifts in datasets has been known for a long time, and it has been applied to digital images since at least the early 1970s.^[1][2] The present-day applications are almost innumerable, including image analysis, image compression, velocimetry, and strain estimation. Much early work in DIC in the field of mechanics was led by researchers at the University of South Carolina in the early 1980s^[3][4][5] and has been optimized and improved in recent years.^[6] Commonly, DIC relies on finding the maximum of the correlation array between pixel intensity array subsets on two or more corresponding images, which gives the integer translational shift between them. It is also possible to estimate shifts to a finer resolution than the resolution of the original images, which is often called "sub-pixel" registration because the measured shift is smaller than an integer pixel unit. For sub-pixel interpolation of the shift, other methods do not simply maximize the correlation coefficient. An iterative approach can also be used to maximize the interpolated correlation coefficient by using non-linear optimization techniques.^[7] The non-linear optimization approach tends to be conceptually simpler and can handle large deformations more accurately, but as with most nonlinear optimization techniques ^{[citation needed]}, it is slower.

The two-dimensional discrete cross correlation ${\displaystyle r_{ij}}$ can be defined in several ways, one possibility being:

${\displaystyle r_{ij}={\frac {\sum _{m}\sum _{n}[f(m+i,n+j)-{\bar {f}}][g(m,n)-{\bar {g}}]}{\sqrt {\sum _{m}\sum _{n}{[f(m,n)-{\bar {f}}]^{2}}\sum _{m}\sum _{n}{[g(m,n)-{\bar {g}}]^{2}}}}}.}$

Here f(m, n) is the pixel intensity or the gray-scale value at a point (m, n) in the original image, g(m, n) is the gray-scale value at a point (m, n) in the translated image, ${\displaystyle {\bar {f}}}$ and ${\displaystyle {\bar {g}}}$ are mean values of the intensity matrices f and g respectively.

However, in practical applications, the correlation array is usually computed using Fourier-transform methods, since the fast Fourier transform is a much faster method than directly computing the correlation.

${\displaystyle \mathbf {F} ={\mathcal {F}}\{f\},\quad \mathbf {G} ={\mathcal {F}}\{g\}.}$

Then taking the complex conjugate of the second result and multiplying the Fourier transforms together elementwise, we obtain the Fourier transform of the correlogram,${\displaystyle \ R}$:

${\displaystyle R=\mathbf {F} \circ \mathbf {G} ^{*},}$

where ${\displaystyle \circ }$ is the Hadamard product (entry-wise product). It is also fairly common to normalize the magnitudes to unity at this point, which results in a variation called phase correlation.

Then the cross-correlation is obtained by applying the inverse Fourier transform:

${\displaystyle \ r={\mathcal {F}}^{-1}\{R\}.}$

At this point, the coordinates of the maximum of ${\displaystyle r_{ij}}$ give the integer shift:

${\displaystyle (\Delta x,\Delta y)=\arg \max _{(i,j)}\{r\}.}$

Deformation mapping

For deformation mapping, the mapping function that relates the images can be derived from comparing a set of subwindow pairs over the whole images. (Figure 1). The coordinates or grid points (x_i, y_j) and (x_i^*, y_j^*) are related by the translations that occur between the two images. If the deformation is small and perpendicular to the optical axis of the camera, then the relation between (x_i, y_j) and (x_i^*, y_j^*) can be approximated by a 2D affine transformation such as:

${\displaystyle x^{*}=x+u+{\frac {\partial u}{\partial x}}\Delta x+{\frac {\partial u}{\partial y}}\Delta y,}$

${\displaystyle y^{*}=y+v+{\frac {\partial v}{\partial x}}\Delta x+{\frac {\partial v}{\partial y}}\Delta y.}$

Here u and v are translations of the center of the sub-image in the X and Y directions respectively. The distances from the center of the sub-image to the point (x, y) are denoted by ${\displaystyle \Delta x}$ and ${\displaystyle \Delta y}$. Thus, the correlation coefficient r_ij is a function of displacement components (u, v) and displacement gradients

${\displaystyle {\frac {\partial u}{\partial x}},{\frac {\partial u}{\partial y}},{\frac {\partial v}{\partial x}},{\frac {\partial v}{\partial y}}.}$

Basic concept of deformation mapping by DIC

DIC has proven to be very effective at mapping deformation in macroscopic mechanical testing, where the application of specular markers (e.g. paint, toner powder) or surface finishes from machining and polishing provide the needed contrast to correlate images well. However, these methods for applying surface contrast do not extend to the application of free-standing thin films for several reasons. First, vapor deposition at normal temperatures on semiconductor grade substrates results in mirror-finish quality films with RMS roughnesses that are typically on the order of several nanometers. No subsequent polishing or finishing steps are required, and unless electron imaging techniques are employed that can resolve microstructural features, the films do not possess enough useful surface contrast to adequately correlate images. Typically this challenge can be circumvented by applying paint that results in a random speckle pattern on the surface, although the large and turbulent forces resulting from either spraying or applying paint to the surface of a free-standing thin film are too high and would break the specimens. In addition, the sizes of individual paint particles are on the order of μms, while the film thickness is only several hundred nanometers, which would be analogous to supporting a large boulder on a thin sheet of paper.

μDIC

Advances in pattern application and deposition at reduced length scales have exploited small-scale synthesis methods including nano-scale chemical surface restructuring and photolithography of computer-generated random specular patterns to produce suitable surface contrast for DIC. The application of very fine powder particles that electrostatically adhere to the surface of the specimen and can be digitally tracked is one approach. For Al thin films, fine alumina abrasive polishing powder was initially used since the particle sizes are relatively well controlled, although the adhesion to Al films was not very good and the particles tended to agglomerate excessively. The candidate that worked most effectively was a silica powder designed for a high temperature adhesive compound (Aremco, inc.), which was applied through a plastic syringe.

A light blanket of powder would coat the gage section of the tensile sample and the larger particles could be blown away gently. The remaining particles would be those with the best adhesion to the surface. While the resulting surface contrast is not ideal for DIC, the high intensity ratio between the particles and the background provide a unique opportunity to track the particles between consecutive digital images taken during deformation. This can be achieved quite straightforwardly using digital image processing techniques. Sub-pixel tracking can be achieved by a number of correlation techniques, or by fitting to the known intensity profiles of particles.

Photolithography and Electron Beam Lithography can be used to create micro tooling for micro speckle stamps, and the stamps can print speckle patterns onto the surface of the specimen. Stamp inks can be chosen which are appropriate for optical DIC, SEM-DIC, and simultaneous SEM-DIC/EBSD studies (the ink can be transparent to EBSD).^[8]

Digital volume correlation

Digital Volume Correlation (DVC, and sometimes called Volumetric-DIC) extends the 2D-DIC algorithms into three dimensions to calculate the full-field 3D deformation from a pair of 3D images. This technique is distinct from 3D-DIC, which only calculates the 3D deformation of an exterior surface using conventional optical images. The DVC algorithm is able to track full-field displacement information in the form of voxels instead of pixels. The theory is similar to above except that another dimension is added: the z-dimension. The displacement is calculated from the correlation of 3D subsets of the reference and deformed volumetric images, which is analogous to the correlation of 2D subsets described above.^[9]

DVC can be performed using volumetric image datasets. These images can be obtained using confocal microscopy, X-ray computed tomography, Magnetic Resonance Imaging or other techniques. Similar to the other DIC techniques, the images must exhibit a distinct, high-contrast 3D "speckle pattern" to ensure accurate displacement measurement.^[10]

DVC was first developed in 1999 to study the deformation of trabecular bone using X-ray computed tomography images.^[9] Since then, applications of DVC have grown to include granular materials, metals, foams, composites and biological materials. To date it has been used with images acquired by MRI imaging, Computer Tomography (CT), micro-CT, confocal microscopy,^[11] and lightsheet microscopy.^[12] DVC is currently considered to be ideal in the research world for 3D quantification of local displacements, strains, and stress in biological specimens. It is preferred because of the non-invasiveness of the method over traditional experimental methods.^[10]

Two of the key challenges are improving the speed and reliability of the DVC measurement. The 3D imaging techniques produce noisier images than conventional 2D optical images, which reduces the quality of the displacement measurement. Computational speed is restricted by the file sizes of 3D images, which are significantly larger than 2D images. For example, an 8-bit (1024x1024) pixel 2D image has a file size of 1 MB, while an 8-bit (1024x1024x1024) voxel 3D image has a file size of 1 GB. This can be partially offset using parallel computing.^[13][14]

Applications

Digital image correlation has demonstrated uses in the following industries:^[15]

• Automotive
• Aerospace
• Biological
• Industrial
• Research and Education
• Government and Military
• Biomechanics
• Robotics
• Electronics

It has also been used for mapping earthquake deformation.^[16]

DIC Standardization

The International Digital Image Correlation Society (iDICs) is a body composed of members from academia, government, and industry, and is involved in training and educating end-users about DIC systems and the standardization of DIC practice for general applications.^[17] Created in 2015, the iDIC ^[18] has been focused on creating standardizations for DIC users.^[19]

See also

• Optical flow
• Stress
• Strain
• Displacement vector
• Particle Image Velocimetry
• Digital Image Correlation for Electronics

References

1. P. E. Anuta, "Spatial registration of multispectral and multitemporal digital imagery using fast Fourier transform techniques", IEEE Trans. Geosci. Electron., vol. GE-8, p. 353–368, Oct. 1970.
2. T. J. Keating, P. R. Wolf, and F. L. Scarpace, "An Improved Method of Digital Image Correlation", Photogrammetric Engineering and Remote Sensing 41(8): 993–1002, (1975).
3. T. C. Chu, W. F. Ranson, M. A. Sutton, W. H. Peters, Exp. Mech. 25 (1985), 232.
4. H. A. Bruck, S. R. McNeill, M. A. Sutton, W. H. Peters III, Exp. Mech. 29 (1989), 261.
5. W. H. Peters, W. F. Ranson, Opt. Eng. 21 (1982), 427.
6. E.g. M. A. Sutton, J.-J. Orteu, H. W. Schreier, Book - Image Correlation for Shape, Motion and Deformation Measurements, Hardcover ISBN 978-0-387-78746-6.
7. J. Yang, K. Bhattacharya, "Augmented Lagrangian Digital Image Correlation", Exp. Mech. 59 (2019), 187-205. Matlab code: https://www.mathworks.com/matlabcentral/fileexchange/70499-augmented-lagrangian-digital-image-correlation-and-tracking
8. Ruggles TJ, Bomarito GF, Cannon AH, and Hochhalter JD, "Selectively Electron-Transparent Microstamping Toward Concurrent Digital Image Correlation and High-Angular Resolution Electron Backscatter Diffraction (EBSD) Analysis", Microscopy and Microanalysis, 2017.
9. Bay BK, Smith TS, Fyhrie DP, Saad M (1999) Digital volume correlation: Three-dimensional strain mapping using X-ray Tomography. Exp Mech 39(3):217–226.
10. Jianyong Huang, Xiaochang Pan, Shanshan Li, Xiaoling Peng, Chunyang Xiong, and Jing Fang (2011) A Digital Volume Correlation Technique for 3-D Deformation Measurements of Soft Gels. International Journal of Applied Mechanics 3(2) 335-354.
11. Jaiswal, Devina; Moscato, Zoe; Tomizawa, Yuji; Claffey, Kevin P.; Hoshino, Kazunori (2019-05-01). "Elastography of multicellular spheroids using 3D light microscopy". Biomedical Optics Express. 10 (5): 2409–2418. doi:10.1364/BOE.10.002409. ISSN 2156-7085. PMC 6524572. PMID 31143496. Retrieved 2024-07-18.
12. Tomizawa, Yuji; Wali, Khadija H.; Surti, Manav; Suhail, Yasir; Kshitiz; Hoshino, Kazunori (2024). "Lightsheet microscopy integrates single-cell optical visco-elastography and fluorescence cytometry of 3D live tissues". bioRxiv. doi:10.1101/2024.04.20.590392. PMC 11100606. PMID 38766194.
13. M. Gates, J. Lambros & M. T. Heath (2011) Towards High Performance Digital Volume Correlation. 51 491–507
14. J. Yang, L. Hazlett, A. K. Landauer, C. Franck, "Augmented Lagrangian Digital Volume Correlation". Exp. Mech. (2020). Matlab code: https://www.mathworks.com/matlabcentral/fileexchange/77019-augmented-lagrangian-digital-volume-correlation-aldvc
15. "Correlated Solutions – Applications". correlatedsolutions.com. Retrieved 19 October 2017.
16. Van Puymbroeck, Nadège; Michel, Rémi; Binet, Renaud; Avouac, Jean-Philippe; Taboury, Jean (2000). "Measuring earthquakes from optical satellite images". Applied Optics. 39 (20): 3486–3494. Bibcode:2000ApOpt..39.3486V. doi:10.1364/AO.39.003486. PMID 18349918.
17. "Mission". Archived from the original on 2020-03-12.
18. "Society for Experimental Mechanics". sem.org. Retrieved 2021-07-25.
19. iDICs. "Guide · iDICs". iDICs. Retrieved 2022-11-02.

External links

• Mathematica ImageCorrelate function
• Using Digital Image Correlation to Measure Strain on a Tubine Blade
• Image Systems DIC
• DIC in Electronic Design
• DIC Applications in Aerospace
• 3D Optical Strain Measurements
• The International Digital Image Correlation Society (iDICs)