DIC Challenge 2.0: Developing Images and Guidelines for Evaluating Accuracy and Resolution of 2D Analyses

The DIC Challenge 2.0 follows on from the work accomplished in the first Digital Image Correlation (DIC) Challenge Reu et al. (Experimental Mechanics 58(7):1067, 1). The second challenge was required to better quantify the spatial resolution of 2D-DIC codes. The goal of this paper is to outline the methods and images for the 2D-DIC community to use to evaluate the performance of their codes and improve the implementation of 2D-DIC. This paper covers the creation of the new challenge images and the analysis and discussion of the results. It proposes a method of unambiguously defining spatial resolution for 2D-DIC and explores the tradeoff between displacement and strain noise (or measurement noise) and spatial resolution for a wide variety of DIC codes by a combination of the images presented here and a performance factor called Metrological Efficiency Indicator (MEI). The performance of the 2D codes generally followed the expected theoretical performance, particularly in the measurement of the displacement. The comparison did however show that even with fairly uniform displacement performance, the calculation of the strain spatial resolution varied widely. This work provides a useful framework for understanding the tradeoff and analyzing the performance of the DIC software using the provided images. It details some of the unique errors associated with the analysis of these images, such as the Pattern Induced Bias (PIB) and imprecision introduced through the strain calculation method. Future authors claiming improvements in 2D accuracy are encouraged to use these images for an unambiguous comparison.


Introduction
Digital Image Correlation (DIC) is a full-field optical measurement technique widely used in experimental mechanics for quantitative measurements in material testing.2D-DIC uses a single camera to acquire an image used to measure in-plane displacement by correlation of the grey levels between a reference and a deformed image.Strain is subsequently calculated from this displacement field.In engineering, DIC is often used in material testing, where calculating the displacement and strain are important in characterizing the material.The focus is made here on the assessment of the metrological performance of the DIC codes themselves.The proposed method relies on synthetic images to focus only on the error associated with the image processing.Other experimental sources of error, such as the camera or misalignement and lens aberrations, are thus not taken into account.They are, however, expected to impact all the full-field measurement techniques studied here in the same manner.In practice, the experimentalist must be aware of these error sources and minimize them with meticulous experiments to reach the metrological performance limits shown in this work.The first DIC challenge [1] generated 19 image sets that can be used for DIC software verification and validation.Verification determines whether the DIC software has been implemented correctly.Validation checks whether DIC returns measurements that are an accurate reflection of the physical reality.For the first DIC Challenge, verification images were synthetically generated with two different numerical methods to test common issues associated with 2D-DIC, specifically for measuring displacement or strain.The validation images were gathered from experiments with supplementary displacement measurements to compare with the DIC displacement measurements.All are encouraged to use the images for publications, particularly when claiming improvements in DIC algorithms.
While useful, the first DIC Challenge did not provide images with spatial variations of the displacement or the strain that were sufficient to thoroughly evaluate the displacement and strain filtering associated with 2D-DIC codes.
This deficiency led to the creation of the DIC Challenge 2.0, with images designed to easily visualize and assess the spatial resolution of DIC.

Spatial Resolution and Metrological Efficiency Indicator
Because both the DIC displacement calculation and the strain calculation are low-pass spatial filtering operations, attenuation bias can occur if the Quantity of Interest (QOI) varies significantly over a small region of the image.Considering harmonic displacements as an illustration, the amount of attenuation bias is related to the spatial period and magnitude of the displacements.The shorter the spatial period of the displacement the higher the frequency and the more attenuation occurs.This is illustrated in a 1D example in Fig. 1(a) using a displacement change of 1 pixel over 50 pixels to represent a rapidly changing QOI.The Fourier transform represents this ramp transition as a series of sine waves.The first six frequencies of the transform are shown in Fig. 1(b-g in (g).The blue curve is the ground-truth signal, representing an unfiltered signal.The red curve represents the filtered DIC signal of a 17 pixels subset DIC code using a first-order shape function.The DIC filtering is calculated using a first order Savitzky-Golay (SG) filter with a width of 17 pixels (see Appendix for more details) [2].The progressive attenuation of the signals, from no attenuation in (b) to a highly attenuated signal in (g) clearly illustrates the low-pass filtering effect of the DIC calculation.The peak-to-peak attenuation amount is shown in the figure title as a percent of the groundtruth value.The lower frequency (Mode 1) exhibits almost no attenuation at 5%, while the highest frequency (Mode 11) exhibits 92% attenuation.This filtering effect is what causes the sharp displacement change in the GT (Fig. 1(a)) to exhibit a rounded transition in the SG filter data.The sharp displacement change at the beginning and end of the transition contain the information with the highest frequencies, while the more gradually sloped portion the lowest frequencies.DIC lowpass filtering fundamentally smooths high frequencies and thus limits rapid changes in the QOI.In other words, it limits the spatial resolution of the measurement, or "The minimum distance between two localized features that can be independently resolved."DIC low-pass filtering fundamentally limits the spatial resolution of the measurement, or "The minimum distance between two localized features that can be independently resolved."as defined in the DIC Good Practices Guide (GPG) [3] available from (www.idics.org).
The strain spatial resolution is also affected because of both the bias in the displacement measurements as just discussed, but also because the strain approximations explicitly include spatial filtering during their calculation from the displacement fields.This calculation varies depending on the implementation of the code, but for many methods, the strain calculation functions as a second low-pass filter.
It is worth noting that DIC deals with images that are always impaired with pixelwise acquisition noise.DIC spatial filtering does have the benefit of reducing the propagation of this noise to the kinematic measurement but may create biases as just discussed.The ability of DIC to resolve ever smaller periods of the QOI is, therefore, in competition with the measurement resolution also called displacement or strain noise.Indeed, more filtering of the DIC data results in a lower overall noise of the QOI, but at the expense of decreased spatial resolution.It is thus of great interest to understand the relationship between the spatial resolution, bias, and the measurement resolution or QOI noise.The first DIC challenge explored this using Sample 14 and Sample 15 that were analyzed in detail in that paper [1].However, Sample 14 did not provide small enough displacement or strain periods to adequately challenge the 2D-DIC codes.While Sample 15 had extremely small periods, i.e., a ramp function, it did not help visualize the filtering of the DIC code.The 2D Challenge 2.0 images were created to fix this problem by providing displacement and strain gradient images that have appropriate spatial periods to investigate the tradeoff between bias and measurement resolution in DIC.We will express the results using the Metrological Efficiency Indicator (MEI) to score the measurements.The MEI was originally proposed in [4] for the Localized Spectral Analysis (LSA, or Grid Method) and is the product of the spatial resolution at a given bias amount and the measurement resolution, where measurement resolution is defined as 1 of the noise floor.The lower the value of the MEI, the better the performance of the code.It has been extended to local DIC [5] and global DIC [6].Since it does not depend on the DIC analysis parameters associated which each measurement technique, such as subset or element size, this indicator enables direct comparison between different DIC codes [6].

Paper Content and Organization
The paper presents the results of the DIC Challenge 2.0."DIC Challenge 2.0: Image Creation, Participation and Collected Data" describes the data, i.e., images, that constitute the DIC Challenge 2.0, the participants, the collected data, i.e., displacement and strain measurements, and illustrates the full-field maps."Assessment of the Metrological Performances" details the proposed strategy for the assessment of the metrological performances of the collected measurements."Study of the Metrological Performance of Full-field Measurements" assesses and discusses the metrological performances of the provided DIC outputs.Conclusions and future work are then discussed in "Conclusions".

Image Creation
Table 1 lists the DIC Challenge 2.0 image sets available at the website [7].These images are freely downloadable and can be used with attribution in any journal publication.Their maintenance is supported by the Society for Experimental Mechanics (www.sem.org) and the International DIC Society (www.idics.org).There are six image sets, including images with and without added image noise.
Star 1 is a ±0.5 pixel sinusoidal vertical displacement amplitude "star" pattern with varying spatial frequency.The images are noise free.Noise-free in this context means that no image noise was added to the simulation, beyond the truncation of the pixel intensity when converting it from doubleprecision to discrete 8-bit integers as part of the image generation process (see "Image generation -the Boolean model").The inset figure for Star 1 (Table 1) illustrates the frequency variation across the image.The left of the image has a period of 10 pixels for the sinusoid decreasing towards the right hand of the image to a period of 150 pixels1 .Period, defined as 1/frequency, is more natural to use for discussing the variation of the displacement gradients in the image as it gives the distance between points that can be measured, i.e., spatial resolution.The Star 1 data set contains only a reference and a deformed image.Star 2 is the same deformation field as Star 1, but with a heteroscedastic image acquisition noise, modeled specifically on a FLIR (formerly PointGrey) 5 megapixel camera (GRAS-50S5C-C).The noise profile is shown in Fig. 2. In what follows, image noise refers to this heteroscedatic noise.Star 2 contains three images, a noisy reference, an undeformed noise floor image, and a deformed noisy image.The undeformed noise floor image is used to calculate the measurement resolution of the QOI in the absence of motion, while the deformed image is used to calculate the spatial resolution.Star 3 is a noise-free ±5% Lagrangian vertical ( yy = ±0.05[-]) sinusoidal strain field with varying spatial period.The spatial period also linearly varies from 10 pixels to 150 pixels across the image.Only two images are provided, a reference and a deformed image.Star 4 is the same strain deformation image with noise added (Fig. 2).Three images are again provided, a reference, an undeformed noise floor image and a deformed image to measure the spatial resolution.After a preliminary analysis revealed that the DIC filtering was creating too much attenuation at even the largest period, two more image sets were added that were twice the width in order to include lower periods.Star 5 is like Star 2, with noise and a ±0.5 pixel vertical displace- ment field.Star 6 is Star 4 with noise and a vertical strain field of ±5% (Lagrangian).Star 5 and Star 6 both have three images, a noisy reference image, a noise floor image and a noisy deformed image used to calculate the spatial resolution.Star 5 and Star 6 in Table 1 are 4000 × 500 pixels and their underlaying displacements have periods, , varying linearly between 10 pixels at the first pixel, x = 1 pixels, to 300 pixels at x = 4000 pixels according to equation (1).All the inset results in Table 1 are "typical" DIC results and not the theoretical displacement or strain fields.The applied fields all continue smoothly across the image.The results shown in this paper illustrate the typical effect of filtering of the DIC codes (Fig. 4) -and are the prime topic of this paper.

Image Generation -the Boolean Model
The methods used for image generation are important when using them to test the accuracy of DIC codes and assess their metrological performance.At a minimum, they should not introduce any significant bias and preferably have realistic patterns and image noise applied.In the literature, several synthetic image generation techniques exist [8][9][10][11][12].They begin with a reference image I ref defined over a domain Ω .Next, a deformed image, I cur , is created that corresponds to the reference image but deformed with a ground truth displacement u GT .This deformed image is mathematically represented by: I ref is defined pixelwise.When the displacement is a rigid body motion, the calculation of I cur can be performed by a single phase shift of the image in the Fourier domain [13].However, when the displacement varies across the image (1) = 10 300 − 10 4000 (i.e., there is strain), more complex image interpolation is required.To reduce the bias that this interpolation introduces, some approaches use a binning approach [8,14], where Ĩref and Ĩcur are defined on a super-resolved pixel grid.The calculations of I ref and I cur are then realized by summation.
The synthetic image generation adopted here relies on a closed-form expression of the image displacements and corresponding grey values to prevent any bias caused by the interpolation.The closed-form model corresponds to a Boolean model, which creates a realistic pattern consisting of black speckles on a white background.The black speckles are circles, whose center coordinates were uniformly distributed on the image domain, and whose radii follow an exponential distribution.An integration is then performed from this infinite spatial resolution image to the pixelwise image.A Monte Carlo scheme is introduced, with a number of realizations ensuring that integration error is lower than the rounding error that corresponds to the gray value truncation when converting to 8-bit images.The code used for the generation of the data set is available here: https:// membe rs.loria.fr/ FSur/ softw are/ BSpec kleRe nder/ [15].The parameters used for defining the Boolean model and its generation are given in Table 2.

Participation
The DIC challenge was open to all participants with the goal to ensure that the commercial DIC codes were well represented as well as open source (e.g., DICe and Ncorr [16]), university codes (e.g., ALDIC [17]), and global codes (e.g., Correli).In addition to the variety of code implementations, there were variations on the analysis that were used, including subset weighting and higher-ordered shape functions, that are listed without attribution.These variations are important to include as they influence the DIC results.Table 3 lists the codes participating.The codes are not listed in order and are Fig. 2 Heteroscedastic image acquisition noise profile used for the 2D Challenge images for Star 2 and Star 4. This is constructed specifically using the noise profile on data from a PointGrey Grasshopper 5 megapixel, 8-bit, monochrome CCD camera.The left axis shows the grey level variance superimposed for each pixel value found in an image, while on the right axis the variance is normalized by the signal intensity (i.e., grey level).Simulated camera noise is elaborated as detailed in [15]

Data Collected
This section illustrates the results from the Star 5 and Star 6 images.The larger images were needed to ensure that the spatial period was long enough that all codes could process these images and almost reach the ground truth displacement using a range of software settings.Because there were multiple software settings that could be varied, we fixed the subset size for the local codes to between 9 × 9 pixels to 59 × 59 pixels in increments of 10 pixels.For most implemen- tations, this range yielded a variety of filtering and noise results.The global codes were asked to vary the element size (or other similar filtering parameter) to produce a range of results from the highest possible spatial resolution to one with "reasonable" filtering.A similar directive was given for the virtual strain gauge (VSG) size: "Choose any subset or variety of subset or element sizes to produce results that range from the highest spatial resolution to the lowest".This vague direction was required because of the wide variety of strain calculation implementations that were employed by the participating codes.The results were created by the respective code owners/ experts and submitted to the DIC Challenge committee via a comma separated file (.csv).The analyzed results were selected from the center row (referred to here as a line-cut) of the DIC image results and reported with a 1 pixel horizontal spacing for both the noisy undeformed image and the noisy deformed image.The file starts with a short header that contains information on the subset, element size, VSG size or other parameters deemed important by the participant.The code output was arranged in columns of data with the first row indicating the horizontal pixel position with 1 pixel spacing for both the noise floor image and for the deformed image.The 1 pixel spacing was important in the regions with highest spatial frequency to avoid interpolating the DIC results and, therefore, introducing more filtering.An interested reader can refer to [5] where the advantages of such dense DIC on metrological performances is discussed.Figure 3 shows a typical displacement line-cut obtained here with technique Local.Affine.B.

Illustration of Full-field Results
All participants submitted a line slice in the horizontal direction of the vertical displacement obtained for both the deformed image and the noise image.These are shown in Fig. 4(a) for a typical local DIC code for the 5 subset sizes.This plot illustrates the tradeoff between filtering of the DIC code and measurement resolution.That is, results with the highest spatial resolution also have the highest measurement resolution, and vice versa for the largest subset size.Figure 4(b) similarly shows typical results for a global DIC code for a variety of length parameters (element sizes).Figure 5 shows the strain results for a local DIC code.These results are representative of nearly all the submitted results.

Assessment of the Metrological Performances
The thorough assessment of the metrological performances necessitates the evaluation of the measurement resolution, n, and of the spatial resolution, , to calculate, in turn, the Metrological Efficiency Indicator MEI.The noise image line-cut was used to calculate the standard deviation ( n = 1 ) of the displacement measurement resolution and is reported as the QOI measurement resolution for the results.The deformed image line-cut was used to calculate the spatial resolution, defined here as a cutoff period.Indeed, the central row is at a constant amplitude of ±0.5 pixel vertical displacement or ±5% yy strain across the entire image.For both the dis- placement and strain results, an attenuation of the signal of 10% was chosen as a reasonable cutoff for defining the spatial resolution, denoted thus 10% .The value chosen for the attenuation is crucial for the evaluation of the metrological performances.It has been shown in [6] how its value impacts the results for local and global versions of DIC.The attenuation of 10% chosenn here corresponds to a middle regime which does not give an advantage to one specific version of DIC, global versions being particularly robust for high attenuation and whereas local versions behave well with low attenuation, cf [6].
The jagged line in Fig. 3(a) corresponds to the Pattern Induced Bias (PIB).PIB is the consequence of the interaction between the shape function, the image intensity gradients and the displacement field, which causes the algorithm to converge away from the lowpass filtered version of the true answer [20,21].Synthetic data is use in Fig. 6 to highlight the PIB effect.For this purpose, two sets of 50 pairs of images are using the displacement field for the Star 1 image.The first set relies on 50 independent speckle patterns on top of which independent pixel noise is added to each image.The second set relies on one unique speckle pattern on top of which noise independent noise is added to each image.Figure 6 illustrates line cuts of the retrieved displacement maps from the image pairs, from which the red curve corresponds to the average value.The dashed lines shown in Fig. 6 are the theoretical displacement-bias result caused by the theoretical shape function attenuation.The averaged results from second set, i.e. from the 50 independent speckle patterns, is smoother due to the averaging out the PIB error.Such results clearly show that image pattern drives the DIC solution.The analytical link between shape function, image pattern and displacement field is proposed and discussed in [20].
Because the line cut from the results crossed the 10% fractional attenuation of the displacement magnitude point at multiple locations (see Fig. 7) due to the PIB, it was impossible to simply locate the spatial period 10% at which the signal first crossed the 10% fractional attenuation line from the raw data.Furthermore, as it would not be practical to ask all participants to submit results for 50 unique patterns for averaging to yield a smoother curve, we decided to fit a polynomial to the results to find a unique spatial period 10% where the DIC results first cross the 10% fractional attenuation.We determined that a 12th order polynomial is a good comprise providing the "best-fit" to capture the shape of the results, without overfitting.This determination was made using 2 metrics.
First, we compared this 12th order polynomial fit with the theoretical solution, which exists for non-weighted local DIC approach, cf.Appendix.Figure 7(a-b) shows DIC measurements obtained with a local DIC approach (Local.Affine.B, with a subset width of 17 pixels (a) or 45 pixels (b)), on top of which the theoretical solution (red line) and different fits: the yellow line is the 5th order polynomial fit and finally the purple line corresponds to the 12th order polynomial fit.Measured displacement is plotted versus the period of the vertical sine wave of the GT displacement used for the image generation.This clearly highlights the lowpass filtering effect of DIC and thus facilitates the calculation of the cutoff period 10% .It is worth noting that the 12th order polynomial fit is closer to the theoretical solution, whereas the 5th order polynomial fails for short periods.The inset emphasizes the values reached by these two fits close to the 10% fractional bias threshold.The spatial resolution that corresponds to where the fits cross the 10% bias threshold are 67.8 pixels (theoretical solution), 67.2 pixels (12th order polynomial fit) or 68.1 pixels (5th order polynomial fit).The difference is thus relatively small showing that the polynomial fits behave well.
Second, versatility of the fitting method was also studied.Indeed, the theoretical solution is method dependent.For instance, it does not exist for all DIC approaches, e.g.global and miscellaneous.It thus seemed prudent to choose a polynomial fit that reasonably matches the data while also reproducing the correct theoretical fit for the local affine solution shown Fig. 7(a-b).Figure 7(c-d) illustrates the fitting results when the proposed fits are applied to data provided from Global.A with elements of 75 pixels width (c) and Misc.B with length parameter set to 9 pixels (d).A fit is proposed, which best identifies the coefficients of the theoretical solution, in the least square sense.As expected, the Because the measurement techniques are based on a wide variety of algorithms, versatility offered by polynomial fits is necessary.The 5th order polynomial does not provide enough degrees of freedom to correctly represent the trend of the measured data, with the fit over-estimating the period by about 20 pixels in Fig. 7(d).This leaves the 12th order polynomial fit as the best compromise for determining the spatial resolution.This fit has been used for all data, both strain and displacement in the rest of the article.

Spatial Resolution versus Length Parameter, e.g., Subset/element Size
After fitting with the 12th order polynomial, the cutoff period 10% was found for different subset sizes (as reported by each participant) to define the spatial resolution for each code.The results for all codes are shown in Fig. 8.The theoretical spatial resolution for local DIC with an affine shape -Except for approaches Misc.C-D and Global.G, the spatial resolution obtained with the theoretical solution and a length parameter corresponding to the subset width appears as a generous estimate of a DIC code's expected resolution.We expect the DIC versions that perfectly match theoretical solutions to correspond to the straightforward implementations, whereas specific implementations can offer better performance.

Spatial Resolution versus Measurement Resolution
We investigated the tradeoff between spatial resolution and measurement resolution by plotting the QOI noise versus cutoff period for all codes in Fig. 9.The noise was calculated as the standard deviation of the displacement along the same line-cut for the undeformed, but noisy image.The different measurement resolution and resolution results corresponded to different subset sizes as shown for Local.Affine.B in Fig. 4. The other subsetbased codes have the leftmost point for their smallest subset reported.Global codes had the "smallest" practical element size on the left to largest element size on the right, generally corresponding to the trends observed in filtering for the subset size in the local codes.All codes showed the expected improved measurement resolution corresponding to an increase in spatial filtering as subset -As in Fig. 8, two clusters are easily identified, which correspond to affine and quadratic kinematics.In Fig. 9, clusters are defined by the spatial resolution as opposed to being defined by the length parameter.-For all methods, plotting the measurement resolution versus the spatial resolution for all given subset or element sizes highlights the fact that these quantities are inversely proportional.The slope is indeed almost exactly −1 , with the exception of assessing the lowest spatial resolution for Quad approaches (for the smallest length parameters) or Misc.F (also for the smallest parameters).This proportionality directly yields the definition of the Metrological Efficiency Indicator (MEI), defined as the product between the spatial resolution and the measurement resolution.

Metrological Efficiency Indicator
We rely here on the Metrological Efficiency Indicator (MEI) for comparing the metrological performances of the different measurement tools [4,5].When displacements are considered, MEI is the measurement resolution times the spatial resolution, i.e., the spatial period determined at a 10% cutoff via a 12th order polynomial.For strain, MEI corresponds to the square of the measurement resolution times the spatial resolution.It writes thus: with n the measurement resolution, and 10% the spatial resolution.The lower the value of the MEI, the better the performance of the code.Moreover, as already discussed in the literature, the MEI should remain constant when changing the subset size.A constant MEI is analytically demonstrated for another full-field measurement technique, i.e. the Localized Spectrum Analysis (LSA), and empirically observed for Local and Global DIC, cf [4][5][6].The MEI values for the codes considered here are shown in Fig. 10.It is worth noting that except for the few miscellaneous approaches, for which the property is not proven, all measurement techniques exhibit a constant Metrological Efficiency Indicator, when considering larger subset or element sizes, with the exception of the very smallest sizes (or lowest index In what follows, MEI is calculated as the average of the three lowest MEI values for each code. Figure 11 shows the MEI for all the codes plotted together.A wide variation can be seen between some of the codes, but most of the codes lay between ±15% of each other for a given kinematic approach (affine or quadratic).Relative differences are computed with respect to the identified "straightforward" local DIC implementations (labeled "Ref" in Fig. 11), that match the theory.They correspond to Local.Affine.B (bar with red edge) for the affine approaches and to Local.Quad.A Fig. 10 Metrological Efficiency Indicator (MEI) versus index of submission.The index of submission corresponds to the requested element or subset size ranges ordered from the smallest to the largest.
Except when it corresponds to the smallest element/subset size, submission index have a relatively low impact to the MEI for all but the miscellaneous codes Fig. 11 MEI of all the codes broken up by code type (lower is better).The y-axis of the right-hand side highlights the corresponding spatial resolution for a fixed measurement noise of ±0.01 pixel.Code Local.Affine.G is an outlier and not used in the calculation of standard deviation.Bar colors correspond to the same as in Figs. 8, 9 or 10. References used for calculating the percentage are defined by the edged bar, red for the approaches submitted or identified as affine and blue for the quadratic ones.Finally, background colors highlight the identified approach: affine corresponds to red and quadratic to blue (bar with blue edge) for the quadratic approaches.Because the MEI is the product of the displacement measurement resolution and the spatial resolution, we can easily calculate the spatial resolution at a uniform noise level for a fixed measurement resolution ±0.01 pixel, a common displacement noise floor.The local quadratic codes performed slightly better than the global codes as expected [22,23], which does not reflect the conclusion proposed in [24].The difference is primarily because the earlier comparisons did not take into account spatial resolution.This figure reveals that : -Methods cluster in two categories: affine and quadratic kinematics.These clusters are represented with the red backgrounds (for approaches submitted of identified as affine) and blue backgrounds (for approaches submitted or identified as quadratic).Since the MEI incorporates the measurement resolution, the categorization is slightly different than the one proposed considering the spatial resolution only, as illustrated in Fig. 8. -The metrological performances of the second order approaches are very close.The extrema scores are only a few percentages different when compared with the straightforward implementation.The latter thus seems to corresponds to the implementation that features the best metrological performance.-Measurement technique Local.Affine.C, whose associated participant submitted as affine kinematics, performs as well as quadratic kinematics.Specific post-processing may be implemented, such as deconvolution [25], which increases the metrological performance of affine strategies

Strain Results Analysis
For Star 6 with a Lagrangian ±5% strain in the vertical direc- tion ( yy ), the participants were requested to submit results over a range of virtual strain gauge sizes (VSG) from the smallest possible VSG up to one with "reasonable" filtering, but less than 10% decrease at the largest spatial periods on the right of the image.Initial results were checked and when needed, smaller VSG's were supplied.The VSG as defined in the GPG [3] depends on the local code's step size, subset size, and strain window used in the calculation of the strain.While this calculation is valid for local codes, it has no application for global or other miscellaneous implementations of both DIC and the strain calculation.
The VSG is therefore used in this context as shorthand for "the region of the image over which the strain is calculated."Because of the variety of codes used, the actual VSG size is not reported.The results, however, do include the smallest feasible VSG for that code implementation.For this evaluation, knowing the VSG size is not important.Instead, like the displacement, the strain noise (measurement resolution) and spatial resolution can be plotted as in Fig. 12. Nevertheless the MEI for the strain is defined as the squared of the spatial resolution times the measurement resolution.There is a larger spread in the results due to the extra choices that can be made in the strain calculation, however, the same trends occur, with the same tradeoff in spatial resolution and measurement resolution.The curves are not smooth and not all are monotonically decreasing, because multiple parameters can be changed at the same time causing jumps in either the spatial resolution or the measurement resolution.Nevertheless, almost all curves have a slope of −2 .This indicates that the spatial resolution is proportional to the square of the measurement resolution.
The MEI for the strain is summarized in Fig. 13, where Local.Affine.C and Local.Affine.G are not included because they were not submitted.In the figure, both the mean value and the standard deviation obtained with all the codes are shown in the red text and background.Relative percentages of the MEI with respect to the mean value are also shown for each code.One might expect that if the displacement MEI is approximately the same between two codes using the same strain calculation, they would produce about the same strain MEI.However, this does not seem to be the case when comparing Figs.11 and 13.The large impact of the strain calculation is clearly highlighted with the variation in affine strain MEI and its departure from the displacement MEI trends.For example, Local.Affine.D has the smallest strain MEI in the local affine category, but the largest displacement MEI in that same category (with the exception of Local.Affine.G).There is smaller variations in strain MEI for the quadratic codes (except Global.F).Generally, the quadratic kinematics outperformed the affine when calculating strain.The straightforward implementations Local.Affine.B and Local.Quad.A correspond to the DIC approaches that are the closest implementations to theory, as illustrated with the displacement results.With these implementations, strain gradients are numerically computed by central difference of the displacement pixel-wise maps.Obviously other affine methods must calculate strain in a different manner to achieve the lower strain MEI.Similarly, some quadratic approaches clearly offer better results than the simple central derivation used with Local.Quad.A.

Conclusions
Rather than functioning as a competition, it has always been the mission of the DIC Challenge board to encourage a healthy code development environment by sharing how the various codes compare with each other.Furthermore, each participant is given the code letter for their code, allowing them to see where they sit relative to all other codes, and allowing them to improve their code, or implement new approaches as needed.An important link between the measurement resolution and the spatial resolution has been highlighted in this work and is quantified via the MEI, emphasizing its relevancy.This clearly shows the direct tradeoff Fig. 14 Expected DIC results for the DIC Challenge 2.0 Star images using a SSD criterion and affine or quadratic shape functions.Notation h corresponds to the subset size between spatial resolution and the measurement resolution for the displacement results.Tuning DIC parameters for improving the measurement resolution may impair the spatial resolution depending on the gradients of the measured QOI.The results also show a much wider variation in the strain MEI, indicating that the method of calculation of the strain can have a very large impact on the results.While it seems that 2D-DIC is well understood, there is clearly a substantial amount of variation in the performance among the different codes.While spatial resolution is important, it is not the only criterion for deciding what methods to implement in a code.For example, analysis speed may be more important than bias, or tying directly to model results in the case of global DIC.However, when looking at full-field spatially varying data, this tradeoff between the measurement resolution and the spatial resolution is key.This work provides a useful framework for understanding the tradeoff and analyzing the performance of the DIC software using the provided images.It details some of the unique errors associated with the analysis of these images, such as the PIB and imprecision introduced through the strain calculation method.Our hope is that the images and techniques provided in the DIC Challenge become a standard verification and validation technique for new codes and developers to improves their DIC algorithms.

Comparison with Theory
For most implementations of DIC, the optimal displacement minimizes the SSD criterion on each subset.The corresponding optimal parameters q * satisfies: where I ref (resp.I cur ) is the reference (resp.current) image and v(x, q) is a polynomial basis function that is either 1st order (affine) or 2nd order (quadratic).Let u GT be the ground truth displacement.Considering an infinite resolution texture that tracks the underlying material, I ref (x) = I cur x + u GT (x) .Equation ( 4) can be rewritten as For most implementations of DIC, assuming that the image pattern corresponds to the perfect information carrier, it drives the DIC algorithm to the best solution.The expected DIC solution simply results to the projection of the ground truth displacement to the DIC kinematics, thus neglecting the effect of I cur in equation (5).This leads to (4) q * = Argmin q ∑ x∈Ω I ref (x) − I cur (x + v(x, q)) 2 (5) q * = Argmin q ∑ x∈Ω I def x + u GT (x) − I cur (x + v(x, q)) 2 .
This result was introduced in [26], experimentally validated in [13,27] and thoroughly mathematically demonstrated in [20].It simply corresponds to the effect of the well-known Savitzky-Golay filter.Even though these results are shown only for the SSD, most unweighted criteria lead to the same solution at convergence [28].These equations produced the expected DIC results for the DIC Challenge 2.0 Star image in Figs. 3, 6 and 8 and produced the curves in the Fig. 14 for affine (1st order) and quadratic (2nd order) shape functions.

Fig. 1
Fig.1Illustration of DIC acting as a low-pass filter on a 1D example.Blue is the original unfiltered signal (ground truth or GT) and red is the filtered DIC results.The filtering is calculated using a Savitzky-Golay (SG) filter approximation of the first order.See Appendix for more information.The SG filter is defined to represent a subset size of 17 pixels and linear shape function.Signal (a) represents a rapid QOI change over 50 pixels.The two dashed lines represent the sum-

Fig. 3 Fig. 4 Fig. 5
Fig. 3 (a) Vertical displacement obtained with Local.Affine.B along the line-cut obtained with a subset size of 17 pixels.The spatial period of the vertical sine wave used for the image generation (top axis) is proportional to the image position across the width of the image (bottom axis).(b) Line-cut location in image at the center row with a fixed amplitude of ±0.5 pixel with the DIC results overlaid

Fig. 6
Fig. 6 Line cuts from 50 displacements obtained when applying DIC to 50 image pairs defined with (a) different speckle patterns or (b) one unique speckle but different image noises.The red curve is the

Fig. 7
Fig. 7 Examples of different fitting methods using the DIC measurements.Measurement given by the Local.Affine.B technique (a-b), by the Global.A technique (c) and the Misc.B technique (d).In all plots, measurements are plotted in blue, theoretical solutions in red and the 5th and 12th order polynomial fits introduced for the calculation of

Fig. 8
Fig. 8 Spatial resolution versus the length parameter (subset/element size, or else) for all codes.Displacement results are considered here for the calculation of the spatial resolution.Black lines correspond

Fig. 12
Fig. 12 Star 6 spatial resolution versus strain measurement resolution or measurement resolution.Increasing VSG size going to the right

Table 1
DIC Challenge 2.0 Image sets are available for download at: https:// sem.org/ dicch allen ge/.There are 6 image sets as outlined in the table.The images show example computed displacements from DIC

Table 2
Parameters chosen for the image generations uniquely identified throughout the paper with a code identifier.Sample code identifiers are Local.Affine.A (indicating a local DIC code with affine subset shape functions), Local.Quad.A (local DIC code with quadratic shape functions), Misc.A (not enough information provided to categorize), and Global.A (global DIC code with varying implementations).

Table 3
DIC Challenge 2.0 Image participants (in no particular order) and type of code.Codes in the text are ordered Local.Affine.A, etc. and do not correspond to the order of the table.Data are available here:https:// drive.google.com/