Improving orientation mapping by enhancing the diffraction signal using Auto-CLAHE in precession electron diffraction data

INTRODUCTION

Precession electron diffraction (PED) is a powerful characterization technique used to obtain high-resolution crystal structure/orientation^[1-3] and elastic strain^[4-6] information about materials at the nanoscale level. Some notable PED applications include characterizing thin film microstructures^[7-10], nanocrystalline grain growth behavior^[11,12], material deformation behavior at large strains^[13,14], and submicron and nanoscale martensite variants^[15,16]. When applying PED, the electron beam in the transmission electron microscope (TEM) is converged to a small probe (~1-3 nm) and rasters on the specimen. Precession (typically 0.3°-0.8°) is applied to excite higher-order reflections and reduce the dynamical effect^[1,17,18]. The experimentally acquired diffraction patterns from each pixel are compared to the simulated diffraction patterns in a database to determine the crystal structure and orientation. The information is then used to create phase (crystal structure) and orientation maps. The PED-based orientation mapping is also termed automated crystal orientation mapping (ACOM)^[2,3,8,10]. Compared to electron backscatter diffraction (EBSD, another widely used orientation mapping technique, typical resolution ~20-50 nm), PED offers superior spatial resolution (~3 nm resolution in conventional field-emission gun TEM). In addition, PED offers a larger field of view compared to high-resolution TEM (HRTEM). HRTEM typically allows examination of lattice fringes to deduce crystal orientation in a field of view up to 50 × 50 nm², while PED can provide a field of view up to 6 × 6 μm². Hence, the PED-based orientation mapping fills the length-scale gap between HRTEM and EBSD.

The quality of PED orientation mapping depends heavily on the quality of the diffraction pattern. Various factors can affect the diffraction pattern signal, including sample thickness and crystal orientation. Generally, thinner samples are preferred as longer rel-rods intersect with the Ewald sphere, resulting in more diffraction spots and accurate template matching between the experimentally acquired and simulated diffraction patterns^[19]. Diffraction patterns acquired at or close to zone axes are also preferable since they offer more diffraction spots for template matching. Conversely, diffraction patterns acquired far away from zone axes have fewer spots, and the intensity of the spots that are not at the exact Bragg condition diminishes rapidly, potentially leading to poor orientation indexing. While sample thickness can be controlled by the researcher, it is difficult to control the exact diffraction condition of individual grains in polycrystalline or deformed single-crystal samples. Therefore, there is a need for a robust and efficient algorithm to enhance diffraction spot information when it is not ideal for template matching.

Previous work from various research groups has reported that preprocessing PED data is critical to realizing the full potential of the subsequent algorithms. For example, Bergh et al. demonstrated the importance of binning and center beam alignment before background removal and diffraction spot identification to resolve overlapping diffraction patterns^[20]. Zhao et al. effectively reduced noise in PED raw data through various filters (e.g., Gaussian, non-local means, and Wiener) to enable precise diffraction spot position identification and strain mapping^[4]. Folastre et al. employed sub-pixel adaptive image processing and linear filtering to enhance pattern matching for crystal orientation determination and phase recognition^[21]. However, none of these data preprocessing techniques were specifically designed to amplify the signals of low-intensity diffraction spots.

In this study, we introduce a new algorithm called “Auto-CLAHE” for enhancing diffraction data in PED. The name is short for “automatic contrast-limited adaptive histogram equalization”. Auto-CLAHE is based on a popular image processing technique called CLAHE (contrast-limited adaptive histogram equalization)^[22-24], which applies histogram equalization to small regions of an image independently to prevent over-enhancement of local contrast. The amount of enhancement is controlled by a clip limit, with higher clip limits corresponding to greater enhancement. In our method, the clip limit for each diffraction pattern is automatically assigned based on its average intensity. Our results show that Auto-CLAHE significantly improves the signal-to-noise ratio of low-intensity diffraction spots, particularly those from patterns acquired away from zone axes. This improvement in diffraction pattern quality enables better template matching, resulting in less noise and more accurate orientation mapping.

MATERIALS AND METHODS

An indented pure magnesium (Mg) TEM specimen was used as the model system to test the Auto-CLAHE algorithm. The bulk sample was a hot-rolled commercially pure Mg purchased from MetalMart International Inc. The sample was annealed at 500 °C for two hours under an argon flow atmosphere to homogenize the microstructure and remove most of the dislocations, then mechanically polished to 1,200 grit SiC abrasive sandpaper and chemically polished in a 10% nitric acid in methanol before nanoindentation. Nanoindentation was carried out using a KLA iMicro nanoindentation testing system (KLA Instruments) equipped with a Berkovich tip. Nanoindentation was performed on a grain close to the [0001] direction so the indentation is approximately parallel to the c-axis of the crystal. The indentation depth was 600 nm. The focused ion beam (FIB, Helios G4, ThermoFisher, 30 kV Ga⁺ beam) lift-out method was used to prepare the cross-sectional TEM specimen to investigate the microstructure under the indent impression. The TEM characterizations were performed in an FEI Tecnai TEM operated at 200 keV and equipped with a NanoMEGAS ASTAR system with a Stingray CCD camera for PED experiments. The precession angle was 0.3° and the step size was 20 nm when acquiring the map. The 0.3° precession angle was selected because it offers a good combination of reduction in the dynamical effect and retaining a decent beam spot size (~3 nm). One precession per pixel was used, and the diffraction pattern rate was approximately 0.06 s per pixel.

The Auto-CLAHE method was developed as a Python program, utilizing several scientific libraries to handle data. The source code can be found in our GitHub repository linked under the “Availability of data and materials” section. In the code, the HyperSpy and PixSTEM libraries are used to open and read the PED raw data .blo files. The average intensity of each diffraction pattern is calculated. The diffraction patterns are recorded as 8-bit images. The pixel intensity ranges from 0 (dark) to 255 (bright). Next, each diffraction pattern is processed with CLAHE (from the OpenCV library). The clip limit value is calculated using the equation:

There is an inverse relationship between the clip limit value and the average intensity of the diffraction pattern. If the diffraction pattern is at or close to a zone axis, many diffraction spots are excited. The average intensity of the pattern is high, the clip limit is small, and little enhancement is required. In contrast, if the diffraction pattern is far away from zone axes, a few diffraction spots are excited. The average intensity of the pattern is low, the clip limit is large, and signal enhancement is applied. In the equation, the numerator coefficient of 10 in the Auto-CLAHE algorithm was determined by a combination of trial and error and visual inspection to ensure optimal quality of the enhanced diffraction patterns. It is worth noting that if the average intensity of a diffraction pattern is too high, the “10/Average_intensity” term may round down to 0. Using it alone as the clip limit can cause the CLAHE algorithm to default to non-CLAHE, which will produce erroneous results. To address this issue, another coefficient of 1 was introduced to help preserve image information when processing bright diffraction patterns. With this setup, we have successfully applied this approach to generate enhanced diffraction patterns in various datasets, but users can adjust the coefficients based on their own PED datasets.

The user interface of the algorithm is described as follows. First, the user selects a PED dataset. Next, an analysis window containing a virtual bright-field image of the selected dataset appears. The virtual bright-field image is created by taking the average intensity of the direct beam in the diffraction patterns of each pixel. The user can click on any point on the virtual bright-field (VBF) image to preview the raw and enhanced diffraction pattern at that pixel. Finally, the entire dataset is enhanced with Auto-CLAHE and exported as a new .blo file, which can be further analyzed using the NanoMEGAS commercial software.

RESULTS AND DISCUSSION

Figure 1 shows the cross-sectional VBF image of the indented Mg specimen as the model system. The indent impression is apparent at the top. The region under the indent impression is plastically deformed, and the diffraction contrast is complex. The plastic zone is tilted to the $${[11 \overline{2} 0]}\\$$ zone axis. Since plastic deformation under the indent can change the crystal orientation^[25], the exact orientation will vary from pixel to pixel. Three diffraction patterns were selected within the dataset to represent various diffraction “quality levels”, which decrease going down the right-side column of the figure. Here, we consider diffraction patterns that contain more diffraction spots as good quality as they facilitate more accurate template matching. The first diffraction pattern (marked with a green dot) was taken at the $${[11 \overline{2} 0]}\\$$ zone axis. It has many highly visible diffraction spots, and its average intensity is high. As a result, it does not require enhancement. The second diffraction pattern (marked with a blue dot) is slightly away from the $${[11 \overline{2} 0]}\\$$ zone axis. It can be regarded as a medium-quality pattern, as many of the diffraction spots are obscured. However, some diffraction spots are clearly visible, and the crystal symmetry information is still retained. This type of diffraction pattern requires some enhancement for more diffraction spots to be visible. The third diffraction pattern (marked with a red dot) is regarded as a low-quality pattern. Nearly all diffraction spots aside from the center spot are obscured, and the average intensity is low. The pattern requires a high level of enhancement to properly reveal the diffraction spots. From these observations, it can be deduced that the level of image enhancement should have an inverse relationship with the original average intensity of the diffraction pattern.

Figure 1. A virtual bright-field TEM image of the indented Mg specimen, along with the selected diffraction patterns. The patterns shown on the right generally decrease in “quality” going down the column.

Prior to investigating the effect of CLAHE, we first explored the use of conventional image processing techniques to enhance diffraction information, focusing on brightness, contrast, and gamma adjustment^[26] on selected diffraction patterns from the indented Mg dataset, as depicted in Figure 2. While brightness adjustment uniformly increases the intensity of each pixel and uncovers previously obscured spots, it also raises the brightness of the background and lowers contrast, making it less suitable for analysis. Contrast adjustment also uncovers obscured spots but creates excessive contrast between the spots, resulting in overcorrection of bright spots in the original diffraction patterns, which appear overly saturated and enlarged. Gamma correction performs better than the other techniques by revealing hidden diffraction spots while maintaining a dark background and consistent brightness between spots. However, the low-intensity spots are still a bit dim after the gamma correction, and it is challenging to further enhance them without making the diffraction patterns grainy. Moreover, gamma correction operates uniformly on the entire image, which means that it cannot adapt to local contrast variations^[26]. This can lead to suboptimal results in images with significant variations in contrast across different regions (e.g., bright direct beam spots vs. dim diffracted beam spots in the low “quality” diffraction patterns). Hence, a more advanced enhancement method is needed for more effective data processing.

Figure 2. Selected diffraction patterns processed using brightness, contrast, and gamma adjustments.

The CLAHE algorithm demonstrated remarkable signal enhancement in the diffraction patterns, and the results of applying different clip limits to the selected Mg diffraction patterns are shown in Figure 3. Increasing the clip limit consistently improved the visibility of low-intensity diffraction spots across all patterns. However, as the clip limit increased, the radii of all diffraction spots also increased, including those with originally high intensity, which became even brighter. In comparison to conventional image processing techniques described in the previous paragraph, CLAHE better preserves a dark background and avoids over-saturation in the originally bright spots while allowing weak diffraction spots to be well-illuminated without sacrificing the overall quality of the diffraction pattern.

Figure 3. Selected diffraction patterns processed with CLAHE. Different clip limit (CL) values are applied. For Auto-CLAHE, the algorithm automatically selects the clip limit values. The orientation indexing quality (IQ) of the raw data and the Auto-CLAHE processed data is also shown in the figure.

The Auto-CLAHE algorithm automatically calculates the clip limit for each diffraction pattern, only applying necessary levels of signal enhancement for diffraction pattern indexing. For example, in Pattern 1 in the figure, CLAHE with the calculated clip limit of 2 produces a good result. Greater clip limit values tend to degrade the diffraction pattern quality by augmenting the size of diffraction spots and by leading to the merging of spots. In Pattern 2, a clip limit of 4 was calculated by the algorithm. Lower clip limit values (e.g., 2) may not bring out enough diffraction pattern signal, whereas higher clip limit (e.g., 10) results in excessive spot size augmentation. In Pattern 3, the calculated clip limit of 8 appears to produce a decent result in that many obscured spots are now visible and the crystal symmetry information is revealed. One unique advantage of using Auto-CLAHE to preprocess the PED data is that the signal enhancement is dynamic. For high-quality diffraction patterns, little correction is needed; thus, a lower clip limit is automatically selected. As the diffraction pattern “quality” and overall brightness decrease, a higher clip limit is applied, allowing hidden spots to be effectively revealed.

Next, we quantify the efficacy of Auto-CLAHE in improving the orientation indexing via template matching using the NanoMEGAS commercial software. It is important to note that the quality of template matching and, subsequently, the orientation indexing are affected by background noise and the excitation of the diffraction spots (e.g., intensity) in the experimental data. The challenge of high background noise can be overcome by preparing better quality (e.g., thinner) TEM foils. This leads to more electrons undergoing elastic scattering that forms diffraction spots rather than inelastic scattering that augments the background noise. In addition to matching the high-intensity diffraction spots, the Auto-CLAHE algorithm enhances the signal of low-intensity diffraction spots, which also contributes to the template matching to reach better indexing quality (IQ). The IQ (i.e., template matching quality) of the raw data and the Auto-CLAHE-processed data are shown in Figure 3. Indeed, after the Auto-CLAHE enhancement, all diffraction patterns, regardless of their initial quality, exhibit better IQ as calculated by the NanoMEGAS commercial software. For instance, the IQ of the high-quality diffraction pattern (Pattern 1) improves from 360 to 495, median-quality diffraction pattern (Pattern 2) from 248 to 460, and low-quality (Pattern 3) from 66 to 265.

We further inspected the local intensity information in the raw data and the Auto-CLAHE-processed data to evaluate the impact of signal enhancement on forbidden reflections and noise in diffraction patterns. Figure 4 depicts the selected diffraction patterns before and after Auto-CLAHE processing, together with the intensity profile plotted along the [0002] direction, indicated by the dashed boxes. Mg has the hexagonal closed-packed (HCP) crystal structure; hence, the even-numbered reflections (e.g., 0002, 0004, etc.) are allowed, and the odd-numbered reflections (e.g., 0001, 0003, etc.) are forbidden. However, due to the TEM foil thickness and the limited 0.3° precession angle, some dynamical effects persist, making the forbidden reflections visible but with reduced intensity (grey curves in the intensity profiles for Patterns 1 and 3). Even after the Auto-CLAHE signal enhancement, the lower intensity of the forbidden reflection is retained (maroon curves in the intensity profiles for Patterns 1 and 3). For Pattern 2, due to the specific crystal orientation, the $$000 \overline{1}\\$$ reflection has a higher intensity than the 0002 reflection in the raw data. This information is preserved in the Auto-CLAHE-processed diffraction pattern. The above observations indicate that although the CLAHE algorithm amplifies the weak signals more than the strong ones, it does not arbitrarily amplify weak spots to the detriment of crucial structural information, particularly regarding the relative diffraction spot intensities influenced by structure factors. One should bear in mind that with the signal enhancement, the background level in the vicinity of the diffraction spots can increase. The background is largely attributed to the overlapping of Gaussian profile tails of the diffraction spots, and its level can be assessed by the intensity at the troughs between the peaks. In all cases, the background levels are higher in the Auto-CLAHE processed diffraction patterns than in raw data. However, the adverse effect of the increased background level in diffraction patterns seems to be outweighed by the benefits gained from signal enhancement, which is evidenced by the increase in IQ after Auto-CLAHE processing, as shown in Figure 3. It is also interesting to observe the preservation of very low noise/background levels from the regions in the raw data to the auto-CLAHE-enhanced data, indicated by the black arrows in the intensity profiles in Patterns 2 and 3 [Figure 4]. This provides assurance that the CLAHE algorithm does not introduce noise arbitrarily into areas of low noise in diffraction patterns.

Figure 4. Selected diffraction patterns before and after the Auto-CLAHE processing, together with the diffraction pattern intensity profile along the [0002] direction. The dashed boxes indicate where the intensity profiles are generated. The red arrow in the first diffraction pattern indicates the intensity signal plotting direction.

To test the overall effectiveness of the Auto-CLAHE algorithm, crystal orientation maps were constructed using both raw and enhanced datasets. Figure 5A shows the raw data-constructed orientation maps along the X, Y, and Z directions overlayed with the index maps. The orientations map is created by comparing the experimentally acquired diffraction patterns with the simulated diffraction patterns via template matching. The index map is created by comparing the experimentally acquired diffraction patterns with the simulated ones, with better fits corresponding to brighter pixels. It is very similar to the band contrast map in EBSD. The quality of the orientation maps constructed from the raw data is acceptable. The maps reveal the overall deformed matrix and deformation twinning in the top right corner. (The very top layer is the Pt protective coating, and its indexing information should be discarded.) The deformed microstructure is expected as the plastic deformation of c-axis compressed Mg is predominantly accommodated by <c+a> dislocations^[27,28]. The complex stress state from nanoindentation also activated some deformation twinning^[29,30]. However, it is also apparent that regions of noise are present in both the matrix and the twin, as indicated by the white arrows. This noise is caused by a lack of clarity in the diffraction spots of the raw data. With the obscured diffraction spots, the commercial orientation indexing software is unable to accurately match the experimental data with the simulated ones, resulting in erroneous orientation assignments in several diffraction patterns. Figure 5B shows the Auto-CLAHE-enhanced orientation maps overlayed with the index maps. After preprocessing the dataset using Auto-CLAHE, a vast majority of the noise in the orientation maps is removed, as distinct diffraction spots are revealed in the diffraction patterns, allowing for accurate orientation indexing. Additionally, it is worth pointing out that there is a region in the raw data that appears to be a twin grain, as indicated by the black arrows in Figure 5A. However, in the raw data, it is difficult to distinguish if it is actually present or merely an extension of the noise. In contrast, the filtered data clearly reveals a twin grain in the sample, reducing the uncertainty of the feature identification in the dataset. Based on these results, Auto-CLAHE dramatically improves orientation mapping on the PED raw datasets.

Figure 5. Crystal orientation maps overlapped with index maps with (A) raw data and (B) data processed using Auto-CLAHE.

It is worth noting that determination of the clip limit using the inverse relationship in the equation $$\mathit{{{ Clip\quad limit }} }=\left(\frac{10}{\text { Average_intensity }}\right)+1\\$$is based on empirical considerations. Here, we explore the effect of changing the numerator coefficient on the qualities of diffraction patterns and orientation maps. Figure 6A depicts the selected diffraction patterns and the corresponding orientation map from the raw data. Noise is apparent in both the matrix and the twin, as previously described. In contrast, all Auto-CLAHE-processed data demonstrate much reduced noise in the respective orientation maps compared to the raw data. When choosing 1 as the numerator coefficient, some visible enhancements of the diffraction pattern were observed, and most noise in the orientation map was removed [Figure 6B]. However, low noise level persists in the matrix and the twin (some indicated by the white dashed circles). Increasing the numerator coefficient to 10 (the one we adopted in our algorithm) further enhances the diffraction pattern signals and leads to a near-noise-free orientation map [Figure 6C]. Further increasing the numerator coefficient to 100 results in over-saturation of the diffraction spots and a noisier orientation map [Figure 6D]. The most notable noise is the misidentification of the twinned region as the matrix in the top right part of the map; some noise also appears in the matrix (indicated by the white circles). This is caused by the poorer indexing from the diffraction spot supersaturation, diffraction spots merging, and arbitrarily augmented background. Taken together, these observations are encouraging, suggesting that the Auto-CLAHE algorithm is relatively robust - improved orientation maps can be generated with a wide range selection of the numerator coefficients. However, carefully selecting the numerator coefficient is critical to creating orientation maps with high confidence and low noise. Moreover, we encourage the users to try different numerator coefficients or even different inverse relationships (e.g., $$\mathit{{{ Clip\quad limit }} }=e^{-\alpha(\text { Average_intensity) }}\\$$) based on the diffraction quality to achieve optimal indexing results.

Figure 6. Selected diffraction patterns and reconstructed x-direction orientation maps using the (A) raw data and Auto-CLAHE algorithm-processed data with (B) 1, (C) 10, and (D) 100 numerator coefficients.

Furthermore, we point out that the NanoMEGAS ASTAR commercial software offers the flexibility of adjusting brightness, contrast, and gamma settings after data acquisition before data reconstruction. However, an equal degree of enhancement (determined by the user) is uniformly applied to all acquired diffraction patterns. For instance, a 50% increase in brightness can effectively enhance faint diffraction spots in certain pixels. Yet, this same enhancement level might cause oversaturation and compromise indexing accuracy in pixels containing intense diffraction patterns. In contrast, the auto-CLAHE algorithm employs a dynamic approach to signal enhancement. It selectively applies signal amplification to pixels that contain diffraction patterns with low intensity. This ensures that pixel-specific enhancements are tailored to the actual signal characteristics.

CONCLUSIONS

TEM-based crystal orientation mapping at the nanoscale using PED is a useful tool that can reveal microstructural information that has been inaccessible to SEM-based EBSD. In this work, we present a new algorithm named Auto-CLAHE, which automatically amplifies signals of low-intensity diffraction patterns. The degree of enhancement is tailored dynamically based on the overall intensity of the pattern. The algorithm applies greater enhancement to patterns with fewer spots located away from zone axes, while little or no enhancement is used for patterns with many spots located at a zone axis. By enhancing the visibility of low-intensity diffraction spots, Auto-CLAHE remarkably improves template matching between experimentally obtained and simulated diffraction patterns, leading to orientation maps with considerably higher quality and lower noise. Our findings suggest that Auto-CLAHE offers a convenient and efficient solution for preprocessing PED data. The scientific significance of this work is two-fold. First, our method improves the diffraction signal, enabling nanoscale orientation mapping in previously challenging systems with pixels that contain low-intensity diffraction spots. Second, considering that many microscopists are not trained in digital image processing, our technique serves as an example of successful cross-disciplinary implementation, demonstrating how knowledge from other research areas can advance electron microscopy characterization. We anticipate the Auto-CLAHE approach can be routinely applied to all PED datasets and potentially be extended to enhance the diffraction signal in 4D-STEM datasets^[31] to achieve further improved analyses.