A new AI model generates realistic synthetic microscope images of atoms, providing scientists with reliable training data to accelerate materials research and atomic scale analysis.
(Nanowerk Spotlight) The structure of matter, down to the level of atoms, determines how technologies function. The arrangement of atoms governs how a battery stores energy, how a catalyst speeds up a chemical reaction, and how a semiconductor carries current. Scientists can examine these tiny building blocks directly with scanning transmission electron microscopy (STEM). This technique sends a focused electron beam through a thin slice of material and records how atoms scatter it. The resulting images can reveal columns of atoms with extraordinary detail. Yet those images are never perfect. They are filled with noise from the detection process, distortions introduced by the instrument, and contamination that sticks to samples. Extracting clear information from them remains difficult and slow.
Machine learning has been proposed as a way to accelerate this work. Neural networks can process images at scale and spot features too subtle for human experts. But training such models requires large sets of examples with known answers. In microscopy, those answers are atomic positions and elemental identities. Obtaining them is expensive and often subjective. Different analysts can disagree when interpreting the same image, and manual annotation is time consuming. This shortage of labeled data has held back the automation of atomic-scale analysis.
Researchers have tried to use simulations to fill the gap. Physics-based models of electron scattering can create synthetic STEM images directly from crystal structures. These simulations are abundant and perfectly labeled, since every atomic position is already known. The problem is that simulated images do not match the statistics of experiments. They are too clean. They lack the random electron counting noise, the subtle geometric distortions, and the contamination patterns found in real data. Attempts to add noise by hand capture only part of this complexity. Models trained on such images often fail once they are applied to experimental data.
Generative models seemed to offer a better option. Adversarial networks, called GANs, were able to create sharp synthetic images, but they were unstable to train and sometimes ignored rare but important features. Diffusion models, which generate data by reversing a gradual noise process, proved more stable and diverse. Yet standard diffusion approaches have their own flaw. They tend to smooth out high-frequency detail, favoring broad patterns over the fine structure where STEM signals live. This limitation has kept synthetic STEM images from reaching the level of realism needed for reliable training.
A study published in Advanced Science (“STEMDiff: A Wavelet‐Enhanced Diffusion Model for Physics‐Informed STEM Image Generation”) introduces STEMDiff, a wavelet-enhanced diffusion model for physics-informed STEM image generation. STEMDiff integrates physics-based knowledge of microscopy with a generative approach that preserves high-frequency detail. The goal is to produce synthetic images that look and behave like experiments, so they can be used to train analysis tools without labor-intensive labels.
Schematic of the major components in STEMDiff. a) The training process of the proposed “Label to STEM” conditional diffusion model. At step n, the modified U-Net neural network takes a simulated STEM image Istem and a noisy image Jn as inputs. Each simulated image is paired with a binary label map derived by projecting the atomic coordinates of the input crystal structure onto a 2D grid, where atomic column regions are labeled as 1 and background regions as 0. b) The sampling process of the proposed “Label to STEM” conditional diffusion model, c) The network architecture incorporates a customized skip connection mechanism employing the Noise-Retaining Block (NRB) to ensure experimental noise signatures remain intact throughout the STEM image generation pipeline. (Image: Reprinted from DOI:10.1002/advs.202508266, CC BY) (click on image to enlarge)
The foundation is a conditional diffusion model. During training, the system receives simulated STEM images with noise added and learns to denoise them step by step. Once trained, it can start from random noise and a structural label map to generate new images. The label map is a grid with ones marking atomic column positions and zeros elsewhere, derived directly from the crystal structure. Conditioning on this map ensures that the arrangement of atoms in the output is correct. The diffusion process then fills in the realistic variation and noise.
The central innovation is how the model preserves fine detail. The researchers introduced Noise Retaining Blocks into the network. These blocks use a wavelet transform to break an image into components separated by frequency and orientation. The high-frequency diagonal band, where fine atomic features and subtle artifacts appear, is preserved explicitly. The block then applies an inverse transform and reinserts the components into later layers of the network. This way, high-frequency information is not lost as the model gradually denoises the image.
Training still begins with simulated data, but the authors added steps to bring the simulations closer to reality. They injected Poisson noise, which mirrors the statistics of electron counting. They added Gaussian noise to mimic environmental fluctuations. They simulated distortions such as shear, compression, and jitter. They also reproduced contamination by extracting low-frequency backgrounds from real STEM experiments with Fourier filtering and folding them into the simulated frames. The result is a dataset that combines perfect structural labels with experimental-like imperfections.
The researchers tested their model on monolayer tungsten diselenide. They compared raw simulations, their generated images, and experimental images. Standard simulations missed the high-frequency components that experiments display in Fourier space. The generated images, by contrast, reproduced those features more faithfully. The sampling process was also fast. The model could generate 256 by 256 pixel images in milliseconds, compared to seconds for common diffusion samplers. This speed matters when building large datasets.
Quantitative evaluation backed up these results. The team measured Fréchet Inception Distance, or FID, which compares statistical features of real and synthetic images. Lower scores mean closer match. Their method achieved 0.02, while a previous adversarial approach scored 0.35. They also calculated Kullback-Leibler divergence, which measures the difference between probability distributions, for pixel intensities. Their synthetic images aligned more closely with experimental data than either unmodified simulations or noise-augmented versions.
The authors also analyzed why their model performs better. In standard diffusion, high-frequency energy decreases as the denoising process advances, leaving images smoother than experiments. In STEMDiff, high-frequency power is retained throughout the process. Comparing spectra showed that their outputs matched experiments more closely across a range of frequencies. Pixel intensity histograms confirmed that means and variances were closer to experiment.
To test whether the method was useful in practice, the team trained a fully convolutional neural network to detect atomic columns. They used only images generated by STEMDiff. The training set consisted of label maps and corresponding synthetic images, with 185 examples retained after quality checks. Despite the modest size, the detector identified atomic columns in experimental images of tungsten diselenide and graphene. It also distinguished elements in those cases and worked under low signal-to-noise conditions. That outcome suggests the synthetic images are close enough to experiments to transfer directly.
For users, the workflow is straightforward. Provide a structure file, generate a label map of atomic columns, and let the model create synthetic STEM images from it. Repeating the process yields a dataset with known ground truth positions and experimental-like variation. Such datasets can then train analysis models that would otherwise depend on slow and subjective manual annotation. Because the generated images reproduce both the atomic arrangement and the statistical character of real experiments, the risk of models overfitting to simulation artifacts is reduced.
The authors suggest that their approach could extend beyond STEM. Any imaging method where fine detail is critical might benefit, including X-ray tomography or optical microscopy. Improved atomic-scale analysis could also accelerate progress in areas such as energy storage, catalysis, and water treatment, where connecting structure to performance is vital. While these applications are prospective, the study shows that synthetic data can be made statistically faithful to experiments if the generative model is designed with physics and frequency content in mind.
Some limits remain. Training relies on simulated data, which must already capture the physics of the instrument and sample. Extending the model to new materials or conditions will require fresh simulations. The experiments in this study focused on two-dimensional systems, which are important but not exhaustive. Even with these caveats, the evidence points to a method that narrows the gap between simulation and experiment in a measurable way.
By embedding physical knowledge into a diffusion model and preserving high-frequency detail, the team has demonstrated synthetic STEM images that are both realistic and useful. Their approach offers a practical way to build datasets at scale, making atomic-scale analysis more efficient and reliable.
For authors and communications departmentsclick to open
Lay summary
Prefilled posts
ORCID information
Houyang Chen (University of Chinese Academy of Sciences)
, 0000-0002-3319-5743 corresponding author
Nanowerk Newsletter
Get our Nanotechnology Spotlight updates to your inbox!
Thank you!
You have successfully joined our subscriber list.
Become a Spotlight guest author! Join our large and growing group of guest contributors. Have you just published a scientific paper or have other exciting developments to share with the nanotechnology community? Here is how to publish on nanowerk.com.
