電子斷層掃描因其高分辨率在納米材料三維表征方面受到青睞，但受限于“缺失楔形效應(yīng)”，導(dǎo)致重建圖像出現(xiàn)畸變。目前的算法，特別是機(jī)器學(xué)習(xí)中的神經(jīng)網(wǎng)絡(luò)技術(shù)，雖在糾正這些畸變方面取得了進(jìn)展，但因訓(xùn)練數(shù)據(jù)與實(shí)際情況存在差異，仍面臨著準(zhǔn)確性的挑戰(zhàn)。

由美國(guó)伊利諾伊大學(xué)香檳分校材料科學(xué)與工程系的Qian Chen教授領(lǐng)導(dǎo)的團(tuán)隊(duì)提出了UsiNet，這是一個(gè)無(wú)監(jiān)督投影圖修復(fù)方法，旨在解決電子斷層掃描中常見(jiàn)的缺失楔形效應(yīng)問(wèn)題。

UsiNet的無(wú)監(jiān)督訓(xùn)練機(jī)制免除了對(duì)基準(zhǔn)真值、人工標(biāo)注或傾斜圖像模擬的依賴，顯著提升了其在真實(shí)電子斷層掃描數(shù)據(jù)集處理中的實(shí)用性，特別是在無(wú)法獲取全角度傾斜序列的場(chǎng)合。

該方法在訓(xùn)練時(shí)只需極少量的數(shù)據(jù)集（甚至低至20個(gè)納米顆粒）和較小的傾斜范圍（±45°），使其在處理那些對(duì)束流敏感的聚合物和生物材料時(shí)變得尤為寶貴，因?yàn)檫@些材料的傾斜范圍可能由于束流累積傷害而受到限制。

UsiNet對(duì)窄傾斜范圍的容忍性在原位電子斷層掃描的研究中至關(guān)重要，比如在研究電化學(xué)循環(huán)、催化作用或腐蝕過(guò)程中納米顆粒形態(tài)變化時(shí)，由于需要保持時(shí)間分辨率，僅能收集有限的傾斜序列。

Fig. 5 Orientation-dependent missing wedge artifact and comparison between different reconstruction algorithms.

此外，UsiNet無(wú)需進(jìn)行樣品平均化處理，因此可廣泛適用于諸如用于可充電離子電池的電極納米顆粒、催化劑納米顆粒和納米塑料等多種異質(zhì)納米顆粒系統(tǒng)。在這些系統(tǒng)中，缺失楔形效應(yīng)尤其棘手，因?yàn)樗鼤?huì)引起明顯的各向異性失真。盡管此次展示主要聚焦于膠體納米顆粒，但UsiNet無(wú)監(jiān)督修復(fù)的基本原理也同樣適用于其他包含3D納米尺度形態(tài)細(xì)節(jié)的樣本，如合金的微觀結(jié)構(gòu)域和聚酰胺分離膜的皺折。UsiNet的出現(xiàn)極大地?cái)U(kuò)展了電子斷層掃描技術(shù)在解析材料的形態(tài)、合成及性能之間關(guān)聯(lián)性方面的潛力。

Fig. 6 Comparison of 3D reconstructions of experimentally synthesized NPs with and without inpainting.

UsiNet的應(yīng)用前景極為廣闊，它不僅能夠揭示電池或催化納米材料的退化機(jī)理，還能幫助理解自然形成的納米塑料的形態(tài)與聚集行為，并能優(yōu)化不同組成的納米顆粒的合成流程。該文近期發(fā)表于npj Computational Materials?10:?28 (2024).

Fig. 7 Visualizing the heterogeneity of experimentally synthesized NPs.

Editorial Summary

Electron tomography is favored for its high-resolution in 3D characterization of nanomaterials but is limited by the “missing wedge effect”, leading to distortions in the reconstructed images. Modern algorithms, especially neural network technologies in machine learning, have made advances in correcting these distortions, yet they still face challenges in accuracy due to differences between training data and actual conditions.?

A team led by Prof. Qian Chen from Department of Materials Science and Engineering, University of Illinois at Urbana-Champaign, USA, purposed UsiNet, an unsupervised sinogram inpainting method to correct the missing wedge effect in electron tomography. The unsupervised training in UsiNet does not require ground truth, manual annotation, or tilt image simulation, and thus is practically applicable to real electron tomography datasets where full angle tilt series are not obtainable. The authors demonstrate that UsiNet works with a small number of training dataset (down to 20 NPs) and narrow tilt range (±45°), which can be immediately useful for beam sensitive polymeric and biological materials where the tilt range can be limited by accumulated beam damage. The tolerance with a narrow tilt range could be critical for studies involving in-situ electron tomography—for example, on the evolution of the 3D shapes of NPs during chemical reactions such as electrochemical cycling, catalysis, and corrosion—where only scarce tilt series can be collected to ensure temporal resolution. Moreover, UsiNet does not require sample averaging and can thus apply to a broad range of heterogeneous NP systems such as electrode NPs used in rechargeable ion batteries, catalytical NPs, and nanoplastics. The missing wedge effect is otherwise particularly problematic for heterogeneous systems by generating anisotropic distortion. Although the demonstration focuses on colloidal NPs, the principle of unsupervised inpainting is expected to work for other samples containing 3D nanoscale morphology details, such as microstructural domains in alloys and crumples in polyamide separation membranes. UsiNet brings the full potential of electron tomography in charting the relationships of morphology with synthesis and performance of materials. A wide scope of applications can be enabled by UsiNet, such as uncovering degradation mechanisms of battery or catalytical nanomaterials, understanding morphologies and aggregation behaviors of naturally formed nanoplastics, and optimizing synthetic protocols of NPs with varying compositions. This article was recently published in npj Computational Materials 10: 28 (2024).

原文Abstract及其翻譯

No ground truth needed: unsupervised sinogram inpainting for nanoparticle electron tomography (UsiNet) to correct missing wedges?(無(wú)需基準(zhǔn)真值：用于校正缺失楔形區(qū)的無(wú)監(jiān)督納米顆粒電子斷層掃描圖像重建的投影圖修復(fù))

Lehan Yao,?Zhiheng Lyu,?Jiahui Li?&?Qian Chen?

Abstract?Complex natural and synthetic materials, such as subcellular organelles, device architectures in integrated circuits, and alloys with microstructural domains, require characterization methods that can investigate the morphology and physical properties of these materials in three dimensions (3D). Electron tomography has unparalleled (sub-)nm resolution in imaging 3D morphology of a material, critical for charting a relationship among synthesis, morphology, and performance. However, electron tomography has long suffered from an experimentally unavoidable missing wedge effect, which leads to undesirable and sometimes extensive distortion in the final reconstruction. Here we develop and demonstrate Unsupervised Sinogram Inpainting for Nanoparticle Electron Tomography (UsiNet) to correct missing wedges. UsiNet is the first sinogram inpainting method that can be realistically used for experimental electron tomography by circumventing the need for ground truth. We quantify its high performance using simulated electron tomography of nanoparticles (NPs). We then apply UsiNet to experimental tomographs, where >100 decahedral NPs and vastly different byproduct NPs are simultaneously reconstructed without missing wedge distortion. The reconstructed NPs are sorted based on their 3D shapes to understand the growth mechanism. Our work presents UsiNet as a potent tool to advance electron tomography, especially for heterogeneous samples and tomography datasets with large missing wedges, e.g. collected for beam sensitive materials or during temporally-resolved in-situ imaging.

摘要 復(fù)雜的自然和合成材料，如細(xì)胞亞結(jié)構(gòu)器官、集成電路中的器件架構(gòu)，以及具有微觀結(jié)構(gòu)領(lǐng)域的合金，需要能夠在三維（3D）中研究這些材料的形態(tài)和物理性質(zhì)的表征方法。電子斷層掃描在成像材料的3D形態(tài)方面具有無(wú)與倫比的（亞）納米分辨率，這對(duì)于描繪合成、形態(tài)和性能之間的關(guān)系至關(guān)重要。然而，電子斷層掃描長(zhǎng)期以來(lái)一直受到實(shí)驗(yàn)上不可避免的缺失楔形效應(yīng)的困擾，這導(dǎo)致最終重建中出現(xiàn)了不希望的、有時(shí)甚至是大量的失真。在這里，我們開(kāi)發(fā)并展示了用于納米顆粒電子斷層掃描的無(wú)監(jiān)督投影圖修復(fù)（UsiNet）來(lái)校正缺失的楔形區(qū)。UsiNet是第一個(gè)可以在現(xiàn)實(shí)實(shí)驗(yàn)電子斷層掃描中使用的投影圖修復(fù)方法，它避開(kāi)了對(duì)基準(zhǔn)真值的需求。我們使用模擬的納米顆粒電子斷層掃描來(lái)量化其高性能。然后我們將UsiNet應(yīng)用于實(shí)驗(yàn)層析圖，其中同時(shí)重建了100多個(gè)十面體納米顆粒和極為不同的副產(chǎn)物納米顆粒，且沒(méi)有缺失楔形失真。根據(jù)它們的3D形態(tài)對(duì)重建的納米顆粒進(jìn)行分類，以理解生長(zhǎng)機(jī)制。我們的工作將UsiNet作為推進(jìn)電子斷層掃描的強(qiáng)大工具，特別適用于異質(zhì)樣品和具有大量缺失楔形區(qū)的斷層數(shù)據(jù)集，例如為了敏感材料而收集的數(shù)據(jù)或在時(shí)間解析的原位成像過(guò)程中收集的數(shù)據(jù)。