AI4Seg

This repository provides the core implementation for our manuscript:

"A Physics-Informed Synthetic-to-Experimental Framework for Few-Shot Structural Segmentation of HRTEM Images"

The repository is organized around the synthetic-to-experimental workflow described in the paper, including physics-informed synthetic data generation, experimental data preparation, segmentation model training and evaluation, and foundation model fine-tuning experiments.

Overview

This repository provides the core implementation of a physics-informed synthetic-to-experimental framework for few-shot structural segmentation of HRTEM images.

It contains three main modules: Simulate_data_engine/ for DCRI-based synthetic data generation, Syn_to_Exp/ for synthetic-to-experimental segmentation training and evaluation, and Foundation_models_finetuning/ for foundation model fine-tuning and evaluation.

Repository Structure

.
├── Simulate_data_engine/
│   └── DCRI-based synthetic HRTEM data generation, including structural
│       construction, atomic model preparation, image simulation, and mask generation.
│
├── Syn_to_Exp/
│   └── Synthetic-to-experimental segmentation workflow, including data preparation,
│       model training, fine-tuning, prediction, and evaluation.
│
├── Foundation_models_finetuning/
│   └── Fine-tuning and evaluation of foundation segmentation models using
│       DCRI-generated synthetic HRTEM data.
│
└── README.md

DCRI Framework

The core implementation of the Domain Construction-Relaxation-Imaging (DCRI) framework is provided in:

Simulate_data_engine/

This directory contains scripts for:

• constructing synthetic structural domains;
• generating atomic models with crystalline, grain-boundary, amorphous, and background regions;
• preparing atomic structures for HRTEM simulation;
• performing multislice HRTEM image simulation;
• generating synthetic images and corresponding pixel-level segmentation masks.

Detailed script descriptions and usage examples are provided in the README file inside this directory.

Synthetic-to-Experimental Workflow

The core implementation of the synthetic-to-experimental segmentation workflow is provided in:

Syn_to_Exp/

This directory contains scripts and utilities for:

• preparing experimental HRTEM datasets and pixel-level annotations;
• training segmentation models with experimental-only and synthetic-to-experimental strategies;
• fine-tuning models using DCRI-generated synthetic data and limited experimental labels;
• performing prediction and evaluating segmentation performance on experimental HRTEM images;
• running few-shot segmentation experiments across different model architectures.

Detailed script descriptions and usage examples are provided in the README file inside this directory.

Foundation Model Fine-tuning

The implementation of foundation model fine-tuning experiments is provided in:

Foundation_models_finetuning/

This directory contains scripts for:

• preparing DCRI-generated synthetic datasets for foundation model fine-tuning;
• converting semantic segmentation masks into instance-level annotations for model training;
• fine-tuning foundation segmentation models with synthetic HRTEM data;
• preparing real HRTEM test cases with instance-level ground truth and prompts;
• evaluating fine-tuned models using interactive segmentation and automatic instance segmentation modes.

Detailed script descriptions and usage examples are provided in the README file inside this directory.

Data and Model Availability

The datasets and fine-tuned model checkpoints generated in this study will be made publicly available upon acceptance of the manuscript.

For additional data access requests during the review process, please contact the corresponding author.

Usage

The repository is organized into three connected modules. Users should first refer to the README files inside the main subdirectories:

Simulate_data_engine/
Syn_to_Exp/
Foundation_models_finetuning/

A typical workflow consists of:

1. generating DCRI synthetic HRTEM images and corresponding segmentation masks;
2. preparing experimental HRTEM images and pixel-level labels;
3. training and fine-tuning segmentation models using experimental-only or synthetic-to-experimental strategies;
4. evaluating trained models on held-out experimental test images;
5. fine-tuning and evaluating foundation segmentation models using DCRI-generated synthetic data, when needed.

More detailed commands, configuration examples, and script-level explanations are provided in the corresponding subdirectory README files.

Notes

This repository provides the core implementation associated with the manuscript. Datasets, fine-tuned checkpoints, and additional processed results will be linked after manuscript acceptance.

Citation

If you use this code, dataset, or workflow, please cite our paper:

[to be added upon acceptance]

License

[to be added upon acceptance]