ACE-v2: Learning Anatomically Consistent Embeddings via Patch Matching with Composition and Decomposition
We present ACE-v2, a novel SSL framework that learns deep anatomical representations from unlabeled chest radiographs by explicitly modeling three fundamental properties of human anatomy: uniqueness, consistency, and composition/decomposition. To this end, ACE-v2 introduces (1) multi-scale, center-aligned crops to capture structure-specific embeddings, (2) a grid-based cropping scheme with controlled overlap to establish fine-grained anatomical consistency, and (3) a composition-aware strategy to align whole-part relationships across views. These objectives are integrated via a cyclic pretraining strategy based on student-teacher architecture to progressively accumulate anatomical knowledge.
⭐
| Model name | Backbone | Pretrained dataset | Input Resolution | model |
|---|---|---|---|---|
| ACE-v2SwinV1-B | SwinV1-base | ChestX-ray14 | 448x448 | Dropbox | BaiduDisk |
| ACE-v2SwinV2-B | SwinV2-base | ChestX-ray14 | 512x512 | Dropbox | BaiduDisk |
| ACE-v2 | SwinV2-base | ChestX-ray1M | 512x512 | Dropbox | BaiduDisk |
- ChestX-ray14
- RSNA Pneumonia
- Shenzhen
- JSRT
- SIIM
- COVID-QU-Ex
- ChestLandmarks
- ChestX-ray1M: consisting of 1.04M chest X-ray images from 17 datasets, detailed as follow:
| Index | Pretraining Datasets-Xray | Numbers (train split) |
|---|---|---|
| 1 | MIMIC-CXR | 377028 |
| 2 | CheXpert | 223414 |
| 3 | PadChest | 160828 |
| 4 | ChestX-ray14 | 86524 |
| 5 | JSRT | 247 |
| 6 | Mendeley-V2 | 5232 |
| 7 | Montgomery | 138 |
| 8 | Shenzhen | 662 |
| 9 | RSNA Pneumonia | 26684 |
| 10 | Indiana ChestX-ray (Open-I) | 7883 |
| 11 | COVIDx | 1223 |
| 12 | COVID-19 RADIOGRAPHY_DATABASE | 21165 |
| 13 | VinDr-CXR | 15000 |
| 14 | RSNA International COVID-19 Open Radiology Database (RICORD) | 1005 |
| 15 | PLCO | 89716 |
| 16 | NIH TB | 12769 |
| 17 | SIIM | 10675 |
| Total | 1040193 |
- python >= 3.9
- Pytorch (pytorch.org)
Create a new environment and activate it:
$ conda create -n ACE python==3.9
$ conda activate ACE
Install Pytorch according to the CUDA version (e.g., CUDA 12.6):
$ pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126
Clone the repository:
$ git clone https://github.com/jlianglab/ACE.git
$ cd ACE
$ cd ACE_v2
$ pip install -r requirements
# Train ACE-v2 model based on DDP
CUDA_VISIBLE_DEVICES="0,1,2,3" python -m torch.distributed.launch --nproc_per_node=4 --master_port 28301 main.py --arch swin_base --batch_size_per_gpu 16 --image_size 448 --data_path your_pretrain_image_path --output_dir your_saving_path
- Load the model
Load the SwinV1-base code in swin_transformer.py:
from models.swin_transformer import SwinTransformer
model = SwinTransformer(img_size= 448, patch_size=4, window_size=7, embed_dim=128, depths=(2, 2, 18, 2), num_heads=(4, 8, 16, 32), num_classes=3, use_dense_prediction=True)
Load the SwinV2-base code in swin_transformer_v2.py:
from models.swin_transformer_v2 import SwinTransformerV2
model = SwinTransformerV2(img_size= 512, patch_size=4, window_size=16, embed_dim=128, depths=(2, 2, 18, 2), num_heads=(4, 8, 16, 32), num_classes=3, use_dense_prediction=True)
- Load the pretrained weights
checkpoint = torch.load(model_path, map_location='cpu')
checkpoint = checkpoint['teacher']
checkpoint = {k.replace("backbone.", ""): v for k, v in checkpoint.items()}
msg = model.load_state_dict(checkpoint, strict=False)
print(msg)
- Finetune the pretrained model on target tasks
Refer the finetuning codes in Finetune.
- Evaluate the learned and emergent properties of the pretrained model
Refer the evaluation codes in Property.
If you use this code or use our pre-trained weights for your research, please cite our paper:
This research has been supported in part by ASU and Mayo Clinic through a Seed Grant and an Innovation Grant, and in part by the NIH under Award Number R01HL128785. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. This work has utilized the GPUs provided in part by the ASU Research Computing and in part by Sol and Bridges-2 at Pittsburgh Supercomputing Center through allocation BCS190015 and the Anvil at Purdue University through allocation MED220025 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296. The content of this paper is covered by patents pending.
Released under the ASU GitHub Project License.

