PhonoP2C 是一个结合中文上文进行拼音转汉字的模型研究项目，提供数据预处理、模型训练、推理演示和部署导出工具。输入是中文上文和拼音音节，输出是汉字。
PhonoP2C is a research project on converting pinyin into Chinese characters using preceding Chinese text as context. It provides data preprocessing, model training, inference demos, and deployment export tools. It takes Chinese context and pinyin syllables as input and produces Chinese characters.
训练后的模型可导出为 phono-core 使用的模型包；Euphony 则提供基于该引擎的 Windows 输入法。使用现成模型进行应用集成，可从 phono-core 的 SDK 和演示程序开始；本仓库用于训练、实验与导出。
Trained models can be exported as packages for phono-core, the inference engine used by the Euphony Windows input method. For application integration with existing models, start with phono-core's SDK and demos. This repository supports training, experiments, and export.
当前 Pixi 环境面向 Linux x86_64，使用 Python 3.13、CUDA 13 和 cu130 版 PyTorch。在仓库根目录安装环境，后续命令通过 pixi run 执行。完整依赖见 pixi.toml。
The Pixi environment targets Linux x86_64 with Python 3.13, CUDA 13, and the cu130 build of PyTorch. Install it from the repository root and run subsequent commands through pixi run. See pixi.toml for dependencies.
pixi install按 数据集配置设置语料来源和输出目录；默认工作流使用 datasets/pretrain_base 下的语料，生成 datasets/pretrain_v2 下的 MDS 训练集与 Arrow 验证集。大语料可先用 tools/subset.py 抽取子集，使用前需设置该脚本的输入、输出路径。详细步骤见 中文 / English。
Set corpus sources and output paths in the dataset configuration. The default workflow reads corpora under datasets/pretrain_base and produces MDS training data and Arrow validation data under datasets/pretrain_v2. For large corpora, optionally configure the input and output paths in tools/subset.py and use it to extract a subset. See the preprocessing guide for details.
pixi run python main.py task=preprocess检查 训练配置和 设备配置，再启动训练。检查点包含 pre_model/ 和 post_model/ 两个子目录。模型参数、训练选项与评估指标见 中文 / English。
Review the training configuration and device configuration, then start training. Checkpoints contain pre_model/ and post_model/ subdirectories. See the training guide for model parameters, training options, and evaluation metrics.
pixi run python main.py task=train将 <checkpoint-dir> 替换为包含两个子模型的检查点目录。--text 提供中文上文，--pinyin 接收以空格分隔的拼音音节。
Replace <checkpoint-dir> with a checkpoint directory containing both sub-models. --text supplies Chinese context, and --pinyin takes space-separated pinyin syllables.
pixi run python demo.py --checkpoint <checkpoint-dir> --text "今天" --pinyin tian qi hen hao在 导出配置中设置检查点、输出目录和量化方式，再导出 ExecuTorch 模型。随后使用 tools/build_pack_v2_2.py，结合模型配置、词表及所需的分词资源组装 phono-core v2.2 模型包。完整打包参数与示例见 中文 / English。
Set the checkpoint, output directory, and quantization mode in the export configuration, then export the ExecuTorch models. Use tools/build_pack_v2_2.py with the model configuration, vocabularies, and required segmentation resources to assemble a phono-core v2.2 package. See the export guide for packaging arguments and examples.
pixi run python main.py task=export
pixi run python tools/build_pack_v2_2.py --help模型采用编码器—解码器架构：拼音编码器表示待转换的音节序列，中文解码器结合上文生成汉字。训练同时学习中文历史建模和拼音条件下的汉字预测。两部分在代码中分别命名为 post 和 pre；两遍前向、注意力与损失定义见 模型与训练说明。
The model uses an encoder-decoder architecture: a pinyin encoder represents the input syllables, and a Chinese decoder generates characters using preceding context. Training learns both Chinese history modeling and pinyin-conditioned prediction. The encoder is named post and the decoder pre in code. See the model and training guide for the two-pass forward process, attention, and losses.
| 路径 / Path | 用途 / Purpose |
|---|---|
main.py, tasks/ |
任务入口与训练流程 / Task entry points and training |
datasets_pipeline/, tools/subset.py |
数据处理与语料抽样 / Data processing and corpus sampling |
model/, tokenizer/, loss/, metrics/ |
模型、分词器、损失和评估 / Models, tokenizer, losses, and evaluation |
export/, tools/build_pack_v2_2.py, demo.py |
导出、打包与推理 / Export, packaging, and inference |
config/, vocabs/ |
运行配置与词表 / Configuration and vocabularies |
运行测试 / Run tests:
pixi run python -m pytest tests详细文档入口 / Documentation index: 中文 · English。
本项目使用 Apache License 2.0。问题与建议请提交至 Issues。
Licensed under Apache License 2.0. Report problems and suggestions through Issues.