Speech LLM Engineer · Supervised Fine-Tuning
I work on supervised fine-tuning (SFT) for speech interaction at iFLYTEK. My work involves preparing and refining training data, fine-tuning models, and analyzing how training changes their responses. I focus on making responses more natural while preserving reasoning and general capabilities.
I use the team's existing evaluation workflows to compare model versions and make targeted fixes to scripts and processes as needed. Previously, my research at Tianjin University focused on image restoration and fusion.
Email · Personal website · Research · Writing · Zhihu · Projects
| Interest | Where to start |
|---|---|
| Speech LLM SFT: training data and model behavior | Reviewing rewritten SFT answers walks through conditions, explanations, certainty and missing input information. A small text-only study adds a first training comparison, a question-rewording follow-up, and code and raw outputs. |
| Published research: AIR-Fusion | IJCAI 2026 paper on image restoration and infrared–visible fusion. The project page presents the method and published results. |
| Independent ASR study: Whisper decoding | Beam-size comparison on 40 recordings, with paired transcripts and reproduction scripts. |
The SFT and ASR experiments are small independent studies using open models and public or synthetic data. The SFT study is text-only; its results do not establish general capability retention or speech quality.
Interactive All-in-One Image Restoration and Fusion
Bing Cao, Qiang Zhang†, Xingxin Xu, Pengfei Zhu
IJCAI 2026 · † First student author
Restoring degraded images and fusing infrared and visible information with text instructions and a pretrained diffusion model.
Paper, figures and citation are public. Implementation and model weights are not publicly available.
Project page · Paper materials · PDF · Publication · BibTeX
Selected articles and answers in Chinese. All writing · Zhihu
- pass@10 比 pass@1 低,先检查评测口径 — 区分采样覆盖率、平均正确率和多轮成功率,附构造案例与复核脚本。
- 从文本问答到语音 SFT:一条训练样本要保留什么? — 用构造案例检查朗读稿、实际音频、监督目标和数据划分;未生成音频或运行训练。
- SFT 原题高分,换个问法会怎样?一次小模型实测 — 固定模板、换问法,以及格式和数值分别判分。
- 训练答案改写后,怎么检查它还适不适合做 SFT? — 用构造案例检查条件、解释、确定性和输入信息。 Zhihu edition.
- 口语化 SFT 后,模型为什么可能答得更顺,却答得不准? — 改写答案时,哪些信息容易一起被删掉。
- 为什么大模型在微调后,就失去了通用能力? — 从具体错误检查加载、模板和训练数据。
- 如何评估 LLM? — 相同正确率下,错误分布可能有什么不同?
- beam 从 1 改成 5:40 条录音里,哪些结果变了? — 真实音频实验,附逐句输出、固定协议和复现脚本。 Zhihu edition.
- AIR-Fusion:用扩散模型同时做图像恢复与融合 — 结合论文原图解释 CMBA、TCR 和实验结果。 Zhihu edition.
- SFT Rewrite Checker — Compare original and rewritten answers, inspect rule-based review cues, and export decisions with notes. Data stays in the current browser page. Quick start · Code and examples.
- Speech Evaluation Demo — A small personal project for comparing reference transcripts and recognition outputs, calculating character error rate, and inspecting character differences. Python / SQLite. Browse sample results.
- SubTracker — Track subscriptions, monthly totals and renewal dates in your browser. JavaScript / IndexedDB. Try it.
| Period | Institution | Focus |
|---|---|---|
| 2026–present | iFLYTEK · 科大讯飞 | Supervised fine-tuning for speech interaction |
| 2023–2026 | Tianjin University · 天津大学 | Computer Technology · Computer vision |
| 2019–2023 | Anhui University · 安徽大学 | Data Science and Big Data Technology |
You can call me Nate.