My bachelor thesis, done at intergator in Dresden: Optimierung der automatisierten Dokumentenklassifikation in technischen Umgebungen durch den Einsatz von Transformer-basierten Modellen — automating the classification and metadata extraction of German technical drawings (engineering plans, site maps, soil surveys) using transformer models.
The documents are messy real-world plans, so the interesting part was getting document-AI models to read structure out of them rather than just OCR'ing text.
I fine-tuned and compared two families of document models:
- Donut (
donut/) — an OCR-free vision-encoder-decoder. Fine-tuned for document-type classification and DocVQA-style field extraction, including a SageMaker training run. - LayoutLMv3 (
layoutlmv3/) — layout-aware token classification, trained to tag the relevant fields on a page. donut+layoutlmv3/— combining the two for metadata extraction.create_dataset/— building and labelling the dataset from the raw plans.main.py— the end-to-end pipeline: run the models, OCR, then fuzzy-match the extracted plan numbers against the EPLASS numbering convention, wrapped in a small Gradio UI.
A few sample plans and the plan-number reference sheet are included so the notebooks have something to run against.
This is research/thesis code — notebooks with experiments, not a packaged library. The follow-on layout-analysis pipeline I built lives in DocSeg2.