Field-Level Data Consistency Verification in Merchant Vessel Documents Using a Multi-Stage Framework
Keywords:
Multi-Stage Verification, Fuzzy Matching, Semantic Similarity, Natural Language InferenceAbstract
At present, the verification of consistency between two merchant vessel documents that address the same subject matter and are prepared or processed by different operators remains dependent on manual checks and is considerably time-consuming. Accordingly, this study aims to design and develop a document consistency comparison framework for two interrelated document sets by a multi-stage process consisting of rule-based techniques assisted by Natural Language Processing in an offline environment. The proposed framework is intended to enhance verification efficiency and processing speed while mitigating the risk of data leakage. The verification procedure comprises four sequential stages: 1) Exact Matching, 2) Fuzzy Matching, 3) Semantic Similarity, and 4) Natural Language Inference. The experiment was conducted using four categories of data comprising 2,000 files in total: 500 report files (.docx), 500 text-based reference files (.pdf), 500 image-based reference files (.pdf), and 500 gold-label files (.csv) containing ground-truth annotations for 10 target fields. These datasets were employed to assess the performance of the proposed framework using multiple evaluation metrics. The results on the same dataset used to tune the thresholds indicate that the proposed framework improves the Pass rate. For Text-based documents, the Pass rate increased from 87.90% to 96.50%, whereas for Image-based documents, it increased from 72.40% to 85.10%. The highest binary F1-scores were 0.982 and 0.920, respectively.
References
[1] A. M. Rombach and P. Fettke, “Deep learning based key information extraction from business documents: Systematic literature review,” ACM Computing Surveys, vol. 58, no. 2, pp. 1-37, 2026. DOI: 10.1145/3749369.
[2] S. D. Atagong, H. E. Z. Tonnang, K. Senagi, M. Wamalwa, K. M. Agboka, and J. Odindi, “A review on knowledge and information extraction from PDF documents and storage approaches,” Frontiers in Artificial Intelligence, vol. 8, Art. no. 1466092, 2025. DOI: 10.3389/frai.2025.1466092.
[3] H. Zheng, S. Wang, and L. Huang, “A comprehensive survey on document-level information extraction,” in Proceedings of the Workshop on the Future of Event Detection, Miami, FL, USA, 2024, pp. 58-72. DOI: 10.18653/v1/2024.futured-1.6.
[4] K. Silva and T. Silva, “A review on document information extraction approaches,” in Proceedings of the Student Research Workshop Associated with RANLP 2021, Online, 2021, pp. 174-179. DOI: 10.26615/issn.2603-2821.2021_024.
[5] A. M. Rombach, J. Lahann, T. Niesen, and P. Fettke, “Utilizing deep learning for field-level information extraction from German real estate tax notices,” Journal of Emerging Technologies in Accounting, vol. 22, no. 1, pp. 101-118, 2025. DOI: 10.2308/JETA-2023-028.
[6] R. Arroyo, J. Yebes, E. Martínez, H. Corrales, and J. Lorenzo, “Key information extraction in purchase documents using deep learning and rule-based corrections,” in Proceedings of the First Workshop on Pattern-based Approaches to NLP in the Age of Deep Learning, Gyeongju, Republic of Korea, 2022, pp. 11-20. https://aclanthology.org/2022.pandl-1.2/
[7] R. Ed-Daoudi, F. Z. Zakka, M. Ouqassou, and B. Ettaki, “A hybrid OCR-XGBoost-Transformer pipeline for resume parsing with spatial-semantic integration,” Informatica, vol. 49, no. 12, 2025. DOI: 10.31449/inf.v49i12.9588.
[8] Z. Yan, Z. Ye, J. Ge, J. Qin, J. Liu, Y. Cheng, and C. Gurrin, “DocExtractNet: A novel framework for enhanced information extraction from business documents,” Information Processing & Management, vol. 62, no. 3, Art. no. 104046, 2025. DOI: 10.1016/j.ipm.2024.104046.
[9] G. Wang, J. Yu, X. Zhang, T. Deb, X. Liu, and P. He, “A multi-stage pipeline for accurate handwritten information extraction from financial forms,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 6030-6038.
[10] S. Mysore, A. Cohan, and T. Hope, “Multi-vector models with textual guidance for fine-grained scientific document similarity,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Seattle, WA, USA, 2022, pp. 4453-4470. DOI: 10.18653/v1/2022.naacl-main.331.
[11] E. Hassan, A. S. Talaat, and M. A. Elsabagh, “Intelligent text similarity assessment using RoBERTa with integrated chaotic perturbation optimization techniques,” Journal of Big Data, vol. 12, no. 1, Art. no. 164, 2025. DOI: 10.1186/s40537-025-01233-3.
[12] I. Perikos and S. Souli, “Natural language inference with transformer ensembles and explainability techniques,” Electronics, vol. 13, no. 19, Art. no. 3876, 2024. DOI: 10.3390/electronics13193876.
[13] R. Gubelmann, I. Katis, C. Niklaus, and S. Handschuh, “Capturing the varieties of natural language inference: A systematic survey of existing datasets and two novel benchmarks,” Journal of Logic, Language and Information, vol. 33, no. 1, pp. 21-48, 2024. DOI: 10.1007/s10849-023-09410-4.
[14] V. A. Negru, R. Vacareanu, C. Lemnaru, M. Surdeanu, and R. Potolea, “MorphNLI: A stepwise approach to natural language inference using text morphing,” in Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, NM, USA, 2025, pp. 6953-6968. DOI: 10.18653/v1/2025.findings-naacl.385.
[15] L. Madaan, D. Esiobu, P. Stenetorp, B. Plank, and D. Hupkes, “Lost in inference: Rediscovering the role of natural language inference for large language models,” in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Albuquerque, NM, USA, 2025, pp. 9229-9242. DOI: 10.18653/v1/2025.naacl-long.466.
[16] ภวิกา อ้วนละมัย และคณะ, “การพัฒนาระบบแนะนำหนังสือด้วยวิธีการแบบอิงเนื้อหา,” วารสารวิศวกรรมและเทคโนโลยีอุตสาหกรรม มหาวิทยาลัยกาฬสินธุ์, ปีที่ 3, ฉบับที่ 3, หน้า 36-53, 2568. DOI: 10.14456/jeit.2025.13.
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2026 Journal of Engineering and Industrial Technology, Kalasin University
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
ลิขสิทธิ์ของวารสาร
เนื้อหาและข้อมูลในบทความที่ลงตีพิมพ์ในวารสารศูนย์ดัชนีการอ้างอิงวารสารไทย ถือเป็นข้อคิดเห็นและความรับผิดชอบของผู้เขียนบทความโดยตรงซึ่งกองบรรณาธิการวารสาร ไม่จำเป็นต้องเห็นด้วย หรือร่วมรับผิดชอบใด ๆ
บทความ ข้อมูล เนื้อหา รูปภาพ ฯลฯ ที่ได้รับการตีพิมพ์ในวารสารศูนย์ดัชนีการอ้างอิงวารสารไทย ถือเป็นลิขสิทธิ์ของวารสารศูนย์ดัชนีการอ้างอิงวารสารไทย หากบุคคลหรือหน่วยงานใดต้องการนำทั้งหมดหรือส่วนหนึ่งส่วนใดไปเผยแพร่ต่อหรือเพื่อกระทำการใด จะต้องได้รับอนุญาตเป็นลายลักอักษรจากวารสารศูนย์ดัชนีการอ้างอิงวารสารไทยก่อนเท่านั้น
