Tesis Mahasiswa NIM 14240001

Tesis Mahasiswa NIM 14240001

• By Suharyadi;
• Tesis
• 0

Sistem Retrieval-Augmented Generation (RAG) membutuhkan konteks relevan agar jawaban didukung bukti. Penelitian ini mengembangkan RAG TIAS melalui pipeline multi-stage yang mengintegrasikan BM25, FAISS, Reciprocal Rank Fusion, neural re-ranking berbasis cross-encoder, dan context validation berbasis aturan. Evaluasi menggunakan 201 chunk dan 50 query. Pada ablasi terkontrol dengan candidate pool tetap, neural re-ranking meningkatkan Recall@10 dari 0.7639 menjadi 0.8136, MRR@10 dari 0.6550 menjadi 0.8040, dan Hit@10 dari 0.88 menjadi 0.94; hanya peningkatan MRR@10 yang signifikan. Token F1 generatif meningkat dari 0.4071 menjadi 0.4348, tetapi tidak signifikan. Context validation mengelompokkan 40 query sebagai OK dan 10 sebagai WEAK tanpa ABSTAIN serta tidak meningkatkan Token F1. Hasil menunjukkan bahwa manfaat utama pipeline terletak pada kualitas peringkat, sedangkan kontribusi generatif dan context validation perlu ditafsirkan secara terbatas.

• Katakunci Context Validation, Hybrid Retrieval, Neural Re-ranking, Retrieval-Augmented Generation, TIAS
• URL https://nusamandiri.ac.id/
• Penerbit Universitas Nusa Mandiri, Jakarta DKI Jakarta

REFERENSI

6008. Vaswani et al., “Attention is All you Need,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2017, pp. 5998–6008.

[2]      Z. Ji et al., “Survey of Hallucination in Natural Language Generation,” ACM Comput. Surv., vol. 55, no. 12, pp. 1–38, Dec. 2023, doi: 10.1145/3571730.

[3]      P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2020, pp. 9459–9474.

[4]      V. Karpukhin et al., “Dense Passage Retrieval for Open-Domain Question Answering,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Stroudsburg, PA, USA: Association for Computational Linguistics, 2020, pp. 6769–6781. doi: 10.18653/v1/2020.emnlp-main.550.

[5]      X. Ma, K. Sun, R. Pradeep, and J. Lin, “A Replication Study of Dense Passage Retriever,” Apr. 2021.

[6]      S. Robertson and H. Zaragoza, “The Probabilistic Relevance Framework: BM25 and Beyond,” Foundations and Trends® in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009, doi: 10.1561/1500000019.

[7]      Kementerian Luar Negeri Republik Indonesia, Peraturan Menteri Luar Negeri Nomor 15 Tahun 2020 tentang Perubahan Atas Peraturan Menteri Luar Negeri Nomor 11 Tahun 2019 Tentang Kebijakan Pemberian Hibah Kepada Pemerintah Asing/Lembaga Asing Untuk Periode Jangka Menengah. Indonesia, 2020, pp. 1–16.

[8]      Lembaga Dana Kerja Sama Pembangunan Internasional, “The Indonesian AID Scholarship.” Accessed: Feb. 14, 2026. [Online]. Available: https://tias.kemenkeu.go.id/landing/

[9]      T. Q. Sun and R. Medaglia, “Mapping the challenges of Artificial Intelligence in the public sector: Evidence from public healthcare,” Gov. Inf. Q., vol. 36, no. 2, pp. 368–383, Apr. 2019, doi: 10.1016/j.giq.2018.09.008.

[10]    R. Nogueira and K. Cho, “Passage Re-ranking with BERT,” Apr. 2020.

[11]    Y. Gao et al., “Retrieval-Augmented Generation for Large Language Models: A Survey,” Mar. 2024.

[12]    N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” Aug. 2019.

[13]    G. V. Cormack, C. L. A. Clarke, and S. Buettcher, “Reciprocal rank fusion outperforms condorcet and individual rank learning methods,” in Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, New York, NY, USA: ACM, Jul. 2009, pp. 758–759. doi: 10.1145/1571941.1572114.

[14]    O. Ram et al., “In-Context Retrieval-Augmented Language Models,” Trans. Assoc. Comput. Linguist., vol. 11, pp. 1316–1331, Nov. 2023, doi: 10.1162/tacl_a_00605.

[15]    C. D. . Manning, Prabhakar. Raghavan, and Hinrich. Schütze, Introduction to information retrieval. Cambridge University Press, 2009.

[16]    J. Guo et al., “A Deep Look into neural ranking models for information retrieval,” Inf. Process. Manag., vol. 57, no. 6, p. 102067, Nov. 2020, doi: 10.1016/j.ipm.2019.102067.

[17]    Ricardo Baeza-Yates and Berthier Ribeiro-Neto, Modern Information Retrieval: The Concepts and Technology Behind Search. New York: ACM Press, 2011.

[18]    J. Johnson, M. Douze, and H. Jegou, “Billion-Scale Similarity Search with GPUs,” IEEE Trans. Big Data, vol. 7, no. 3, pp. 535–547, Jul. 2021, doi: 10.1109/TBDATA.2019.2921572.

[19]    M. Douze et al., “The Faiss library,” Oct. 2025.

[20]    N. Hamzah and Z. P. Putra, “Attention-Based Explainability for Cross-Lingual Neural Information Retrieval in Low-Resource Languages: A LaBSE Framework with a Bilingual Medical Context Case Study,” Artificial Intelligence and Applications, Jun. 2026, doi: 10.47852/bonviewAIA62027530.

[21]    M. Firmansyah, D. Marutho, A. Ilham, I. Saputra, and E. Vogiatzi, “Resource Efficient Semantic Retrieval Pipeline via Generative Captioning and Text-to-Text Transformers for Bridging the Modality Gap,” Journal of Intelligent Computing & Health Informatics, vol. 6, no. 2, p. 84, Oct. 2025, doi: 10.26714/jichi.v6i2.19240.

[22]    W. A. G. Kodri, M. Haris, and R. Fitriadi, “Fine-Hybrid: Integration of BM25 And Finetuned SBERT to Enhance Search Relevance,” Teknika, vol. 14, no. 2, pp. 213–222, Jul. 2025, doi: 10.34148/teknika.v14i2.1229.

[23]    H. Tohir, N. Merlina, and M. Haris, “UTILIZING RETRIEVAL-AUGMENTED GENERATION IN LARGE LANGUAGE MODELS TO ENHANCE INDONESIAN LANGUAGE NLP,” JITK (Jurnal Ilmu Pengetahuan dan Teknologi Komputer), vol. 10, no. 2, pp. 352–360, Nov. 2024, doi: 10.33480/jitk.v10i2.5916.

[24]    A. Asai, Z. Wu, Y. Wang, A. Sil, and H. Hajishirzi, “Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection,” Oct. 2023.

[25]    S.-Q. Yan, J.-C. Gu, Y. Zhu, and Z.-H. Ling, “Corrective Retrieval Augmented Generation,” Oct. 2024.

[26]    A. Fauzi and G. Ginabila, “INFORMATION RETRIEVAL SYSTEM PADA FILE PENCARIAN DOKUMEN TESIS BERBASIS TEXT MENGGUNAKAN METODE VECTOR SPACE MODEL,” Jurnal Pilar Nusa Mandiri, vol. 15, no. 1, pp. 41–46, Mar. 2019, doi: 10.33480/pilar.v15i1.61.

[27]    N. S. Sunendar and I. Saputra, “COMPARATIVE PERFORMANCE OF TRANSFORMER AND LSTM MODELS FOR INDONESIAN INFORMATION RETRIEVAL WITH INDOBERT,” Jurnal Pilar Nusa Mandiri, vol. 21, no. 2, pp. 228–233, Sep. 2025, doi: 10.33480/pilar.v21i2.6920.

[28]    O. Khattab and M. Zaharia, “ColBERT,” in Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA: ACM, Jul. 2020, pp. 39–48. doi: 10.1145/3397271.3401075.

[29]    W. Sun et al., “Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA: Association for Computational Linguistics, 2023, pp. 14918–14937. doi: 10.18653/v1/2023.emnlp-main.923.

[30]    Agus Rahmat Fadillah and Irwansyah Saputra, “Implementation of Knowledge-Based Graph Neural Networks for Reasoning and Ranking Medical Entities from CORD-19 Texts,” Journal of Novel Engineering Science and Technology, vol. 4, no. 03, pp. 86–98, Nov. 2025, doi: 10.56741/jnest.v4i02.1049.

[31]    G. Izacard and E. Grave, “Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Stroudsburg, PA, USA: Association for Computational Linguistics, 2021, pp. 874–880. doi: 10.18653/v1/2021.eacl-main.74.

[32]    O. Yoran, T. Wolfson, O. Ram, and J. Berant, “Making Retrieval-Augmented Language Models Robust to Irrelevant Context,” May 2024.

[33]    W. Yu et al., “Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA: Association for Computational Linguistics, 2024, pp. 14672–14685. doi: 10.18653/v1/2024.emnlp-main.813.

[34]    Y. Wang, R. Ren, J. Li, W. X. Zhao, J. Liu, and J.-R. Wen, “REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering,” Nov. 2024.

[35]    Suharyadi and I. Saputra, “Hybrid Ensemble Retrieval-Augmented Generation for Indonesian Legal Consultation with Keyword Boosting,” Journal of Novel Engineering Science and Technology, vol. 4, no. 02, pp. 71–85, Jul. 2025, doi: 10.56741/jnest.v4i02.1042.

[36]    R. Nogueira, W. Yang, K. Cho, and J. Lin, “Multi-Stage Document Ranking with BERT,” Oct. 2019.

[37]    L. Xiong et al., “Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval,” Oct. 2020.

[38]    G. Izacard et al., “Unsupervised Dense Information Retrieval with Contrastive Learning,” Aug. 2022.

[39]    S. Siriwardhana, R. Weerasekera, E. Wen, T. Kaluarachchi, R. Rana, and S. Nanayakkara, “Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering,” Trans. Assoc. Comput. Linguist., vol. 11, pp. 1–17, Jan. 2023, doi: 10.1162/tacl_a_00530.

[40]    D. Edge et al., “From Local to Global: A Graph RAG Approach to Query-Focused Summarization,” Feb. 2025.

[41]    A. Suryadi and I. Saputra, “Privacy-Preserving Automated QA Dataset Generation for Fine-Tuning LLMs with Local Models and Information Retrieval,” JURNAL INFOTEL, vol. 17, no. 4, pp. 825–838, Jan. 2026, doi: 10.20895/infotel.v17i4.1388.

[42]    J. Lin, X. Ma, S.-C. Lin, J.-H. Yang, R. Pradeep, and R. Nogueira, “Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA: ACM, Jul. 2021, pp. 2356–2362. doi: 10.1145/3404835.3463238.

[43]    R. F. Woolson, “Wilcoxon Signed‐Rank Test,” in Wiley Encyclopedia of Clinical Trials, Wiley, 2008, pp. 1–3. doi: 10.1002/9780471462422.eoct979.

[44]    J. Kaplan et al., “Scaling Laws for Neural Language Models,” Jan. 2020.

[45]    C. Raffel et al., “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” 2020. [Online]. Available: http://jmlr.org/papers/v21/20-074.html.

[46]    F. F. Marcelly, I. Saputra, and M. B. Andra, “EVALUATING PREPROCESSING EFFECTS IN NAME RETRIEVAL USING CLASSICAL IR AND CNN-BASED MODELS,” Jurnal Pilar Nusa Mandiri, vol. 21, no. 2, pp. 218–227, Sep. 2025, doi: 10.33480/pilar.v21i2.6884.