Implementasi Retrieval Augmented Generation untuk Layanan Informasi Kampus dengan Chatbot Berbasis AI
Abstract
Information technology plays a crucial role in enhancing knowledge and awareness, particularly in educational institutions with intense competition among universities. Students and the general public still face difficulties in obtaining the necessary information due to the numerous available platforms and the lack of effective search features. This research aims to develop an AI-based chatbot, specifically using Large Language Models (LLM), that can offer efficient and responsive solutions for information services in educational institutions. By integrating the chatbot into the Telegram application, it is expected to facilitate users in quickly and accurately obtaining information. This chatbot is built with a Retrieval Augmented Generation (RAG) approach that enhances LLM capabilities and helps in obtaining fast and accurate information. The chatbot is developed using software such as Langchain to manage the RAG process, Python as the programming language, ChromaDB as the vector database, and Gemini AI to support the LLM model. Performance evaluation is conducted using RAGAS to ensure the quality and accuracy of responses. Testing with 10 sample questions showed positive results, a context precision score of 93%, faithfulness 100%, answer relevancy 96%, context recall 100%, answer correctness 71%, and answer similarity 93%. These results indicate that an RAG-based chatbot can be an effective tool to improve information accessibility in educational institutions.
References
F. Febriyanto, F. Pramudya, P. Sokibi, K. Kusnadi, and R. Taufiq Subagio, “Sosialisasi Penerapan Sistem Informasi dan Pendataan Pengunjung pada Keraton Kasepuhan Kota Cirebon,” JPUCIC, vol. 1, 2022.
M. Asfi, Kusnadi, and Kristianto, “Analisis Kepuasan Alumni Terhadap Pelayanan Akademik dengan Metode Importance Performance Analysis Berbasis Web (Studi Kasus: Universitas CIC),” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 4, pp. 1007–1015, 2020, doi: 10.30865/mib.v4i4.2343.
D. Bill and T. Eriksson, “Fine-tuning a LLM using Reinforcement Learning from Human Feedback for a Therapy Chatbot Application,” 2023.
L. Indahsari, K. Kusnadi, and T. E. Putri, “Rancang Bangun LINE Chatbot Informasi dan Edukasi Kesehatan Mental Menggunakan Algoritma Jaro Winkler,” Jurnal Eksplora Informatika, vol. 10, no. 2, pp. 68–79, Mar. 2021, doi: 10.30864/eksplora.v10i2.428.
L. Monigatti, “Retrieval-Augmented Generation (RAG): From Theory to LangChain Implementation,” Towards Data Science. Accessed: Feb. 21, 2024. [Online]. Available: https://towardsdatascience.com/retrieval-augmented-generation-rag-from-theory-to-langchain-implementation-4e9bd5f6a4f2
Y. Gao et al., “Retrieval-Augmented Generation for Large Language Models: A Survey,” arXiv e-prints, p. arXiv:2312.10997, Dec. 2023, doi: 10.48550/arXiv.2312.10997.
P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” arXiv e-prints, p. arXiv:2005.11401, May 2020, doi: 10.48550/arXiv.2005.11401.
Z. Feng, X. Feng, D. Zhao, M. Yang, and B. Qin, “Retrieval-Generation Synergy Augmented Large Language Models,” arXiv e-prints, p. arXiv:2310.05149, Oct. 2023, doi: 10.48550/arXiv.2310.05149.
R. S. Pressman, Software Engineering: A Practitioner’s Approach, 7th ed. New York: McGraw-Hill, 2010.
U. Jamil, Youtube. Accessed: May 13, 2024. [Online]. Available: https://www.youtube.com/watch?v=rhZgXNdhWDY&list=PLilBdZxzK8rTnz7hpfxXNeSro15uGw_cO&index=3
N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” CoRR, vol. abs/1908.10084, 2019, [Online]. Available: http://arxiv.org/abs/1908.10084
F. Arasyi, “indo-sentence-bert: Sentence Transformer for Bahasa Indonesia with Multiple Negative Ranking Loss,” huggingface. Accessed: Mar. 09, 2024. [Online]. Available: https://huggingface.co/firqaaa/indo-sentence-bert-base
P. Sai, “The Ultimate Guide on Retrieval Strategies – RAG (part-4),” Chatgen.ai. Accessed: May 06, 2024. [Online]. Available: https://chatgen.ai/blog/the-ultimate-guide-on-retrieval-strategies-rag-part-4/
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Yoni Tribber, Kusnadi, Marsani Asfi
This work is licensed under a Creative Commons Attribution 4.0 International License.
http://creativecommons.org/licenses/by/4.0
