RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models

About

As one of the most advanced techniques in AI, Retrieval-Augmented Generation (RAG) techniques can offer reliable and up-to-date external knowledge, providing huge convenience for numerous tasks. Particularly in the era of AI-generated content (AIGC), the powerful capacity of retrieval in RAG in providing additional knowledge enables retrieval-augmented generation to assist existing generative AI in producing high-quality outputs. Recently, large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation, while still facing inherent limitations, such as hallucinations and out-of-date internal knowledge. Given the powerful abilities of RAG in providing the latest and helpful auxiliary information, retrieval-augmented large language models have emerged to harness external and authoritative knowledge bases, rather than solely relying on the model's internal knowledge, to augment the generation quality of LLMs.

In this tutorial, we comprehensively review existing research studies in retrieval-augmented large language models (RA-LLMs), covering three primary technical perspectives: architectures, training strategies, and applications. As the preliminary knowledge, we briefly introduce the foundations and recent advances of LLMs. Then, to illustrate the practical significance of RAG for LLMs, we categorize mainstream relevant work by application areas, detailing the challenges of each and the corresponding capabilities of RA-LLMs specifically. Finally, to deliver deeper insights, we discuss current limitations and several promising directions for future research.

Our Survey Paper: RAG-Meets-LLMs: Towards Retrieval-Augmented Large Language Models

Slides: Slides-Part-1 and Slides-Part-2

TARGET AUDIENCE AND PREREQUISITES FOR THE TUTORIAL

The audience of this tutorial could be college students, researchers in academic institutions, and industrial AI labs who are interested in Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG). The audience is expected to have basic knowledge of artificial intelligence, language models, and retrieval techniques. However, this tutorial will be presented at the college junior/senior level so that it can be comfortably followed by academic researchers or industrial practitioners who are interested in this emerging field but not quite familiar with it. After attending this tutorial, the audience is expected to have a comprehensive understanding of Retrieval-Augmented Large Language Models and learn how to design a solution for a customized problem.

Tutorial Syllabus

The topics of this tutorial include (but are not limited to) the following:

Retrieval-Augmented Generation (RAG)

Large Language Model (LLM)

Pre-training

Fine-tuning

In-context Learning

Prompting

The tutorial outline is shown below:

Introduction of RA-LLMs (15 minutes)

Architecture of RA-LLMs and Main Modules (30 minutes)

RA-LLM architecture overview
Retriever in RA-LLMs
Retrieval results integration
Pre/Post-retrieval techniques
Special RA-LLM paradigms

Coffee Break (20 minutes)

Learning Approach of RA-LLMs (30 minutes)

Training-free methods
Training-based Methods

Applications of RA-LLMs (30 minutes)

NLP application
Downstream tasks
Domain-specific applications

Challenges and Future Directions of RA-LLMs (15 minutes)

Trustworthy LLMs/RAG/RA-LLMs
Multi-modal RA-LLMs
Quality of external knowledge
Mamba-based RA-LLMs

Q&A (10 minutes)

About

TARGET AUDIENCE AND PREREQUISITES FOR THE TUTORIAL

Event Dates

Tutorial Syllabus

Organization

Tutorial TUTORS

Wenqi Fan

Yujuan Ding

Shijie Wang

Liangbo Ning

Hengyun Li

Dawei Yin

Tat-Seng Chua

Qing Li