Leveraging ensemble LLMs and contextual embeddings for case-based reasoning in the legal domain.

Abeyratne, Ramitha

Abstract

This research investigates the integration of Case-Based Reasoning (CBR) with Retrieval-Augmented Generation (RAG) for Large Language Models (LLMs) to enhance the reliability of legal question-answering systems. Thus far, we have developed a structured retrieval mechanism using CBR to improve the contextual relevance of generative outputs. Additionally, we introduced two novel alignment-based evaluation metrics—weighted and unweighted—which demonstrated superior performance over existing baselines in assessing QA responses. Our experimental validation on a legal dataset confirmed the effectiveness of the CBR-RAG approach in improving response accuracy. Moving forward, we aim to refine weighting strategies for alignment metrics and enhance textual representations to improve evaluation robustness. Furthermore, we plan to extend our study beyond the legal domain by conducting a comparative analysis across multiple datasets, ensuring broader applicability of the CBR-RAG framework.

Citation

ABEYRATNE, R. 2025. Leveraging ensemble LLMs and contextual embeddings for case-based reasoning in the legal domain. In Martin, K. and Ye, X. (eds.) ICCBR-WS 2025: joint proceedings of the workshops and doctoral consortium at the 33rd International conference on case-based reasoning (ICCBR-WS 2025) co-located with the 33rd International conference on case-based reasoning (ICCBR 2025), 30 June 2025, Biarritz, France. CEUR workshop proceedings, 3993. Aachen: CEUR-WS [online], pages 68-73. Available from: https://ceur-ws.org/Vol-3993/short1.pdf

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 33rd International conference on case-based reasoning workshops and doctoral consortium (ICCBR-WS 2025) co-located with the 33rd International conference on case-based reasoning (ICCBR 2025)
• Start Date: Jun 30, 2025
• Acceptance Date: Apr 6, 2025
• Online Publication Date: Jun 12, 2025
• Publication Date: Jul 8, 2025
• Deposit Date: Aug 1, 2025
• Publicly Available Date: Aug 1, 2025
• Publisher: CEUR-WS
• Peer Reviewed: Peer Reviewed
• Series Title: CEUR-workshop proceedings
• Series Number: 3993
• Series ISSN: 1613-0073
• Book Title: ICCBR-WS 2025
• Keywords: Case-based reasoning (CBR); Retrieval augmented generation (RAG); Large language models (LLMs); LLMs-as-Judges; Case alignment; Embeddings
• Public URL: https://rgu-repository.worktribe.com/output/2959168
• Publisher URL: https://ceur-ws.org/Vol-3993/