All Outputs (5)

AlignLLM: alignment-based evaluation using ensemble of LLMs-as-judges for Q &A. (2025)
Presentation / Conference Contribution
ABEYRATNE, R., WIRATUNGA, N., MARTIN, K., NKISI-ORJ, I. and JAYAWARDENA, L. 2025. AlignLLM: alignment-based evaluation using ensemble of LLMs-as judges for Q&A. In Bichindaritz, I. and López, B. (eds.) Case-based reasoning research and development: proceedings of the 33rd International conference on case-based reasoning 2025 (ICCBR 2025), 30 June - 3 July 2025, Biarritz, France. Lecture notes in computer science (LNCS), 15662. Cham: Springer [online], pages 21-36. Available from: https://doi.org/10.1007/978-3-031-96559-3_2

Evaluating responses generated by large language models (LLMs) is challenging in the absence of ground-truth knowledge, particularly in specialised domains such as law. Increasingly, LLMs themselves are used to evaluate the responses they generate; h... Read More about AlignLLM: alignment-based evaluation using ensemble of LLMs-as-judges for Q &A..

Leveraging ensemble LLMs and contextual embeddings for case-based reasoning in the legal domain. (2025)
Presentation / Conference Contribution
ABEYRATNE, R. 2025. Leveraging ensemble LLMs and contextual embeddings for case-based reasoning in the legal domain. In Martin, K. and Ye, X. (eds.) ICCBR-WS 2025: joint proceedings of the workshops and doctoral consortium at the 33rd International conference on case-based reasoning (ICCBR-WS 2025) co-located with the 33rd International conference on case-based reasoning (ICCBR 2025), 30 June 2025, Biarritz, France. CEUR workshop proceedings, 3993. Aachen: CEUR-WS [online], pages 68-73. Available from: https://ceur-ws.org/Vol-3993/short1.pdf

This research investigates the integration of Case-Based Reasoning (CBR) with Retrieval-Augmented Generation (RAG) for Large Language Models (LLMs) to enhance the reliability of legal question-answering systems. Thus far, we have developed a structur... Read More about Leveraging ensemble LLMs and contextual embeddings for case-based reasoning in the legal domain..

Unsupervised similarity-aligned ensemble metrics for evaluating legal Q&A LLM responses. [Dataset] (2025)
Data
ABEYRATNE, R. 2025. Unsupervised similarity-aligned ensemble metrics for evaluating legal Q&A LLM responses [Dataset]. Hosted on GitHub [online]. Available from: https://github.com/RAbeyratne/aligned_ensemble_judge

This repository contains code and datasets related to the paper titled "Unsupervised Similarity-Aligned Ensemble Metrics for Evaluating Legal Q&A LLM Responses".

SCaLe-QA: Sri Lankan case law embeddings for legal QA. (2024)
Presentation / Conference Contribution
JAYAWARDENA, L., WIRATUNGA, N., ABEYRATNE, R., MARTIN, K., NKISI-ORJI, I. and WEERASINGHE, R. 2024. SCaLe-QU: Sri Lankan case law embeddings for legal QA. In Martin, K., Salimi, P. and Wijayasekara, V. (eds.) 2024. SICSA REALLM workshop 2024: proceedings of the SICSA (Scottish Informatics and Computer Science Alliance) REALLM (Reasoning, explanation and applications of large language models) workshop (SICSA REALLM workshop 2024), 17 October 2024, Aberdeen, UK. CEUR workshop proceedings, 3822. Aachen: CEUR-WS [online], pages 47-55. Available from: https://ceur-ws.org/Vol-3822/short6.pdf

SCaLe-QA is a foundational system developed for Sri Lankan Legal Question Answering (LQA) by leveraging domain-specific embeddings derived from Supreme Court cases. The system is tailored to capture the unique linguistic and structural characteristic... Read More about SCaLe-QA: Sri Lankan case law embeddings for legal QA..

CBR-RAG: case-based reasoning for retrieval augmented generation in LLMs for legal question answering. (2024)
Presentation / Conference Contribution
WIRATUNGA, N., ABEYRATNE, R., JAYAWARDENA, L., MARTIN, K., MASSIE, S., NKISI-ORJI, I., WEERASINGHE, R., LIRET, A. and FLEISCH, B. 2024. CBR-RAG: case-based reasoning for retrieval augmented generation in LLMs for legal question answering. In Recio-Garcia, J.A., Orozco-del-Castillo, M.G. and Bridge, D (eds.) Case-based reasoning research and development: proceedings of the 32nd International conference of case-based reasoning research and development 2024 (ICCBR 2024), 1-4 July 2024, Merida, Mexico. Lecture notes in computer science, 14775. Cham: Springer [online], pages 445-460. Available from: https://doi.org/10.1007/978-3-031-63646-2_29

Retrieval-Augmented Generation (RAG) enhances Large Language Model (LLM) output by providing prior knowledge as context to input. This is beneficial for knowledge-intensive and expert reliant tasks, including legal question-answering, which require e... Read More about CBR-RAG: case-based reasoning for retrieval augmented generation in LLMs for legal question answering..