Unsupervised similarity-aligned ensemble metrics for evaluating legal Q&A LLM responses. [Dataset]

Abstract

This repository contains code and datasets related to the paper titled "Unsupervised Similarity-Aligned Ensemble Metrics for Evaluating Legal Q&A LLM Responses".

Citation

ABEYRATNE, R. 2025. Unsupervised similarity-aligned ensemble metrics for evaluating legal Q&A LLM responses(LINK ONLY). Hosted on GitHub [online]. Available from: https://github.com/RAbeyratne/aligned_ensemble_judge

• Online Publication Date: Feb 17, 2025
• Publication Date: Feb 20, 2025
• Deposit Date: Mar 17, 2025
• Publicly Available Date: Mar 17, 2025
• Keywords: Ensemble large language models (LLMs); Legal Q&A; LLM-as-a-judge; Embeddings; Reverse generation; Semantic similarity; Case-alignment
• Public URL: https://rgu-repository.worktribe.com/output/2754880
• Publisher URL: https://github.com/RAbeyratne/aligned-ensemble-judge
• Related Public URLs: https://rgu-repository.worktribe.com/output/2754840 (Conference paper associated with this output)
• Collection Date: Feb 20, 2025
• Collection Method: A codebase for implementing and evaluating the metrics was used. Several datasets were produced including ALQU master dataset, SLSC master dataset, four ALQA results datasets and four SLSC results datasets.