How to evaluate AI models for legal work

A practical, task-based process for evaluating AI systems in legal workflows. This site does not publish legal-task model rankings or verify any provider's legal suitability.

This guide is an evaluation aid, not legal advice, a compliance determination, or a model recommendation. No model on this site has been validated for legal work by this guide.

• Legal research and citation checking

  Check whether each authority exists, is current for the relevant date and jurisdiction, and actually supports the proposition attached to it. A plausible-looking citation is not evidence of support.

• Contract review and clause extraction

  Measure whether the system finds obligations, exceptions, defined terms, dates, and conflicts across the full document. Track omissions as well as incorrect flags.

• Summaries and document review

  Compare summaries with a reviewed source record. Score material omissions, unsupported additions, uncertainty handling, and whether quoted text is attributed correctly.

• Drafting and first-pass analysis

  Evaluate whether the output follows the task, uses supplied facts without inventing them, identifies unresolved assumptions, and remains easy for a lawyer to verify and revise.

• 1. Define one bounded task

  Record the practice area, jurisdiction, applicable date, intended user, inputs, expected output, and the attorney responsible for review. Do not combine research, drafting, and contract review into one score.

• 2. Build a reviewable test set

  Start with public or synthetic examples. Include ordinary matters, edge cases, incomplete facts, and questions that lack enough support to answer. Have qualified legal reviewers create or approve reference answers and keep a separate holdout set.

• 3. Score evidence, not confidence

  Use a rubric for factual and legal support, authority existence and relevance, jurisdiction and date fit, completeness, omissions, uncertainty, and safe escalation. A fluent answer or high general benchmark score is not a legal accuracy measure.

• 4. Compare exact configurations

  Record provider, model/version, API or product surface, system and user prompts, retrieval sources, tool settings, and evaluation date. Run the same cases more than once and have reviewers score outputs without seeing the model name when practical.

• 5. Review deployment and professional duties

  Ask counsel, security, and procurement reviewers to assess confidentiality, data retention and training use, access controls, logging, subprocessors, contract terms, client communication, fees, and human supervision for the proposed service and workflow.

Data provided by artificialanalysis.ai. Context limits: the source attached to each model. General model measurements do not establish healthcare or legal suitability.