AI Contract Summary and Clause Risk Review

Project overview

We built a contract review assistant for a commercial legal team that handles a high volume of supplier agreements, NDAs, service contracts and leases. Lawyers upload a contract and receive a structured summary, a table of key terms and a list of clauses that differ from their negotiating playbook, each linked to the relevant passage.

The engagement covered discovery with the lawyers, playbook formalisation, the retrieval and extraction pipeline in Python with LangChain and LlamaIndex, a review workspace and integration with the firm's document management system. It is part of our generative AI development work, and relies on the grounding techniques described in our RAG development services.

The client and the challenge

Junior lawyers spent long stretches reading routine agreements line by line to produce first-pass summaries for partners. Standard positions on liability caps, indemnities, termination and governing law lived in a playbook document that people applied inconsistently, and the same risky wording could be accepted in one contract and pushed back in another.

The firm wanted faster first-pass review without delegating legal judgement to software, answers that always pointed to the exact clause, no hallucinated terms, and strict confidentiality: client contracts could not be used to train any external model or leave the approved environment.

Solution architecture & delivery

Uploaded contracts are converted to text with their structure preserved: numbered clauses, headings, schedules and defined terms. LlamaIndex builds a clause-level index for each document, with every node keeping its clause number and page position, and a separate index of defined terms so the model can resolve words like "Services" or "Confidential Information" when it reads a clause.

We worked with the partners to turn the playbook into structured rules: for each clause type, the preferred position, acceptable fallbacks and positions that must be escalated. A LangChain pipeline then runs a sequence of focused steps rather than a single prompt. It first classifies each clause by type, then for each playbook topic retrieves the matching clauses and asks an LLM to compare them with the rule, returning a structured result: compliant, fallback, escalate or missing, plus a short reason and the cited clause numbers.

The summary is generated last, from the extracted terms and findings rather than from the raw contract, which keeps it consistent with the risk table. A verification step checks that every cited clause exists and that quoted text matches the source; any finding that fails is dropped and flagged for manual reading instead of being shown with a wrong citation.

In the review workspace, lawyers click a finding to jump to the highlighted clause, accept or dismiss it, and add their own notes before exporting a review memo. Models run under an agreement that excludes customer data from training, documents are encrypted and stored in the firm's environment, and access follows matter-level permissions from the document management system. Dismissed findings are reviewed monthly to refine the playbook rules, a feedback loop similar to the one in our legal case management system.

Key features we built

• Structured contract summaries generated from extracted terms
• Key term table: parties, dates, term, renewal, fees, governing law
• Clause-by-clause comparison with the firm's playbook
• Compliant, fallback, escalate and missing risk labels
• Every finding linked to the exact clause and page
• Citation verification that drops unsupported findings
• Accept, dismiss and annotate findings, then export a review memo
• Matter-level permissions inherited from document management

How we delivered it

1. Discovery. Sessions with partners and associates to understand review workflows, contract types and how the playbook was applied in practice.
2. Playbook formalisation. Converting the written playbook into structured positions per clause type and building a test set of past contracts reviewed by the firm's own lawyers.
3. Build. Two-week sprints on document parsing, indexing, the clause comparison pipeline, citation verification and the review workspace, measured against the test set each sprint.
4. Pilot. Use on NDAs and supplier agreements by a small group of lawyers, who reviewed every finding and logged misses and false flags.
5. Roll-out and support. Additional contract types, playbook updates and model version evaluations under a support agreement, following our approach to AI agent security.

Technology choices

• Python — used for document conversion, the review pipeline, citation verification, evaluation against the test set and the workspace API.
• LangChain — orchestrates the multi-step review chain of classification, retrieval, comparison and summary, with structured output parsing at each step.
• LlamaIndex — builds clause-level and defined-term indexes that keep clause numbers and page positions, so every answer can cite its exact source.

Outcomes

• Lawyers start from a structured summary and risk table instead of a blank first read
• Playbook positions are applied the same way to every contract
• Every AI finding can be checked in one click against the source clause
• Unsupported findings are removed rather than presented with incorrect citations
• Partners receive consistent review memos regardless of which associate prepared them

FAQ