Impressions from CLS 2024
The Computational Legal Studies workshop (CLS 2024) at Singapore Management University was an discussion forum to get feedback on ideas in draft model.
Deepdive Labs presented “TRACE: A Framework to Assist Auditors in Evaluating Regulatory Compliance”, a work-in-progress. You can see the paper and prototype here.
Divya and I attended the Computational Legal Studies workshop (CLS 2024) at Singapore Management University last week. It was a very nice experience getting back to academia and presentations after a significant amount of time for both of us. The attendees were a mixture of academics and startup folks (Deep Dive Labs included), who were able to present their work related to the field of law and regulations. This came closely on the heels of TechLawFest held at the Sands Expo and Convention Center. Here are a few of the presentations we found very interesting:
The opening day keynote “Introducing Computational Legal Studies“ by Professor Jerrold Soh, Deputy Director, Centre for Computational Law (CCLAW) at Yong Pung How School of Law was a fantastic introduction to computational law. His keynote was able to straddle the lines between computer scientists, legal students and startup professionals. We learned quite a bit from his presentation about how the legal profession views this new world of AI and its role in computational law.
What we absolutely loved about the work was reading through some of the judgements that were ranked highest. The opening paragraph could probably be the start of a P.G Woodhouse or a R.K Narayanan novel!
The talk by Professor Lim How Khang, Director, Centre for Computational Law at Yong Pung How School of Law titled “From Judgment to Data: Decoding Judicial Writing with Large Language Models” on analyzing the text of judgments by far was the most technically in-depth presentation. It could well be shown in a computer science conference. His team has analyzed all the judgments in Singapore from the year 2000 for a number of criteria like readability, explainability and multiple other factors.
The section of the talk on using the Llama 70B model locally had technical depth; he talked about the machine configurations, comparisons with other models to come to a judgment if the result was “good enough”, the resource considerations and tradeoffs, and also mentioned techniques to speed up the processing speed, like prefix caching etc.. A meaningful extension that struck us about this work is it could just easily be expanded and extended to other countries and evaluate the judgment texts in different countries. This could be helpful to judges to understand how their judgements are impacting the general public and societal good – a very noble vision.
A presentation titled “Refining Regulatory Drafting with AI/ML: Detecting Contradictions in Indonesian Legal Articles” by Evandri, a “Happy Legal Researcher” at the legal-civic startup indexalaw and Alvin, the AI/ML Engineering Manager at Pinhome used text embeddings and a RAG (retrieval-augmented generation) approach to analyze a set of regulations to find potential conflicts within each regulation. The reason this was most impressive was because of the real-world impact. They had managed to reach out to regulators in Indonesia and show them the potential conflicts in regulations which they had uncovered and had persuaded the regulators to either revoke, update, or merge the regulations based on their internal processes. Such real-world impact is seriously cool. Kudos to these guys.
Daniel Katz, Director of The Law Lab, Illinois Tech, wearing multiple hats as a professor and an entrepreneur associated with 273 ventures presented a keynote with an interesting nugget - a LLM called KL3M that is fairly trained, with a certification to boot! This is an analogy between fair trade certifications and fairly trained certifications. The idea was that just as fair trade ensured the farmers would get a fairer deal for their work, fair training has similar goals around data provenance used for LLM training. There is going to be a performance hit on such models, but the assertion by such model developers is that it will be useful enough for a limited set of use cases. Details are yet to be understood, but the analogy of getting an equivalent of a fair trade certification onto LLMs was a new one for us.
There were a lot of other good discussions around computable contracts and the legality of training versus copyright in the forum. We are very grateful to have been afforded an opportunity to present in that forum and certainly hope the workshop continues next year and we will have an opportunity to participate in it again.
