Homepage Program Systems: Theory and Applications Русская версия
ISSN 2079-3316 Bilingual online scientific Online scientific journal of the Ailamazyan Program System Institute of the Ailamazyan PSI of PSI of Russian Academy of Science of RAS 12+ 
Volume 17 (2026) . Issue 1 (70) . Paper No. 2 (503)

Artificial intelligence and machine learning

Research Article
DOIDigital Object Identifier 10.25209/2079-3316-2026-17-1-21-56

Legal judgement analysis using large language models

Yuri Serdyuk1Correspondent author, Natalia Vlasova2, Seda Momot3, Elena Suleymanova4

1-4Ailamazyan Program Systems Institute of RAS, Ves'kovo, Russia
1 Yuri Serdyuk — Correspondent author Yuri@serdyuk.botik.ru

Abstract. The article examines the use of the latest-generation large language models (LLMs) — such as ChatGPT, Grok, DeepSeek, GigaChat, and YandexGPT — for analyzing legal judgments. The analysis involved civil, administrative, and criminal cases. A dataset of legal judgments was compiled from the database of judicial and regulatory acts of the Russian Federation, the official portal of the Moscow courts of general jurisdiction, and the website of the Russian Agency for Legal and Judicial Information. Several types of large model tests were proposed and implemented, ground-truth selection principles were outlined, and queries (prompts) were formulated. The models were tested on their ability to predict appellate decisions, map crime descriptions to law articles, and evaluate decisions of multiple judicial authorities in a single case. The ability of the models to make their own consistent decisions was also examined. Testing showed that the correct prediction rate of LLMs on real-world juducial decisions rarely surpasses 50%. A brief overview of recent publications on the use of AI in legal practice is provided. (In Russian).

Keywords: large language models, LLM, legal judgements, dataset, prompt, AI in law, LegalAI

MSC-20202020 Mathematics Subject Classification 68T37; 91F99, 68T05, 68Q60MSC-2020 68-XX: Computer science MSC-2020 68Txx: Artificial intelligence MSC-2020 68T37: Reasoning under uncertainty in the context of artificial intelligence MSC-2020 91-XX: Game theory, economics, finance, and other social and behavioral sciences MSC-2020 91Fxx: Other social and behavioral sciences (mathematical treatment) MSC-2020 91F99: None of the above, but in this section MSC-2020 68T05: Learning and adaptive systems in artificial intelligence MSC-2020 68Qxx: Theory of computing MSC-2020 68Q60: Specification and verification (program logics, model checking, etc.)

For citation: Yuri Serdyuk, Natalia Vlasova, Seda Momot, Elena Suleymanova. Legal judgement analysis using large language models. Program Systems: Theory and Applications, 2026, 17:1, pp. 21–56. (In Russ.). https://psta.psiras.ru/2026/1_21-56.

Full text of article (PDF): https://psta.psiras.ru/read/psta2026_1_21-56.pdf.

The article was submitted 24.12.2025; approved after reviewing 03.02.2026; accepted for publication 17.02.2026; published online 23.02.2026.

© Serdyuk Y., Vlasova N., Momot S., Suleymanova E.
2026
Editorial address: Ailamazyan Program Systems Institute of the Russian Academy of Sciences, Peter the First Street 4«a», Veskovo village, Pereslavl area, Yaroslavl region, 152021 Russia;   Website:  http://psta.psiras.ru Phone: +7(4852) 695-228;   E-mail: ;   License: CC-BY-4.0License text on the Creative Commons site
© Ailamazyan Program System Institute of Russian Academy of Science (site design) 2010–2026