Immediate performance
Does AI improve the work produced?
Compare the memos students produce while assistance is available.
The legal education study
Project Sophia
Learning legal judgment with AI: a proposed study informing Phase 1 of Juridical AI.
Study in development
- Intended participants
- First-year law students undertaking their first memo-writing exercises
- Proposed design
- Two randomly assigned groups; three proposed memo-writing exercises
- Research focus
- Immediate assisted performance and later unaided skill
The question
What remains when AI assistance is removed?
Does AI assistance improve students’ immediate memo-writing performance, and what happens to their legal research and reasoning skills when that assistance is later removed?
The intended participants are first-year law students undertaking their first memo-writing exercises.
Project Sophia focuses on the development of student skills and is intended to inform Juridical Learn, the platform’s initial focus on legal education.
Experimental design
Three exercises. Two assigned groups.
Students will be randomly assigned to Group A or Group B, using comparable but distinct memo-writing tasks.
On narrow screens, scroll the table horizontally to compare both groups. You can also focus the table and use the arrow keys.
| Exercise | Group A | Group B | Purpose |
|---|---|---|---|
| Exercise 1 | Without AI | Without AI | Establish baseline performance |
| Exercise 2 | Without AI | With AI | Compare immediate performance with and without AI |
| Exercise 3, after a delay | Without AI, if included | Without AI | Examine later unaided performance and transfer |
What the study seeks to distinguish
Assisted performance is one part of the picture.
Later unaided skill
Do stronger skills carry forward?
Examine the legal research and reasoning students demonstrate in a later task without AI.
Observable process
Which actions help explain performance?
Study the research and verification actions students take while completing their work.
Candidate measures, still provisional.
Possible measures include memo quality, authority selection and verification, legal reasoning, time spent, and confidence relative to performance.
AI permissions remain to be specified.
The AI condition’s configuration and permitted uses remain to be specified. This two-group design does not independently compare unrestricted AI with structured Juridical assistance.
Connection to Juridical Learn
A platform shaped by the evidence.
The proposed platform could administer exercises, apply the assigned AI permissions, record observable work, collect submissions, and support human assessment.
That record could connect authorities consulted, verification decisions, and revisions to the final memo and its feedback.
The study is designed to test assumptions about learning. Improved AI-assisted output would not, by itself, establish improved unaided skill.
Findings may support, challenge, or reshape the platform concept.
See the platform vision