Developing self-help information for pro se litigants can increase efficiency in courts, improve the public opinion of the civil justice system, and increase access to justice.1 Legal professionals sometimes can struggle to interpret legal information so that it is easy for lay people to understand and inspires confidence. Creating such content is an expensive endeavor. Graphically appealing designs and templates must be commissioned from professional designers, and content changes often require expensive site rework.
In 2017 a group of stakeholders in Ohio, including the Supreme Court of Ohio, the Ohio Access to Justice Foundation, the Ohio Judicial Conference, the Ohio Clerk of Courts Association, the Ohio State Bar Association, and the Alliance of Ohio Legal Aids, set out to design and develop a new, statewide legal assistance portal called Ohio Legal Help. Although Ohio had existing self-help resources, much of this information required significant updates or was too localized for use statewide. Therefore, the Ohio Legal Help team was faced with the question of how to effectively develop legal content, which was both understandable and actionable.
Presenting legal information in a traditional narrative style might have been the fastest and least expensive way to develop content, but Ohio Legal Help was concerned readers might not receive or understand critical information presented in this format. So, other information presentation styles were considered. Readers with limited literacy may find a numbered list displaying visually distinct steps to be more accessible, but it would require significant assistance from external website designers, thereby increasing total project costs. Similarly, a Frequently Asked Questions (FAQs) style presentation may help users find the answers to their specific questions quickly. It would require external website designer support.
To help guide the selection of a content presentation format, Ohio Legal Help commissioned a study to learn how potential consumers of this information would understand and react to three different formats. Rather than decide based on assumptions or conventional wisdom, this data-driven approach would give Ohio Legal Help empirical evidence to guide its investment of staff time and budget. This study focused on answering the following question: Does how legal information is organized and presented online affect how individuals comprehend and potentially act on the information?
Research Design Overview
To determine whether individuals comprehend and are prepared to act on legal information as a function of how this information is presented, a survey experiment was designed wherein an experimenter systematically varied one or more factors among research participants to assess the effect of that variation on one or more measured outcomes.2
The hallmark of an experiment involves a procedure called random assignment, which means assigning individuals to conditions based only on chance, in which each person has a non-zero probability of being assigned to a condition.3 For instance, a researcher might use a coin toss to place employees into one of two groups, assigning all employees whose coin toss results in a “heads” to a group receiving an experimental condition and all “tails” to a control group condition. In this scenario, each employee has a 50 percent chance of being assigned to the experimental group and a 50 percent chance of being assigned to the control group. The random nature of these assignments yields two groups of employees resembling each other in terms of experience, ability, age, and other dimensions. Random assignment allows researchers to make strong claims about causal relationships between experimental manipulations and the effects that are observed. This rules out other explanations (such as preexisting differences among group members) as unlikely.
At the beginning of our study, each participant was randomly assigned to one of four conditions: three conditions focused on the topic of rent-escrow4 or a “control” condition in which no rent-escrow information was presented. Each of the three rent-escrow conditions featured the same legal content but organized this information in different ways:
- numbered condition, in which rent-escrow information was presented in a sequential numbered list format;
- FAQ condition, in which rent-escrow information was presented in blocks of questions and answers;
- narrative condition, in which rent-escrow information was presented in explanatory paragraphs; and
- control condition, in which no rent-escrow information was presented; rather, participants in this condition answered simple math problems (e.g., third- and fourth-grade level), which helped ensure participants engaged in some amount of information processing.
The graphic below shows snippets of the three rent-escrow conditions.
After viewing this information, each research participant answered the same questions measuring their understanding of rent escrow and what their likely behavior would be if faced with a situation in which placing one’s rent into escrow would be an appropriate response.
Overall, 1,602 adult Ohioans (age 18+) completed this experimental survey online because a key objective was to identify the optimal way to present legal information through the Ohio Legal Help portal. Since Ohio Legal Help wanted to ensure the information was accessible to those with a lower socioeconomic status, half of the research participants had a 2017 household income less than $30,000, while half had a household income greater than this amount. The survey was fielded from December 18-31, 2018.
Key Research Findings: Comprehension
After reviewing information about rent escrow or completing simple math exercises, study participants were shown four potential definitions of rent-escrow and were asked to identify the most accurate description. As shown below, study participants in each rent-escrow condition were more likely than those in the control condition to define the rent-escrow concept correctly. This difference was statistically significant.5
Study participants then answered a subjective knowledge question, which measured how much they thought they knew about the topic of rent escrow. Again, study participants who were assigned to any of the three rent-escrow conditions were more likely to feel they were knowledgeable about rent-escrow, compared to those assigned to the control condition. This difference was statistically significant.
Key Research Finding: Behavioral Intent
In addition to providing free and accurate legal information to Ohio residents, Ohio Legal Help seeks to enable users to take action to resolve their legal issues. With this in mind, the study included two measures related to behavioral intent.
If participants were faced with a hypothetical scenario for which rent escrow would be a reasonable response, would they be able to identify the correct action to take? The majority of participants who were assigned any of the three rent-escrow conditions were able to identify the correct action to start the rent-escrow process (i.e., writing a letter to a landlord requesting repairs), as compared to participants assigned to the control condition. This difference was statistically significant.
Certainty in one’s attitude is predictive of behavior.6 To what extent did survey participants feel certain their selected action is the correct one to begin the rent-escrow process? Most study participants who were assigned to any of the three rent-escrow conditions felt certain that “writing to their landlord” is the best action to take, which suggests their intention to act in this manner is likely to be strong. However, relatively few of those assigned to the control condition felt certain this action is the right one to take, which suggests that their intention to act in this manner is likely to be weak. This difference was statistically significant.
Those with lower household incomes (i.e., <$30,000) answered the four questions above in a similar way as those with greater household incomes. This suggests Ohio Legal Help’s rent-escrow information was presented in an accessible, actionable format for residents across a range of socioeconomic levels.
Implications
As compared to those in the control condition, participants in each of the three experimental conditions consistently exhibited more knowledge about rent-escrow, could apply this knowledge to hypothetical situations, and were more certain about this behavior. Furthermore, certainty in the action they could take to protect their legal self-interest was stronger among those assigned to the narrative condition who had lower socioeconomic status.
Based on these findings, Ohio Legal Help decided to present its online legal content in primarily a narrative format. Before the findings of the study, Ohio Legal Help favored presenting the content in the numbered-list format, which would have required significant investment in development. However, the findings suggested that legal content written in a narrative format would be similarly effective for improving pro se litigants’ access to justice. In certain situations, it may be more effective than other formats.
It is our earnest hope that the recounting this experience serves as a useful reminder for all legal professionals to continue checking untested “gut” assumptions. By doing so, we improve access to justice for pro se litigants and reaffirm our commitment to meaningful access to justice for all.7
ABOUT THE AUTHORS
Susan Choe is the executive director of Ohio Legal Help.
Orie Kristel is the CEO and co-owner of Illuminology, the social-science-research firm that designed, deployed, analyzed, and reported the key findings from this study.
Rachel Harris is the content manager of Ohio Legal Help.
Amanda Scott is co-owner of Illuminology and a member of the American Society of Trial Consultants.
Ohio Legal Help is a nonprofit organization that was founded in 2018 to help all Ohioans access the civil justice system. We provide plain-language legal help information, interactive self-help tools, and connections to local legal and community resources that can help people resolve their legal issues. Our legal help information is written by our content team and reviewed by expert private and legal aid lawyers. This ensures that Ohio Legal Help users have information that is easy to understand, action oriented, and accurate.
- Administrative Office of the Courts, Judicial Council of California, “A Report to the California Legislature: Family Law Information Centers: An Evaluation of Three Pilot Programs,” San Francisco, March 2003.
- S. Nock and T. Guterbock, “Survey Experiments,” in P.V. Marsden and J.D. Wright (eds.), Handbook of Survey Research (Bingley, UK: Emerald, 2010), pp. 837-64.
- W. Shadish, T. Cook, and D. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference (Boston: Houghton Mifflin Company, 2002).
- The topic of rent-escrow was selected for the study due to the step-by-step and action-oriented nature of the legal proceeding.
- A statistically significant difference is one that is unlikely to have occurred due to chance alone (i.e., p<.05).
- Z. L. Tormala and D. D. Rucker, “Attitude Certainty: Antecedents, Consequences, and New Directions,” Consumer Psychology Review 1 (2018): 72-89.
- Conference of Chief Justices and Conference of State Court Administrators, “Resolution 5: Reaffirming the Commitment to Meaningful Access to Justice for All,” as proposed by the Access, Fairness and Public Trust Committee, annual meeting, July 25-29, 2015, Omaha, Nebraska.
