AI finds racial restrictions in millions of real estate records

California law requires counties to remove racially restrictive language, which has been constitutionally unenforceable since 1948, from property deeds. Researchers trained large-scale language models to help them do that.

When Dan Ho bought a home in Palo Alto, he said, “We believe that this property cannot be used or occupied by anyone of African, Japanese, Chinese, or Mongolian descent, except for the number of people it can accommodate.” He recalled that he had to sign a document that said, “No.” It’s like a servant to the white man. ”

“This is stunning evidence of housing discrimination in this region, which has been constitutionally unenforceable since 1948,” said William Benjamin Scott and Luna M. Scott Professor of Law, Director of RegLab, and Human-Centered Research Fellow at Stanford University. Mr Ho said. Artificial Intelligence (HAI).

Racially restrictive covenants still litter the deed record across the country, even though the Supreme Court has ruled that such provisions are unenforceable.

In 2021, California enacted a law requiring the state’s 58 counties to create a program to identify and compile conduct records containing racial codes.

But the new law also poses difficult challenges. In Santa Clara County alone, there are 24 million deed documents dating back to 1850, totaling 84 million pages. “Prior to this collaboration, our team manually read nearly 100,000 pages over several weeks to identify racial codes,” said Luis Chiaramonte, assistant county clerk and recorder. Ta. “It was a difficult task.”

To comply with the law, some California counties contract with commercial vendors. For example, Los Angeles County hired a company for $8 million to conduct this scan over a seven-year period. In other jurisdictions, citizen teams are bravely crowdsourcing these efforts, with thousands of volunteers pouring into deed records. However, not all jurisdictions have such resources.

Santa Clara County, home to Silicon Valley, took a different approach. Stanford University’s Regulatory, Evaluation, and Governance Laboratory (RegLab) partnered with the county to harness the power of AI, particularly large-scale language models, to assist in this monumental task.

“Santa Clara County has aggressively reviewed millions of documents to remove discriminatory language from property records,” Chief Operating Officer Greta Hansen said in a statement. “We are grateful for our partnership with Stanford University, which has allowed the county to significantly expedite this process and save taxpayer money and staff time.”

The team, led by Stanford University’s RegLab and including HAI affiliates and Professor Peter Henderson from Princeton University, will curate a collection of racial codes from various jurisdictions around the country to detect racial codes. near-perfectly trained a state-of-the-art open language model. Accuracy.

The team released the paper at https://reglab.github.io/racialcovenants/ to help all jurisdictions facing this challenge more effectively identify, compile, and We are making the model available for development.

“We estimate that this system will save 86,500 person hours and cost less than 2% compared to comparable proprietary models,” said co-lead author and RegLab fellow Faiz Surani. The research team focused on the most problematic records, 5.2 million deeds from 1902 to 1980.

The team also figured out how to cross-reference historical maps to find most of these assets. They included text descriptions of the map (e.g., “Map (that) was recorded…June 6, 1896, page 25 of map book ‘I’.”) in Santa Clara County Surveyor’s Office administrative records. The geographical location was determined by matching. .

“We thought that of all the items that come out of state law, it could be nearly impossible to map,” Chiaramonte said.

These maps revealed surprising insights into the evolution and spread of racial covenants in Santa Clara County.

The researchers estimated that in 1950, one in four properties in Santa Clara County was subject to racial covenants. Only 10 developers were responsible for one-third of the codes identified, suggesting that developers had a large say in how Santa Clara County was built. The Racial Code excluded African Americans at the same rate as Asian Americans, even though the African American population was less than one-tenth of the Asian American population. The research team found a notable example of a cemetery owned by the city of San Jose that contained only “white” burial deeds. This contradicts the conventional historical explanation that racial codes were only used between private parties after the Supreme Court prohibited state-based racial zoning.

“We believe this is a compelling example of academic and government collaboration to make this type of legislative mandate much more achievable and to shed light on historical patterns of housing discrimination.” ” said co-principal Milak Suzgan, J.D./Ph.D. D. Computer science student.

Chiaramonte agreed. “This collaboration opens new avenues for how technology can be leveraged to accomplish the critical task of identifying, mapping, and editing racial codes. The time required to review has been significantly reduced.