One proposal introduced Wednesday and co-sponsored by a Democratic presidential candidate, Sen. Cory Booker, would require big companies to test the “algorithmic accountability” of their high-risk AI systems, such as technology that detects faces or makes important decisions based on your most sensitive personal data.”Computers are increasingly involved in so many of the key decisions Americans make with respect to their daily lives — whether somebody can buy a home, get a job or even go to jail,” Sen. Ron Wyden said in an interview with The Associated Press. The Oregon Democrat is co-sponsoring the bill.”When the companies really go into this, they’re going to be looking for bias in their systems,” Wyden said. “I think they’re going to be finding a lot.”The Democrats’ proposal is the first of its kind, and may face an uphill battle in the Republican-led Senate. But it reflects growing — and bipartisan — scrutiny of the largely unregulated data economy — everything from social media feeds, online data brokerages, financial algorithms and self-driving software that are increasingly impacting daily life. A bipartisan Senate bill introduced last month would require companies to notify people before using facial recognition software on them, while also requiring third-party testing to check for bias problems.Academic studies and real-life examples have unearthed facial recognition systems that misidentify darker-skinned women , computerized lending tools that charge higher interest rates to Latino and black borrowers, and job recruitment tools that favor men in industries where they already dominate.”There’s this myth that algorithms are these neutral, objective things,” said Aaron Rieke, managing director at advocacy group Upturn. “Machine learning picks up patterns in society — who does what, who buys what, or who has what job. Those are patterns shaped by issues we’ve been struggling with for decades.”
New Standards
President Donald Trump’s administration is also taking notice and has made the development of “safe and trustworthy” algorithms a major objective of the White House’s new AI initiative . But it would do so mostly by strengthening an existing industry-driven process of creating technological standards.
“There’s a need for greater transparency and data comparability,” and for detecting and reducing bias in these systems, said Commerce Undersecretary Walter Copan, who directs the National Institute of Standards and Technology. “Consumers are essentially flying blind.”
Dozens of facial recognition developers, including brand-name companies like Microsoft, last year submitted their proprietary algorithms to Copan’s agency so that they could be evaluated and compared against each other. The results showed significant gains in accuracy over previous years.
But Wyden said the voluntary standards are not enough.
“Self-regulation clearly has failed here,” he said.
In a bolder move from the Trump administration, the federal Department of Housing and Urban Development has charged Facebook with allowing landlords and real estate brokers to systematically exclude groups such as non-Christians, immigrants and minorities from seeing ads for houses and apartments.
Booker, in a statement about his bill, said that while HUD’s Facebook action is an important step, it’s necessary to dig deeper to address the “pernicious ways” discrimination operates on tech platforms, sometimes unintentionally.