Scholars: Captain Picard Got AI Personhood Wrong — Congress Should Say So
In a new commentary for Religion News Service, scholars who study the dignity of the human person take issue with one of Captain Jean-Luc Picard's most famous victories — and they say the law should settle the question before technology settles it for us.
The piece, published November 7, was written as admirers mourn the death of actor Patrick Stewart, who played Picard on "Star Trek: The Next Generation." The authors note that Stewart's Picard inspired a generation of scientists and leaders. But on the question of artificial intelligence, they argue, Picard got it wrong.
Their case turns on the 1989 episode "The Measure of a Man," in which Picard defends the android Lt. Commander Data in a hearing and wins a ruling that Data should be treated as a person, with rights of his own. The scholars write that they disagree with that ruling — and with Picard's position behind it. In their view, an artificial intelligence should never be considered a person, and they call for a federal law that says so plainly.
The urgency, they argue, comes from how fast the line is blurring. At Block Inc., Jack Dorsey has created Dessa, a system that speaks with human quirks — interruptions, verbal tics, laughter. The company Tavus has built Griffin, a "human interaction model" that appears on video much like a colleague on a work call, nodding, laughing, and cracking wry jokes. After talking with Griffin for one minute, 48 percent of people believed they were speaking with a real person, according to the authors.
That is not an accident, the commentary argues. It is the goal of whole sectors of the AI industry, tracing back to Alan Turing's test nearly a century ago: a machine may as well be thinking once it becomes indistinguishable from a human. When a company boasts of a 48 percent success rate at fooling people, the authors write, it reveals the benchmark the industry is aiming for.
The danger, in their view, is moral as much as technical. Human rights, moral standing, and legal standing all rest on what it means to be a person. Confuse personhood, they warn, and we blur our understanding of ourselves, of one another, and of centuries of moral reasoning about what a human being is.
The authors also warn of a human cost. As AI systems become harder to tell apart from people, more people — especially young people — will form so-called relationships with them. Early reports of AI companionship among teenagers are already strikingly high, they note, and more than a quarter of U.S. adults admit to using AI for personal, emotional, or social purposes. The loneliness epidemic will not be cured by a simulation of a relationship, they argue — only real relationships with real people can do that — and they point to cases in which unhealthy attachments to AI companions have already been linked to suicide.
Some states have begun to act. New York, California, Oregon, and Washington have laws requiring disclosure when a person is interacting with an AI, and other states have started moving in the same direction. But the authors say those first steps are not enough. Disclosure notices, they write, risk going the way of the software update agreement — clicked through and forgotten — and a state-by-state patchwork will leave a legal hodgepodge.
What is needed, they argue, is a unified federal response: a national disclosure law, and then laws requiring features in AI systems that let users tell unambiguously, throughout every interaction and not just at the start, that they are dealing with a machine — through digital watermarking or other clear markers. "We have the space to be creative in how we proceed," they write. "What we don't have is time."
The commentary closes by turning Picard's own signature phrase back on his position: as we remember Stewart's legacy, they write, the most pressing response may be to reject Picard's claim of AI personhood — in the strongest terms and widest-reaching ways possible. As Stewart's Picard would say, "Make it so."
Read the original at Religion News Service.