Politics

/

ArcaMax

Commentary: AI will rip off consumers unless they fight back

Cathy O'Neil, Bloomberg Opinion on

Published in Op Eds

My great uncle Fred, a former World War II prisoner living in the Cascade Mountains of Oregon, had little to his name but a four-wheel-drive truck. When the treacherous winter storms came, he would drive around, offering to help poor souls who had gotten stuck in the snow. But he was no good Samaritan: He charged exorbitant prices, sussing them out to determine the maximum he could extract. He had all the power and he knew how to use it. Call it siege pricing.

Today’s online markets, with the help of big data and artificial intelligence, are poised to become like digital doppelgängers of my mercenary great uncle — monitoring consumers, seeking weaknesses, extracting the maximum. People need to fight back while they still can.

The practice of surge pricing — charging different amounts depending on conditions — isn’t always bad. Consider raising the prices of plywood or bottled water before a storm. It discourages hoarding and scalping, as well as encouraging sellers to get supplies where they’re needed. In the context of a thriving competitive market, this can work fine even if it doesn’t always seem fair.

So when does reasonable surge pricing become exploitative siege pricing?

As a data scientist who used to work in the travel sector, I have some experience of airline pricing. The normal pattern is to make money on flexibility and perks when business travelers book (typically during the week and especially on Tuesday), and on better routes and sitting together when leisure travelers are buying (usually on the weekends, and especially a few weeks before a long weekend).

But the combination of surveillance and AI can empower sellers to go beyond general categories, sussing out exactly how much money an individual buyer is willing to part with. A person using an expensive Mac laptop, or whose zip code and purchase history suggest affluence, might see a higher price.

Last year, a Federal Trade Commission study found that intermediaries are actively offering retailers tools to achieve this. Another recent investigation found that online grocer Instacart charged significantly different prices for the same items purchased from the same stores at the same time. (Instacart said its differentials resulted from a randomized experiment that it ended.)

Retailers generally say they avoid individualized pricing, for fear of alienating customers. Yet other techniques, perhaps intended to improve the customer’s experience, can have a similar effect. Consider the way online stores such as Amazon rearrange themselves depending on who is shopping. Recently, a friend and I searched for “beanbag chair” on Amazon at the same time. Our first results offered approximately the same product, with a similarly foreign brand name, except mine cost $188 and hers $88. I had to scroll well beyond my first page to find her listing, which I wouldn’t have known existed if she hadn’t told me.

In economic parlance, individualization has great potential to reduce the “consumer surplus” — that is, the difference between what a buyer actually pays and what they’d be willing to pay if pushed to the limit. The British economist Alfred Marshall, who formalized the concept more than a century ago, touted it as a key benefit of free market capitalism: Getting something for less than you would have paid feels great. Yet in a world of personalized price discrimination, this advantage of the free market can break down. And if one seller dominates a market, or if all the sellers use similar technology, consumers have no escape.

 

In principle, price discrimination could also benefit the poor, granting them access to goods and services they otherwise couldn’t afford. Instead, it’s all too often used to exploit them at their most vulnerable moments. Suppose you desperately need cash. Typing “I need a loan fast” into a search engine — or opening an app provided by your employer — can quickly connect you with lenders eager to charge you an exorbitant interest rate.

The constantly shape-shifting world of online marketing can feel overwhelming, too pervasive to change. Don’t succumb to learned helplessness. Governments have implemented rules to shed light on the pricing of goods and services such as cars, rental apartments and cab rides. Consumers can and should demand better of their online experience, too. Some states are already considering limits on discrimination — such as on what kinds of personal information sellers can use to set prices. The overarching aim should be to establish a better balance of power between sellers and buyers.

My great uncle was a sophisticated marketer. He chose the right moment, analyzed his target and pressed his advantage to the limit. Now some of the world’s biggest companies have the surveillance systems and technology they need to all be my great uncle. Will we let them, or will we resist the siege?

_____

This column reflects the personal views of the author and does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.

Cathy O'Neil is the CEO of ORCAA, an algorithmic auditing company, and the author of "The Shame Machine: Who Profits in the New Age of Humiliation."

_____


©2026 Bloomberg L.P. Visit bloomberg.com/opinion. Distributed by Tribune Content Agency, LLC.

 

Comments

blog comments powered by Disqus

 

Related Channels

The ACLU

ACLU

By The ACLU
Amy Goodman

Amy Goodman

By Amy Goodman
Armstrong Williams

Armstrong Williams

By Armstrong Williams
Austin Bay

Austin Bay

By Austin Bay
Ben Shapiro

Ben Shapiro

By Ben Shapiro
Betsy McCaughey

Betsy McCaughey

By Betsy McCaughey
Bill Press

Bill Press

By Bill Press
Bonnie Jean Feldkamp

Bonnie Jean Feldkamp

By Bonnie Jean Feldkamp
Cal Thomas

Cal Thomas

By Cal Thomas
Clarence Page

Clarence Page

By Clarence Page
Danny Tyree

Danny Tyree

By Danny Tyree
David Harsanyi

David Harsanyi

By David Harsanyi
Debra Saunders

Debra Saunders

By Debra Saunders
Dennis Prager

Dennis Prager

By Dennis Prager
Dick Polman

Dick Polman

By Dick Polman
Erick Erickson

Erick Erickson

By Erick Erickson
Froma Harrop

Froma Harrop

By Froma Harrop
Jacob Sullum

Jacob Sullum

By Jacob Sullum
Jamie Stiehm

Jamie Stiehm

By Jamie Stiehm
Jeff Robbins

Jeff Robbins

By Jeff Robbins
Jessica Johnson

Jessica Johnson

By Jessica Johnson
Jim Hightower

Jim Hightower

By Jim Hightower
Joe Conason

Joe Conason

By Joe Conason
John Stossel

John Stossel

By John Stossel
Josh Hammer

Josh Hammer

By Josh Hammer
Judge Andrew P. Napolitano

Judge Andrew Napolitano

By Judge Andrew P. Napolitano
Laura Hollis

Laura Hollis

By Laura Hollis
Marc Munroe Dion

Marc Munroe Dion

By Marc Munroe Dion
Michael Barone

Michael Barone

By Michael Barone
Mona Charen

Mona Charen

By Mona Charen
Rachel Marsden

Rachel Marsden

By Rachel Marsden
Rich Lowry

Rich Lowry

By Rich Lowry
Robert B. Reich

Robert B. Reich

By Robert B. Reich
Ruben Navarrett Jr.

Ruben Navarrett Jr

By Ruben Navarrett Jr.
Ruth Marcus

Ruth Marcus

By Ruth Marcus
S.E. Cupp

S.E. Cupp

By S.E. Cupp
Salena Zito

Salena Zito

By Salena Zito
Star Parker

Star Parker

By Star Parker
Stephen Moore

Stephen Moore

By Stephen Moore
Susan Estrich

Susan Estrich

By Susan Estrich
Ted Rall

Ted Rall

By Ted Rall
Terence P. Jeffrey

Terence P. Jeffrey

By Terence P. Jeffrey
Tim Graham

Tim Graham

By Tim Graham
Tom Purcell

Tom Purcell

By Tom Purcell
Veronique de Rugy

Veronique de Rugy

By Veronique de Rugy
Victor Joecks

Victor Joecks

By Victor Joecks
Wayne Allyn Root

Wayne Allyn Root

By Wayne Allyn Root

Comics

Dave Whamond Bart van Leeuwen Jimmy Margulies Harley Schwadron Dave Granlund Drew Sheneman