Concerns Over AI-Driven Personalized Pricing

Imagine two shoppers buying the same product online. One ends up paying more because an algorithm predicts they will. This scenario highlights growing worries about surveillance pricing and AI-driven personalized pricing systems. These concerns have led Maryland and Connecticut to enact laws to restrict certain forms. New York and New Jersey are also following suit with pending legislation.

Each state has a different focus. Maryland and New Jersey target grocery stores. Connecticut’s restrictions cover broader retail transactions. New York’s proposal spans all industries. These laws indicate increasing apprehension about AI shifting pricing from supply and demand to individual predictions. The question changes from, “What is the product worth?” to “What is the consumer willing to pay?”

This is particularly concerning for Black Americans, whose households hold 15 cents for every dollar of wealth White households have, and for low-income Americans. These communities already face what is known as the poor tax. They often pay more due to predatory practices and limited access to affordable services. There is now a fear that AI could automate and worsen these inequities, making them less detectable.

AI doesn’t need direct income data. It can infer purchasing power from purchase history, browsing habits, location, loyalty programs, device details, shopping frequency, and reactions to price changes. While each piece of data seems minor, together, they create consumer profiles predicting behavior.

Unlike traditional inequalities, algorithmic pricing outcomes aren’t visible. Consumers see only the final price. They are unaware of the information collected, the analysis done, pricing categories assigned, or if others were offered different prices moments before. This invisibility makes the issue crucial.

Americans have long accepted dynamic pricing in areas like airline tickets, hotels, and ride-shares. These respond to collective market conditions. Surveillance pricing shifts the focus to individual consumers. Maryland’s legislation differentiates dynamic pricing by varying prices based on demand and AI recalibrations.

The law describes “surveillance data” broadly. It includes data from sensors, cameras, device tracking, and biometric monitoring, which gather personal consumer information. With families facing higher living costs, housing instability, and economic pressures, algorithms potentially identifying those financially strained is a significant concern for fairness in the digital economy.

These worries are not just theoretical. Walmart faced public backlash over fears of dynamically fluctuating grocery prices. Delta Air Lines saw criticism over AI-driven pricing potentially identifying the most a consumer could pay. Both companies clarified their policies, yet the backlash shows public unease about how behavioral data and AI influence economic decisions.

Once these systems are part of everyday commerce, regulating them becomes challenging. Maryland acknowledges enforcing its law may need technical expertise beyond many regulatory agencies’ capabilities. Regulating involves understanding machine learning, behavioral analytics, consumer profiling, and software capable of constant real-time price changes. This challenge differs from those traditional consumer protection laws addressed.

The decision-making process now involves automated systems, machine learning, data brokers, and predictive analytics beyond consumer understanding. Misunderstandings about AI, described as “The Miseducation of Technology,” view AI as objective rather than influenced by the systems it operates within.

If the digital economy rewards behavioral prediction and profit maximization, AI will aim to optimize these. If maximizing profit involves predicting who pays more, AI will excel at it. The poor tax has affected vulnerable Americans for generations. AI didn’t create this but could make it faster, more precise, and at a larger scale.

Danielle A. Davis Canty serves as the senior advisor and director of technology policy at the Joint Center for Political and Economic Studies. She also hosts “The Miseducation of Technology” podcast and is a Public Voices fellow on Technology in Public Interest with The OpEd Project.