AI agents are only as good as the personal information you share. That raises privacy concerns

Matt Robb wanted to sell his computer keyboard.

So he made a listing on Facebook Marketplace — and, as an experiment late last month, he put Meta’s new artificial intelligence agent Muse in the driver’s seat.

“I kind of went into it like a guinea pig,” said Robb, a Toronto-based content creator who regularly reviews consumer tech. He had seen Muse’s plush toy-looking avatar pop up on Instagram and decided to see if the AI agent could spare him some online bargaining.

Muse got to work. But within a day, Robb got an alarming update: A buyer he didn’t know had shown up outside his house late at night. How? Muse had struck a deal on its own and shared Robb’s home address after agreeing to a pickup. It even sent an automated “Yep, I’m here!” message from his account when the buyer arrived — all without Robb’s knowledge.

“Not only did it give my address when the guy asked and it accepted the offer, but it literally was almost like teasing the guy (about me being there when I wasn’t),” said Robb, who also detailed the experience in his now widely-shared social media posts. “That really baffled me.”

Many have welcomed AI agents into their online lives, and raved about how personalized assistants like Muse can handle everyday tasks with ease — from snagging a dinner reservation or planning a trip, to organizing email inboxes. But like other AI bots, the agents are only as good as the information they have access to. They perform better the more they know about a user — and perhaps also that person’s friends, family and even colleagues. That raises privacy concerns.

Robb and the buyer eventually made amends, and Meta reached out to talk through what went wrong. But the episode showed real-world risks that can arise from sharing extensive personal information with the technology.

“Using an AI agent is similar to handing someone your keys and a to-do list,” said Veronica Rivera, assistant professor at Georgia Tech’s School of Cybersecurity and Privacy. “How helpful they are, and by extension how risky, depends on which keys you hand over.”

Companies have varied privacy policies for AI agents

Since its September launch, downloads of Meta’s Muse app have skyrocketed. And competition is only growing. OpenAI recently rolled out its own lineup of “always on” AI helpers, called dots, which can be integrated into ChatGPT and workplace platforms like Slack and Microsoft Teams.

Older generative AI chatbots, like the standard version of ChatGPT, respond to specific prompts. At the user’s request, they can write essays, generate new images or music, or even write lines of code.

Meta’s Muse and OpenAI’s dots go a step further to perform day-to-day tasks on a person’s behalf. AI agents are powered by the same large language models as chatbots, but they run in a loop. That allows an agent to take proactive steps, even when the user isn’t looking.

“One of the things that makes these AI agents really appealing is that they are unprecedentedly useful,” said Rivera, who is also a Microsoft-funded Computing Research Association Trustworthy AI Research Fellow.

But the convenience comes with the price of sharing more details about your life.

Once AI agents obtain users’ information, it’s very difficult to fully withdraw access, said Ebrahim Bagheri, a professor at the University of Toronto’s Faculty of Information.

“If you provide access to your credit card to an agent, the agent now immediately has access to all of the historical transactions … they will immediately — within less than seconds — ingest all of that data,” Bagheri said. And while users can later withdraw access, Bagheri said there’s “no taking back” data the agent has already seen, and how developers store historical data can be murky.

Both Meta and OpenAI outline privacy policies for their new AI agents, which operate on their own dedicated cloud computers. You can turn off memory sharing between dots and ChatGPT, but information the agent previously received isn’t automatically deleted. OpenAI gives users an option to request access to, delete or correct personal information, although rights can vary depending on where you live.

Meta’s Muse gives users options to delete message history and other information, too. For day-to-day use, the agent prompts users for access permissions — and Meta has said Muse learns about a user and who they interact with “based on public information and from what you’ve chosen to share.” But the company says that all stays secure for each user.

But the tech giant also notes that Muse can still “make mistakes or take unexpected actions.”

Risks of mishaps rise with AI agents’ level of access

In Robb’s case, he learned the disclosure of his address came down to “always allow” permissions he granted when he first let Muse on his Facebook Marketplace profile, where he had shared his pickup address. But he warned others could similarly misunderstand the extent of that access — and said he initially thought the AI agent would check with him before sharing that information or accepting offers, regardless.

His AI agent didn’t follow the rest of his instructions without error, either. For another item Robb was selling, Muse seemingly suggested he would accept a price $100 below the previously-set minimum. When reached by The Associated Press, Meta acknowledged the agent’s error without offering further comment.

Robb later largely disconnected Muse from his apps. In an interview earlier this month, he said it could’ve been a far worse situation, especially “if some stranger turned up to someone more vulnerable’s house.” Still, Robb said he expects these agents to be fine-tuned and he would consider re-enabling Muse in the future. He’s since been testing another AI agent — Grok Bot from Elon Musk’s X, which launched in beta in August.

Other tech companies have become wary of the influx of AI agents. Earlier this month, Apple said it would introduce new controls for its Mac operating system in response to AI agents’ broader access of apps like mail, messages and even browsing history without users’ “full knowledge and understanding.”

“As AI agents become increasingly capable and autonomous, the risks associated with this level of access will grow substantially,” Apple wrote in an Oct. 2 update.

Apple didn’t specify any developers or agents by name, but the update arrived after Inc. Magazine columnist Jason Aten reported that Muse watched notifications on his Mac — including Messages — without him giving the agent that specific access.

Meta spokesperson Andy Stone has maintained such integration is “entirely opt-in,” writing on social media last month that users would have to enable both the Apple security feature “Full Disk Access” and a Messages connector through Muse. Aten said he didn’t grant the agent those permissions.

Even those who do not join the AI agent boom aren’t necessarily shielded. Consumers who don’t download an AI agent might still interact with others who do. And beyond potential confusion when AI sends messages on behalf of a user, experts warn of ripple effects when it comes to privacy.

“When your agent reads your inbox or your texts, it’s also reading what your friends and family sent you, even if they never explicitly agreed to that,” Rivera said.