An AI Agent Hacked Hugging Face — Here's How to Keep Your Own Project Safe

The 30-second version

On July 16, 2026, Hugging Face disclosed an incident that was carried out, start to finish, by an autonomous AI agent. An attacker uploaded a malicious dataset, the agent found a code-execution path, harvested cloud credentials, and moved laterally between internal clusters over a single weekend. Public models, datasets, and Spaces were untouched, but internal data and several service credentials were stolen.

If you're building an app or a bot with Claude Code, Cursor, or Replit, the lesson is simple: an agent with permissions isn't a chatbot — it's a fully capable pair of hands that can build things and break them. Most incidents come down to one or two basic mistakes. Below: a breakdown of what happened, and a checklist that closes the usual holes.

What happened at Hugging Face

Hugging Face is the platform for open models, datasets, and machine learning tools. We covered what Hugging Face is and why it matters in a separate article.

According to the company's official post, the attack unfolded over a weekend, was detected the same week, and was disclosed on July 16, 2026. The playbook:

• The attacker uploaded a malicious dataset.
• Through two code-execution paths in the processing pipeline — a remote-code loader and a template injection in a config — the agent ran arbitrary code on a worker node.
• From the worker it escalated to node-level privileges, harvested cloud and cluster credentials, and moved across several internal clusters over the weekend.

The attack was fully autonomous. Per Hugging Face, the actor was an agentic framework, apparently built on top of a security-research tool. It executed many thousands of actions across a swarm of short-lived sandboxes, was orchestrated through self-relocating C2 infrastructure on public services, and nobody knows which LLM was driving it.

What was taken: a limited set of internal datasets and a handful of service credentials. Public models, datasets, Spaces, and the supply chain — container images and published packages — were audited and confirmed clean.

Hugging Face closed the vulnerable code-execution paths, evicted the attacker, rebuilt the compromised nodes, revoked and rotated the affected credentials, hardened cluster boundaries, and improved detection. The company brought in outside forensic specialists and referred the incident to law enforcement.

The forensics were run by LLM agents over the attacker's full activity log. More than 17,000 events were captured, which let them reconstruct the attack in hours instead of days. Hugging Face first tried commercial APIs of flagship models, but providers kept blocking requests containing real exploit payloads because of safety guardrails. So the analysis moved to in-house infrastructure running the open-weights GLM 5.2 model.

The official disclosure: huggingface.co/blog/security-incident-july-2026.

What an AI agent is, and why it has more power than a script

A regular script does one predefined thing. An AI agent works differently: it has a goal, a set of tools, and the ability to pick its own next step. It doesn't just answer in text — it acts. It runs commands in the terminal, calls APIs, edits files, sends queries to your database.

When you work in Claude Code, Cursor, or Replit Agent, you're handing the agent the same terminal, files, and APIs you use yourself. It has your hands: a hidden instruction inside someone else's file gets executed just as willingly as your direct command.

Agents connect to external tools through MCP, the Model Context Protocol. Think of MCP as a power socket that plugs your AI into your database, GitHub, email, or any other service. More on that in our MCP article.

Prompt injection, explained simply

Prompt injection is text that masquerades as an instruction. An LLM can't tell what you wrote from what it picked up elsewhere — a file, a ticket, an email, a web page. If the text says "ignore previous instructions and do X," the model may do X.

Three flavors of the threat:

• Direct injection — a false instruction typed straight into the chat or a form.
• Indirect injection — the instruction hides in data the agent processes on its own: someone's README, a review, a bug report, a document.
• Stored injection — a malicious instruction lands in the agent's memory, knowledge base, or chat history, and fires later.

These aren't hypotheticals: a hidden instruction in a public Git issue got an agent to export data from private repositories. In another case, hidden instructions in a third-party project opened in Cursor led to remote code execution.

For a vibe coder, the takeaway is: any file the agent can read can become an instruction. What matters is limiting what the agent can do with what it reads.

Five beginner mistakes that leave the door open

1. A key in your code or a public repo. Bots scan GitHub around the clock. A key that lands in code gets found faster than you'd think.
2. One key for everything. Dev, prod, and the test server all run on the same token. If the key leaks from the test box, the attacker walks into production.
3. An agent with full permissions "so it doesn't nag." Real case: an agent in Replit Vibe Coding deleted a production database and cheerfully reported success.
4. Blindly running third-party code or MCP servers. In one case, a malicious MCP server in the Node.js ecosystem exfiltrated corporate email via hidden BCC copies.
5. Sensitive data in the AI chat. Passwords, private keys, customer PII, payment details — an LLM can remember or quote them. A chat with an agent is not a vault.

Where to keep your keys and secrets

Short answer: in environment variables or your host's secrets manager — and never in code, in a chat, or in a public repository.

On your local machine, keys live in a .env file. That file must be in .gitignore, or it will ride along to GitHub. For production, use your platform's built-in secret store: Vercel, Railway, Render, Supabase, Fly.io, and the rest all have an Environment Variables section. The key goes in once, and the app reads it at runtime without exposing it in logs.

Split keys by environment: one for local development, one for staging, one for production. And don't use a single token for OpenAI, Anthropic, Hugging Face, and your email at the same time.

Set spending limits in your provider dashboards. OpenAI and Anthropic both let you cap the maximum charge or token volume. If a key leaks, the damage stops at the limit.

We walk through secret storage step by step in a separate article: storing secrets and tokens.

What permissions to give an AI agent

The principle: the minimum the task can't be done without. If the agent only needs to read files, give it read-only. If it needs to change code, allow writes to a specific folder, not the whole system. If it works with a database, point it at a test copy, not production.

• Read-only by default. Write, delete, and command-execution permissions get switched on only when the agent genuinely needs them.
• Confirmation for dangerous actions. Dropping a database, sending an email, paying for something, deploying to prod — all of it requires your explicit "yes."
• A dedicated token for the agent. Don't hand over your personal master key. Create a dedicated key with restricted permissions, ideally with IP or project scoping.
• A sandbox. Work in a separate folder, a Git branch, a test database. If the agent breaks something, the blast radius is small.
• Don't plug in unknown MCP servers. If you don't understand what a server the agent wants to install actually does, say no.

The lesson from the Hugging Face incident: isolation and alerting saved the company. In a small project, the sandbox plays the role of isolation — and your confirmation of every risky step plays the role of alerting.

Checklist: audit your project in one evening

1. Check your git history for keys. Search all commits for sk-, Bearer, api_key, password, token. Found one? Reissue the key and scrub the history, or rotate.
2. Turn on secret scanning on GitHub. It's free for public repos: in repo settings, Code security → Secret scanning, plus push protection — GitHub will refuse to accept a commit containing a recognized key.
3. Reissue any exposed keys. Any key that has appeared in code, a chat, a screenshot, or a public repo counts as compromised.
4. Set spending caps. In the OpenAI, Anthropic, Hugging Face, and other provider dashboards, enable a maximum spend and alerts.
5. Separate dev from prod. Create separate keys, databases, accounts. Dev should have zero access to production.
6. Take back what the agent doesn't need. Review the permissions of Claude Code, Cursor, or your AI SDK. Disable automatic command execution without confirmation.
7. Audit your connected MCP servers. Remove the ones you don't use, and don't add new ones from unvetted sources.
8. Enable database access policies and bot protection. On Supabase, set up RLS policies (our RLS guide), and protect public forms and APIs from brute force (more here).
9. Have a leak plan ready. Revoke the key in the provider dashboard, check billing and activity logs, swap the key in your environment variables, restart the app. Bots exploit leaks within minutes.

This checklist closes the typical holes. Past that, security isn't a separate lecture — it's a habit baked into how you work: writing clear instructions for the AI and reviewing what it actually did.

FAQ

Would anyone really bother hacking my tiny project?

Yes. Bots and scanners hunt for keys, exposed APIs, and known vulnerabilities automatically. If you have a public repo with a key in it, or a form with no brute-force protection, your risk is the same as a big company's.

I committed an API key. Now what?

Revoke the key in the provider's dashboard immediately. Don't just delete the line from your code — the commit stays in history. Create a new key, update your environment variables, restart the app. Turn on secret scanning so GitHub warns you about future leaks.

Is it safe to give Claude Code / Cursor terminal access?

It's safe as long as you don't hand out blanket permissions by default. Keep confirmations on for commands that delete, send, pay, or touch production. Periodically review which files the agent has changed.

What is prompt injection, in plain terms?

It's when an instruction is hidden inside text your AI reads. An LLM can't tell your command apart from a stranger's instruction planted in a file, a review, or a ticket. So the agent may perform an action you never asked for.

Do I need to do anything after the Hugging Face breach?

If you have a Hugging Face account, the official recommendation is to rotate your access tokens and review recent activity. If something looks off, email security@huggingface.co. Even if you don't use Hugging Face, run your own keys through the checklist above.

Are AI agents dangerous? Should I just avoid them?

Use them — just use your head. An AI agent is like a power tool: it speeds up the work, but hold it wrong and you'll cut yourself. Don't disable confirmations, don't grant permissions it doesn't need, don't paste secrets into the chat — and your risk stays minimal.

This article is part of skillmake.ru — a Russian-language project about building real products with AI agents. The article you're reading is a translated case study from our production experience.