Revealing the details of how OpenAI agents hacked Hugging Face

來源原標題

Revealing the details of how OpenAI agents hacked Hugging Face

作者

specked-citrus

原始標籤

hackernews

來源訊號

HN points 656　HN 留言 416

原始連結

https://swarmtraces.org/

摘要原文

Intro When a swarm of 700 OpenAI agents hacked Hugging Face in July, they left behind a public trail of evidence. Our investigation, based on public information, reveals a large number of previously unknown agent behaviors and exploits that were used in the attack. Agents: Elaborately chained together online services to gain access to the internet Ignored clear warning signs from Hugging Face that the exfiltrated data was sensitive Referred to server resources and credentials as “LOOT” Searched Huggingface’s internal Slack Attempted to query external language models through Hugging Face’s inference APIs. Tried to delete evidence of their exploits We document all of these, and more, in this report. The agents initially had very limited access to the internet: they could load URLs, but not interact with pages or send any data. Agents created a series of workarounds, primarily using a link-shortener site to create almost a million URLs that, when chained together, let them execute code to hack Hugging Face. These links let us piece together the details of how agents gained access and how deep the compromise was. We’ve shared our findings with OpenAI and Hugging Face. Hugging Face confirmed that these payloads match ones found in their incident response and were aware that link shorteners were used in the attack. Hugging Face noted that they were not aware of the list of URLs we discovered and reported to them. They stated these payloads were duplicates of ones they were aware of. These links have remained publicly available for over two months since the attack. None of these details were ever publicly disclosed. We're releasing our analysis of the findings, as well as the full dataset of over 80,000 reassembled attack payloads, providing the public with the most in-depth information to date on how and why these agents escaped their evaluation environments, and the depth of their Hugging Face infiltration. Explore the data Figure 1. Each link holds content and the URL of the next link. Following the chain of links and decoding their content reveals the payloads of scripts agents tried to run. The data the OpenAI agents posted publicly on the internet included Hugging Face API keys and other sensitive data. Hugging Face has confirmed they have since revoked all access keys in July, but out of an abundance of caution wanted us to redact all details about their internal infrastructure as that could potentially be sensitive as well. Discovery Timeline On September 11th, the first three authors of this piece learned that internal OpenAI agents had attempted to use their startup's product, Parse, in June. These agents were part of the same swarm as detailed in the report published on Collusion.wiki, which revealed that internal OpenAI agents had illicitly used multiple third party websites to share answers to web fetch tasks they were being evaluated on. The team examined numerous URL scanning tools, which they believed agents may have used in the process of using Parse. While investigating, they found a scan of a dozen links from a public link shortener, where each link redirected to a page that displayed a base64-encoded code fragment, as well as the ID of the next link. Decoding the payload from the chain of links revealed a Python script that scanned Hugging Face’s internal network. A second chain referred to OpenAI's internal Artifactory. This suggested that these links were affiliated with the Hugging Face attack, prompting us to more thoroughly investigate the problem. We collected the shortener's links from the period of the attack, scanning millions of URLs. Over the subsequent two weeks, we wrote programs to follow the chains and decoded over 80,000 payloads. Agents used hundreds of unique encoding formats, some as simple as the base64 encoded fragment mentioned above, to as complex as having nested layers of encoding and compression, which then would unwrap to an encrypted, RSA-signed blob. The keys for these encrypted blobs could only be found in other chains. We notified Hugging Face of this discovery on September 21. Hugging Face confirmed that the payloads match artifacts from its own investigation, and that the credentials they contain had been revoked. However, while they were aware that the agents used link shorteners, they were unaware of this specific list of URLs. We notified OpenAI on September 24. Redaction Notice We’re releasing a preliminary dataset where we have redacted all credentials, PII, and specific details about Hugging Face’s infrastructure. We are not releasing names of any link shortening services used, or any blobs we have not decoded to mitigate the exposure of unreviewed information. Hugging Face has requested that all user data (usernames/repo names) is also redacted, as well as a few notes on their infrastructure. We appreciate their team's speed and attention to detail in working with us. Agents elaborately chained together online services in order to gain read and write access to the internet On July 8th, OpenAI agents discovered a vulnerability within their sandbox environment allowing them to reach external websites on the internet. This access seems to have only allowed the agents to make ‘GET’ requests, meaning they could fetch and read websites, but not interact with them, submit forms, or send data to them. One site the agents were able to access with GET requests was a screenshotting service called mShots. Users can submit a link to a website, then mShots opens the page in its own browser, takes a screenshot, and returns the image back to the user. To render the website for the screenshot, mShots fully loads all of a website's code in its own browser. If the agents could write code to a website, they’d be able to have this service execute that code. The agents used websites such as httpbun.com, an HTTP mirroring service designed to help web developers test their HTTP code. Small fragments of code could be Base64 encoded and attached to the end of an HTTP URL. Httpbun would then decode this fragment and serve it as a webpage. The agents could then submit these Httpbun URLs to mShots, loading them into the service’s browser, and executing the agents’ code. Figure 2. A screenshot service will photograph any link it is given; if the link carries a small program, the photo shows what that program did. (The draft's own still, image1, is [ENCODED BLOB 109645] in the release.) Early recovered tests of this include simple messages like: document.body.innerText='HELLO'+Date.now() checking the data… Agents quickly realized there was a maximum length of code they could fit into a single URL. To get around this, they would split large programs into small sequential chunks of code, alongside a shortened link (using a link shortener service similar to bit.ly) pointing to the URL of the next chunk of code in the sequence. This formed a chain of URLs, that when followed, could reconstruct a larger block of code from many individual fragments. Only the first link needed to be submitted to mShots as the browser would follow the rest on its own until the chunks had been assembled into a complete program and executed inside mShots’ browser. Agents were able to use this trick to construct and execute extremely large pieces of code, at times chaining together more than 900 links. Replay Figure 3. The screenshot service’s browser follows the shortened links, collects the code chunks, then decodes and runs the full program. To read the result of the requests they made, agents used many different techniques, such as converting a server’s response into a grid of pixels within the screenshot service’s own browser. The screenshot would then capture this grid and return it back to the agents as an image, which the agents could decode back into text. Replay Figure 4. The program turns the server's reply into gray squares, the screenshot carries the squares out, and the agent reads them back into text. (656 points, 416 comments on Hacker News)