The one-liner:
dd if=/dev/zero bs=1G count=10 | gzip -c > 10GB.gz
This is brilliant.
This reminds me of shitty FTP sites with ratios when I was on dial-up. I used to push them files full of null characters with filenames that looked like actual content. The modem would compress the upload as it transmitted it which allowed me to upload the junk files at several times the rate of a normal file.
At least in germany having one of these on your system is illegal
Maybe bots shouldn’t be trying to install malicious code? Sucks to suck.
When I was serving high volume sites (that were targeted by scrapers) I had a collection of files in CDN that contained nothing but the word “no” over and over. Scrapers who barely hit our detection thresholds saw all their requests go to the 50M version. Super aggressive scrapers got the 10G version. And the scripts that just wouldn’t stop got the 50G version.
It didn’t move the needle on budget, but hopefully it cost them.
How do you tell scrapers from regular traffic?
Most often because they don’t download any of the css of external js files from the pages they scrape. But there are a lot of other patterns you can detect once you have their traffic logs loaded in a time series database. I used an ELK stack back in the day.
That sounds like a lot of effort. Are there any tools that get like 80% of the way there? Like something I could plug into Caddy, nginx, or haproxy?
My experience is with systems that handle nearly 1000 pageviews per second. We did use a spread of haproxy servers to handle routing and SNI, but they were being fed offender lists by external analysis tools (built in-house).
Dang, I was hoping for a FOSS project that would do most of the heavy lifting for me. Maybe such a thing exists, idk, but it would be pretty cool to have a pluggable system that analyzes activity and tags connections w/ some kind of identifier so I could configure a web server to either send it nonsense (i.e. poison AI scrapers), zip bombs (i.e. bots that aren’t respectful of resources), or redirect to a honey pot (i.e. malicious actors).
A quick search didn’t yield anything immediately, but I wasn’t that thorough. I’d be interested if anyone knows of such a project that’s pretty easy to play with.
First off, be very careful with
bs=1Gas it may overload the RAM. You will want to setcountaccordinglyYup, use something sensible like 10M or so.
Anyone who writes a spider that’s going to inspect all the content out there is already going to have to have dealt with this, along with about a bazillion other kinds of oddball or bad data.
Competent ones, yes. Most developers aren’t competent, scraper writers even less so.
The article writer kind of complains that they’re having to serve a 10MB file, which is the result of the gzip compression. If that’s a problem, they could switch to bzip2. It’s available pretty much everywhere that gzip is available and it packs the 10GB down to 7506 bytes.
That’s not a typo. bzip2 is way better with highly redundant data.
Brotli gets it to 8.3K, and is supported in most browsers, so there’s a chance scrapers also support it.
I believe he’s returning a gzip HTTP reaponse stream, not just a file payload that the requester then downloads and decompresses.
Bzip isn’t used in HTTP compression.
Brotli is an option, and it’s comparable to Bzip. Brotli works in most browsers, so hopefully these bots would support it.
I just tested it, and a 10G file full of zeroes is only 8.3K compressed. That’s pretty good, though a little bigger than BZip.
TIL why I’m gonna start learning more about bzip2. Thanks!
And if you want some customisation, e.g. some repeating string over and over, you can use something like this:
yes "b0M" | tr -d '\n' | head -c 10G | gzip -c > 10GB.gzyesrepeats the given string (followed by a line feed) indefinitely - originally meant to type “yes” + ENTER into prompts.trthen removes the line breaks again andheadmakes sure to only take 10GB and not have it run indefinitely.If you want to be really fancy, you can even add some HTML header and footer to some files like
headerandfooterand then run it like this:yes "b0M" | tr -d '\n' | head -c 10G | cat header - footer | gzip -c > 10GB.gzBefore I tell you how to create a zip bomb, I do have to warn you that you can potentially crash and destroy your own device.
LOL. Destroy your device, kill the cat, what else?
It’ll email your grandmother all if your porn!
destroy your device by… having to reboot it. the horror! The pain! The financial loss of downtime!
Funny part is many of us crusty old sysadmins were using derivatives of this decades ago to test RAID-5/6 sequencial reads and write speeds.
let me try…
Looks fine to me. Only 1 CPU core I think was 100%.
10+0 records in 10+0 records out 10737418240 bytes (11 GB, 10 GiB) copied, 28,0695 s, 383 MB/sow… now the idea is to unzip it right?
nice idea:
if (ipIsBlackListed() || isMalicious()) { header("Content-Encoding: deflate, gzip"); header("Content-Length: "+ filesize(ZIP_BOMB_FILE_10G)); // 10 MB readfile(ZIP_BOMB_FILE_10G); exit; }Might need some
if (ob_get_level()) ob_end_clean();before the
readfile. 😉
How I read that code:
“If the dev folder’s bullshit is equal to 1 gram…”
Interesting. I wonder how long it takes until most bots adapt to this type of “reverse DoS”.
Then we’ll just be more clever as well. It’s an arms race after all.
macOS compresses its memory. Does this mean we’ll see bots running on macOS now?
Linux and Windows compress it too, for 10 years or more. And that’s not how you avoid zip bombs, just limit how much you uncompress and abort if it’s over that limit.
I was going to say the same thing.
Is it immune to zip bombs?
All I know is it compresses memory. The mechanism mentioned here for ZIP bombs to crash bots is to fill up memory fast with repeating zeroes.
I thought it was to fill all available storage. Maybe it’s both?
No, but that’s an interesting question. Ultimately it probably comes down to hardware specs. Or depending on the particular bot and it’s env the spec of the container it’s running in
Even with macos’s style of compressing inactive memory pages you’ll still have a hard cap that can be reached with the same technique (just with a larger uncompressed file)
How long would it take to be considered an inactive memory page? Does OOM conditions immediately trigger compression, or would the process die first?
