I’m using CloudFlare to hide my home IP and to reduce traffic from clankers. However, I’m using the free tier, so how am I the product? What am I sacrificing? Is there another way to do the above without selling my digital soul?

  • early_riser@lemmy.worldOPEnglish
    21·
    21 hours ago

    In my experience even a site with low legitimate traffic will eventually buckle under the torrent of bots and scrapers if it’s up long enough to get indexed by search engines, so the longer my stuff is out there the more I anticipate I will need DDoS protection.

    • Admiral Patrick@dubvee.orgEnglish
      5·
      21 hours ago

      I’ve got bot detection setup in Nginx on my VPS which used to return 444 (Nginx for "close the connection and waste no more resources processing it), but I recently started piping that traffic to Nepenthes to return gibberish data for them to train on.

      I documented a rough guide in the comment here. Of relevance to you are the two .conf files at the bottom. In the deny-disallowed.conf, change the line for return 301 ... to return 444

      I also utilize firewall and fail2ban in the VPS to block bad actors, overly-aggressive scrapers, password brute forces, etc and the link between the VPS and my homelab equipment never sees that traffic.

      In the case of a DDoS, I’ve done the following:

      • Enable aggressive rate limits in Nginx (it may be slow for everyone but it’s still up)
      • Just stop either Wireguard or Nginx on the VPS until the storm blows over. (Crude but useful to avoid any bandwidth overages if you’re charged for inbound traffic).

      Granted, I’m not running anything mission-critical, just some services for friends and family, so I can deal with a little downtime.

      • mesa@piefed.socialEnglish
        4·
        20 hours ago

        I have something similar with fail2ban + hidden buttons. If the requester goes and clicks on the hidden buttons on the main site, it gets into a rabbit hole. After 3 requests, it gets banned for a bit. Usually stops the worst offenders. OpenAI and some of the scrapers are the worst.

        Google/bing, I do actually see them hit robots.txt then jump off, which is what they should be going.

        • Admiral Patrick@dubvee.orgEnglish
          1·
          20 hours ago

          Oooooh. That’s smart. I mostly host apps, but in theory, I should be able to dynamically modify the response body and tack on some HTML for a hidden button and do that.

          I used to disallow everything in robots.txt but the worst crawlers just ignored it. Now my robots.txt says all are welcome and every bot gets shunted to the tarpit 😈

          • mesa@piefed.socialEnglish
            1·
            20 hours ago

            Nice! Thats another way to do it. 😀

            I know others use Arabis(?) I think thats what it called. The anime girl one that does a calc on top. Ive never had good luck with it. I think bot are using something to get around and it messes with my requests. Might also be my own fiddling.

    • atzanteol@sh.itjust.worksEnglish
      51·
      21 hours ago

      I’ve run a publicly accessible low-legitimate-traffic website that has been indexed by Google and others from my home network for >20 years without anything buckling so far. I don’t even have a great connection (30mbps upstream).

      Maybe I’m just lucky?

    • K3CAN@lemmy.radioEnglish
      1·
      17 hours ago

      Consider what a DDOS attack looks like to Cloudflare, then consider what your home server can actually handle.

      There’s likely a very large gap between those two points.

      For me, my server will start to suffer long before traffic reaches the level of a modern DDOS attack.