StuartLemmy
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
fattyfoods@feddit.nl to Open Source@lemmy.ml · 21 hours ago

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

external-link
message-square
83
fedilink
  • cross-posted to:
  • [email protected]
440
external-link

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

fattyfoods@feddit.nl to Open Source@lemmy.ml · 21 hours ago
message-square
83
fedilink
  • cross-posted to:
  • [email protected]
Anubis, which block AI scrapers from scraping websites to death, has been downloaded almost 200,000 times.
  • bdonvr@thelemmy.club
    link
    fedilink
    arrow-up
    13
    ·
    16 hours ago

    Thanks. Anything special configuring it?

    • Captain Beyond@linkage.ds8.zone
      link
      fedilink
      arrow-up
      13
      ·
      edit-2
      16 hours ago

      I keep my server config in a public git repo, but I don’t think you have to do anything really special to make it work with lemmy. Since I use Traefik I followed the guide for setting up Anubis with Traefik.

      I don’t expect to run into issues as Anubis specifically looks for user-agent strings that appear like human users (i.e. they contain the word “Mozilla” as most graphical web browsers do) any request clearly coming from a bot that identifies itself is left alone, and lemmy identifies itself as “Lemmy/{version} +{hostname}” in requests.

Open Source@lemmy.ml

opensource@lemmy.ml

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

  • Open Source Initiative
  • Free Software Foundation
  • Electronic Frontier Foundation
  • Software Freedom Conservancy
  • It’s FOSS
  • Android FOSS Apps Megathread

Rules

  • Posts must be relevant to the open source ideology
  • No NSFW content
  • No hate speech, bigotry, etc

Related Communities

  • [email protected]
  • [email protected]
  • [email protected]
  • [email protected]
  • [email protected]

Community icon from opensource.org, but we are not affiliated with them.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 759 users / day
  • 1.45K users / week
  • 3.23K users / month
  • 9.15K users / 6 months
  • 1 local subscriber
  • 38.7K subscribers
  • 605 Posts
  • 5.04K Comments
  • Modlog
  • mods:
  • Evan@lemmy.ml
  • kevincox@lemmy.ml
  • CrypticCoffee@lemmy.ml
  • Lettuce eat lettuce@lemmy.ml
  • BE: 0.19.9
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org