I’m currently building a personal website and making a smol-version to along with it. The main website will likely live behind Anubis to avoid LLM-scraping and the like.
Since Anubis forces the client to waste compute, this is not very aligned with the smolweb guidelines as I understand them. How do people handle this?
Anything you post on the internet should be considered public knowledge, and that includes LLMs. You might be interested in running an LLM poisoning tarpit instead. Debatable whether that’s effective either, but if it’s the principle of the thing that matters to you, that might be a viable alternative that doesn’t do anything to bother legitimate users.