Especially with the rush from Twitter, there are more and more bots and crawlers coming up from the Fediverse that just can’t manage to send a correct UserAgent.
Many apps, which want to make usage of several platforms, just submit the framework they use for their connection. Like python/1.2.3, Dart/1.23 or http.rb/1.2.3.
The biggest problem with this behavior is, that I as the operator and administrator want to secure the system, but I would lock out legitimate software by targeting only the UserAgent. Especially the python/1.2.3 in particular, but has also attracted malicious attention many times in my log files.
I’m really grateful, that Lemmy developer decided to use a customized UserAgent, so that I as admin can directly see what request it is and where it comes from. That makes it so much easier to make the decision of harm or no harm.
On my servers, I started to block all requests with a “default” or empty UserAgent, but I wonder, how it’s possible to fix this issue in general? Any ideas?