Page 1 of 1

Feckin Bots

PostPosted: Fri May 08, 2020 10:04 pm
by Carleas
Sorry for any connectivity issues the past couple days, it appears a new search engine found our little corner of the web and decided to request 6 pages a second forever.

But like any good jalopy pilot, I executed a remarkable feet feat of duct tape and we should be smooth sailing now. Or at least status-quo sailing.

Carry on.
:banana-dance:

[EDIT: duct taped a word]

Re: Feckin Bots

PostPosted: Fri May 08, 2020 10:06 pm
by Magnus Anderson
Glad here this is sorted out.

Re: Feckin Bots

PostPosted: Sun May 10, 2020 5:16 am
by Oughtist
What is the ontological status of bots?

Re: Feckin Bots

PostPosted: Sun May 10, 2020 2:37 pm
by phenomenal_graffiti
But like any good jalopy pilot, I executed a remarkable feet of duct tape and we should be smooth sailing now. Or at least status-quo sailing.

Carry on.


You're a hero.

Re: Feckin Bots

PostPosted: Mon May 11, 2020 4:25 pm
by MagsJ
Ah.. is that what was slowing the site down, constantly logging me out of ILP, and throwing me off of the site at a moment’s notice?

What kind of Bot was that @ Carleas?

Re: Feckin Bots

PostPosted: Tue May 12, 2020 6:51 am
by Mr Reasonable
Mike Pence is a bot.

Re: Feckin Bots

PostPosted: Tue May 12, 2020 6:01 pm
by Carleas
MagsJ wrote:What kind of Bot was that @ Carleas?

I believe the bot belongs to Huawei, who I understand to be building a new search engine to compete with/replace Google in China, where Google is banned. As part of that, I assume they are trying to get everything about every website at speed, and turned that deluge onto ILP last week. Sadly, ILP will probably not be indexed in that search engine as a result.

The number of requests they were making doesn't seem reasonable: I set up the firewall rule on Friday and on Saturday it blocked 130k requests from the same bot, which is more than one per second, all day every day (they've mostly stopped now that they just get a 503 error). The fact that the bot was named in the user agent string and easily identifiable by a firewall rule suggests they weren't trying to do anything too nefarious, though it did seem to be ignoring the robots.txt file (which tells good bots where they can and can't go; intended for reducing load and not adding junk to search indices rather than security). It also changed IPs for every request, so the normal IP-based rules to weren't catching it, and even Cloudflare must have considered it benign because they didn't register anything amiss.

But yeah, it fucked us up something bad for a while. Sorry if it caused you lost posts or other frustrations, it definitely caused me some frustration -- enough that I actually did something about it.

Re: Feckin Bots

PostPosted: Wed May 13, 2020 4:58 pm
by MagsJ
I didn’t know that Tik Tok was Chinese, until I saw this video on Gram last week.. now it all makes sense. It has been said that Google, et others, did the same without the User’s permission, but yet that’s ok?



https://www.cnet.com/news/tiktok-accuse ... -to-china/ An interesting read..

TikTok, known for its quirky 15-second videos, has been illegally and secretly harvesting vast amounts of personally identifiable user data and sending it to China, according to a proposed class-action lawsuit filed in California federal court last week.

The lawsuit also accuses the company and its Chinese parent company ByteDance of taking user content such as draft videos without their consent and having "ambiguous" privacy policies. It raises concerns that data gathered by TikTok could be used to identify, profile and track users in the US. The company is benefiting from this alleged activity because it uses this data to sell targeted ads, the lawsuit alleges.

"TikTok's lighthearted fun comes at a heavy cost," according to the lawsuit, which was filed on Friday.