Dex-chan lover
- Joined
- Nov 18, 2018
- Messages
- 12,121
Casual chatting thread
- Thread starter DavidianMillerian
- Start date
Dex-chan lover
- Joined
- Mar 17, 2019
- Messages
- 9,895
based high iq schizo. this whole data scraping subplot is full of intrigue.
Dex-chan lover
- Joined
- Jan 17, 2020
- Messages
- 1,644
Dude went out with a bang, fo sho.Sir, a second perma ban (@TheDragonMaster) just hit the forums.
Power Uploader
- Joined
- Aug 13, 2024
- Messages
- 523
We're mostly done with scraping everyone we wanted. The only people left are justforthelulz and kaarme
Dex-chan lover
- Joined
- Jul 29, 2020
- Messages
- 4,866
Forums so barren lately I half expect a tumbleweed to roll across my notifications. Mods, hear me out, slap a ‘Mexican yellow’ theme on this place so we can at least suffer with aesthetic consistency. Bonus points if it plays spaghetti western music every time someone refreshes.
Dex-chan lover
- Joined
- Nov 18, 2018
- Messages
- 12,121
At least we have the excuse that it's the weekend.
Dex-chan lover
- Joined
- Jun 22, 2020
- Messages
- 922
When the Essay Eccentric comes back and drops their Big Data, I'm putting everyone who doesn't post on weekends on a list.
Power Uploader
- Joined
- Aug 13, 2024
- Messages
- 523
Hello, I've got a message from the French guy:
"It's Ready."
"It's Ready."
Dex-chan lover
- Joined
- Jan 17, 2020
- Messages
- 1,644
So are you gonna relay the data here or do I have to reach out to him?
Power Uploader
- Joined
- Aug 13, 2024
- Messages
- 523
Only the scraping is done. The data is currently in a 300mb SQLITE3 file
Supporter
- Joined
- Apr 26, 2020
- Messages
- 3,789
I'm going to go ahead and say this, because it's been chewing on my head for a while now:
Scraping the forums is kinda creepy.
I understand posts and comments are public, and many profiles are public, and there's nothing that's illegal or against site rules about doing that. I do question whether the devs are really supportive of another set of crawlers putting strain on the servers, but given the search engines and (presumably) AI bots running around here, one more set doesn't seem like a huge anomaly. I'm sure they're aware of the behavior anyway.
However, to my mind, there's a big difference between a faceless bot doing that here, along with everywhere else it can get to on the internet, for a giant corporation's indexing or AI efforts, and a user here specifically targeting this site. The former is one of those 'unavoidable costs of using the public internet', while the latter is quite clearly one or two people with some kind of specific goal (even if that goal is just satisfying their own curiosity).
There's also a difference between 'forum lore diving' (which a number of folks like to do for amusement of one kind or another) and @BakedBanana's rumored spreadsheets, both of which are manual processes, and using automated tools to hoover the whole forum up into an indexed, searchable database. The first two are deep, targeted dives for the enlightenment or enjoyment of the user, while the last just seems stalkerish.
I'm not going to run off to the mods trying to make a complaint at this point - there have been a couple back-channel discussions with folks I've been in that have wound up in a 'let's wait and see' holding pattern. I'm also not going to ditch the site. Nor am I particularly concerned about 'what have I posted here - can I be doxxed?'. I have a fairly good idea what I've said over my time here, and if you really want to try to go to the trouble of figuring out who I am in real life, have fun with that. I'm not sure that does you much tangible benefit - I've been on the internet longer than a number of folks here have been alive, and have seen a few long cons over the years. (I also live in a place with this thing called 'the second amendment', in case that somehow influences anyone's plans.) If you just want to screw with me, take a number - I have a job that already does that.
What I am going to do is ask that someone who's involved with this shed some light on the whole thing, either here or in the dedicated thread.
Why is this being done? Is it for a school or research project of some sort? Is it simply idle curiosity? Is there some more commercial motivation behind this?
Who is involved with this? Is it just one or two users, or are there outside parties as well?
Has there been any consideration to allowing users to 'opt-out'? Is this a legal issue in some jurisdictions? Are the involved parties opening themselves up to legal action somehow?
What sort of data security measures are being taken on the part of the involved parties?
What happens to the data when the 'goal' is met, or when the involved parties lose interest?
Based on 30+ years of internet experience, I'm also going to say that, barring a good explanation or motivation, this kind of apparently obsessive-compulsive behavior frequently indicates someone is not entirely well, and would greatly benefit from reconsidering the manner in which they are using the forum (and the internet in general). I have dealt with enough people with mental conditions, both in real life and online, to know that when multiple flags start getting raised, it's time to say something. I believe I have reached that point (honestly, I think I reached it some time ago, but held my tongue in order to let some other folks work through their own opinions on the matter).
Feel free to respond here or in the dedicated thread (the latter might be better if a longer public discussion is preferred, and I considered posting this there, but given how this has been a topic of conversation recently, I opted for this thread instead). If a private conversation is preferred, my PM inbox is open; believe it or not, I actually do have a life in meatspace (particularly on the weekends), so I may not see everything right away, and I may take some time to reply if I want to consider how to do that, but I will eventually reply.
Scraping the forums is kinda creepy.
I understand posts and comments are public, and many profiles are public, and there's nothing that's illegal or against site rules about doing that. I do question whether the devs are really supportive of another set of crawlers putting strain on the servers, but given the search engines and (presumably) AI bots running around here, one more set doesn't seem like a huge anomaly. I'm sure they're aware of the behavior anyway.
However, to my mind, there's a big difference between a faceless bot doing that here, along with everywhere else it can get to on the internet, for a giant corporation's indexing or AI efforts, and a user here specifically targeting this site. The former is one of those 'unavoidable costs of using the public internet', while the latter is quite clearly one or two people with some kind of specific goal (even if that goal is just satisfying their own curiosity).
There's also a difference between 'forum lore diving' (which a number of folks like to do for amusement of one kind or another) and @BakedBanana's rumored spreadsheets, both of which are manual processes, and using automated tools to hoover the whole forum up into an indexed, searchable database. The first two are deep, targeted dives for the enlightenment or enjoyment of the user, while the last just seems stalkerish.
I'm not going to run off to the mods trying to make a complaint at this point - there have been a couple back-channel discussions with folks I've been in that have wound up in a 'let's wait and see' holding pattern. I'm also not going to ditch the site. Nor am I particularly concerned about 'what have I posted here - can I be doxxed?'. I have a fairly good idea what I've said over my time here, and if you really want to try to go to the trouble of figuring out who I am in real life, have fun with that. I'm not sure that does you much tangible benefit - I've been on the internet longer than a number of folks here have been alive, and have seen a few long cons over the years. (I also live in a place with this thing called 'the second amendment', in case that somehow influences anyone's plans.) If you just want to screw with me, take a number - I have a job that already does that.
What I am going to do is ask that someone who's involved with this shed some light on the whole thing, either here or in the dedicated thread.
Why is this being done? Is it for a school or research project of some sort? Is it simply idle curiosity? Is there some more commercial motivation behind this?
Who is involved with this? Is it just one or two users, or are there outside parties as well?
Has there been any consideration to allowing users to 'opt-out'? Is this a legal issue in some jurisdictions? Are the involved parties opening themselves up to legal action somehow?
What sort of data security measures are being taken on the part of the involved parties?
What happens to the data when the 'goal' is met, or when the involved parties lose interest?
Based on 30+ years of internet experience, I'm also going to say that, barring a good explanation or motivation, this kind of apparently obsessive-compulsive behavior frequently indicates someone is not entirely well, and would greatly benefit from reconsidering the manner in which they are using the forum (and the internet in general). I have dealt with enough people with mental conditions, both in real life and online, to know that when multiple flags start getting raised, it's time to say something. I believe I have reached that point (honestly, I think I reached it some time ago, but held my tongue in order to let some other folks work through their own opinions on the matter).
Feel free to respond here or in the dedicated thread (the latter might be better if a longer public discussion is preferred, and I considered posting this there, but given how this has been a topic of conversation recently, I opted for this thread instead). If a private conversation is preferred, my PM inbox is open; believe it or not, I actually do have a life in meatspace (particularly on the weekends), so I may not see everything right away, and I may take some time to reply if I want to consider how to do that, but I will eventually reply.
Dex-chan lover
- Joined
- Jan 17, 2020
- Messages
- 1,644
As much as I love the guy for being an eccentric fella that uses archaic English in his posts, I can't deny that all of this seems off. I've reached out to him and as far as I can tell, it seems like a hobby of him. You know, mathematics and data goes like bread and butter. I shrugged it off because, I guess, this is the first time this kind of thing happened to me on the internet, so my feelings of intrigue was overlapping my sense of caution.
Guy uses Xanax (I think?). I don't have any experience dealing with drugs and drug users but this is enough indication that he's not doing well, right? I mean, his post about Chinese magpies was morbid, ngl.
Anyways, do you mind if I copy and paste this post to him? He could be using VPN to bypass the temp-ban and view the forums as guest, but I guess it's worth a shot. I'll let you know if he gives any reply; I'll be posting it on his dedicated thread, prolly.
Supporter
- Joined
- Apr 26, 2020
- Messages
- 3,789
You have my blessing.
Dex-chan lover
- Joined
- Jan 16, 2025
- Messages
- 273
Casually drops a pin
Dex-chan lover
- Joined
- Nov 18, 2018
- Messages
- 12,121
OI, WHAT IS ALL THAT NOISE IN HERE?!
Dex-chan lover
- Joined
- Jan 17, 2020
- Messages
- 1,644
WANDERER YOU'RE SUPPOSED TO THROW THE GRENADE THROW IT NOWWWWWW
Last edited:
Dex-chan lover
- Joined
- Jul 29, 2020
- Messages
- 4,866
