FaceDeer

It costs so much to make an AAA game these days that it must earn an enormous amount of money to be profitable, which means it needs to appeal to as broad a market as possible, which means nothing niche or unusual. I think movies are having the same problem.

Perhaps be more succinct? You’re really flooding the zone here.

You have tunnel vision on this issue.

No, I’m staying focused.

That is absolutely ridiculous. The pressure AI scraping puts on sites vastly outstrips anything people built for, as evidenced by the fact that the systems are going down.

Yes. Which is why I’m suggesting providing an approach that doesn’t require scraping the site.

It’s ironic that you’re railing against capitalism while espousing exactly the sort of scarcity mindset that capitalism is rooted in, whereas I’m the one taking the “information wants to be free” attitude that would normally be associated with anti-capitalist mindsets.

Do you know how excited I was when LLM tech was announced? Do you know how much it sucked to realize, so soon, that companies were going to do their best to use it to optimize profits?

They do that with everything. Does that mean that everything must therefore become some kind of all-or-nothing battleground wherein companies must be thwarted?

It’s not as simple as, “Oh, you say that you believe in freedom of information, but curious how you don’t want private companies to use it to make money at your expense! Guess you’re a hypocrite.”

Emphasis added. That part is where you’re in error about my view, it’s not at my expense. It doesn’t harm me any.

Tell me what you actually believe, or stop cycling back to this like it’s a damning rebuttal.

I have been.

I’m not “taking their side.” I’m just not actively trying to harm them. The world is not a zero-sum game, it’s often possible for everyone to get what they want without harming each other in the process.

Yes, I know the companies are not the same as normal patrons. I don’t care that they’re not the same as normal patrons. All I’m concerned about is that the normal patrons get access to the data. The solution I proposed does that.

The problem, as I see it, is that’s not all that you are concerned about. Your goal also includes a second aspect; you want those companies to not have access to that data. So my proposal is not acceptable because it doesn’t thwart those companies.

I’m not drawing an equivalence between companies and individual patrons, I’m just saying my goals don’t include actively obstructing those companies. If they can get what they want without interfering with what the normal patrons want, why is that a bad thing?

Bandwidth can’t, though.

Bandwidth is incredibly cheap. The problem these sites are having is not running into bandwidth limits, it’s that providing the pages requires processing to generate them. That’s why Wikipedia’s solution works - they offer all the “raw” data in a single big archive, which takes just as much bandwidth to download but way fewer server resources to process (because there’s literally no processing - it’s just a big blob of data).

Is it okay to hire a bunch of people to check out half a library’s books, then rent them to people for money?

This analogy fails because, as I said, data can be duplicated easily. Making a copy of the data doesn’t obstruct other people from also viewing the data provided you avoid the sorts of resource bottlenecks I described above.

Is your problem really about the accessibility of this data? Or is it that you just don’t want those awful for-profit companies you hate to have access to it? I really get the impression that that’s the real problem here - people hate AI companies, and so a solution that gives everyone what they want is unacceptable because the AI companies are included in “everyone.”

I don’t understand why the burden is on the victims here.

They put the website up. Load balancing, rate limiting, and such go with the turf. It’s their responsibility to make the site easy to use and hard to break. Putting up an archive of the content that the scrapers want is an easy and straightforward thing to do to accomplish this goal.

I think what’s really going on here is that your concern isn’t about ensuring that the site is up, and it’s certainly not about ensuring that the data it’s providing is readily available. It’s that there are these specific companies you don’t like and you just want to forbid them from accessing otherwise freely accessible data.

Unlike water, though, data can be duplicated easily.

That suggestion is exactly the same as what I started with when I said “IMO the ideal solution would be the one Wikimedia uses, which is to make the information available in an easily-downloadable archive file.” It just cuts out the Aaron-Schwarts-style external middleman, so it’s easier and more efficient to create the downloadable data.

If someone did an Aaron-Schwartz-style scrape, then published the data they scraped in a downloadable archive so that AI trainers could download it and use it, would you find that objectionable?

so every single repository should have to spend their time, energy, and resources on accommodating a bunch of venture funded companies that want to get all of this shit for free without contributing to these repositories at all themselves?

Was Aaron Schwartz wrong to scrape those repositories? He shouldn’t have been accessing all those publicly-funded academic works? Making it easier for him to access that stuff would have been “capitulating to hackers?”

I think the problem here is that you don’t actually believe that information should be free. You want to decide who and what gets to use that “publicly-funded academic work”, and you have decided that some particular uses are allowable and others are not. Who made you that gatekeeper, though?

I think it’s reasonable that information that’s freely posted for public viewing should be freely viewable. As in anyone can view it. If they want to view all of it and that puts a load on the servers providing it, but there’s an alternate way of providing it that doesn’t put that load on the servers, what’s wrong with doing that? It solves everyones’ problems.

Even more ironically, you could probably shorten that time even more by having an AI analyze the transcript for you.

I’ve found Firefox’s Orbit extension to be quite handy whenever someone directs me to a 30-minute Youtube video as “proving” whatever point they’re trying to argue. I can pop it open and ask it to tell me what the video says about that point in just a few seconds. I wouldn’t use the AI summary as backing if I was doing surgery on someone, but for a random Internet argument it’s fine.

Also the article’s content doesn’t say what the headline says.

On the plus side, at least on the instance I’m on I was automatically given a link to when this same story was posted here three months ago. Saves some effort.

This seems contradictory. On the one hand you’re saying that these works are wrongly locked behind paywalls, but on the other you’re saying that scraping them is an “assault on the cornerstones of our public knowledge.” Is this information supposed to be freely viewable or not?

IMO the ideal solution would be the one Wikimedia uses, which is to make the information available in an easily-downloadable archive file. That lets anyone who wants the whole thing to have it without having to “hammer” the servers. Meanwhile the servers can be protected by standard load-balancing and DDOS prevention systems.

There are already gaps between the boards. You think the neighbour is a peeper who is unsatisfied with the view between the boards and so has installed a great big obvious “I’m peeping!” Hatch?

And I remember them disabling the features that extension required to function.

Orbit was supposed to be open-sourced when it left beta. Wonder if we’ll ever see that. I’d be happy to be able to hook Orbit up to a local LLM.

You may know IPv6 is ridiculously bigger, but you don’t know it.

There are enough IPv6 addresses that you could give 10^17 addresses to every square millimeter of Earth’s surface. Or 5×10^28 addresses for every living human being. On a more cosmic scale, you could issue 4×10^15 addresses to every star in the observable universe.

We’re not going to run out by giving them to lightbulbs.