#lemmy/#kbin has a problem that #mastodon hasn’t even attempted to solve; groups and what happens when they get popular.
#Communities, #groups, #magazines, whatever they are called are implemented as #Actors in #ActivityPub. They are basically just *very* popular users who boost a *lot*.
You can’t just distribute them across instances the way normal actors do. Whichever server hosts @technology@lemmy.ml or @technology@beehaw.org is going to get HOSED on the regular.
While it’s true that the hosts of popular communities will get more traffic, it’s actually not as bad as it first seems.
Every Lemmy instance with at least one subscriber in that popular community will act as a mirror. That means that users who are just reading posts and comments will not cause any additional load on the home-instance of the popular community, because they are consuming local copies of the posts and comments.
This will actually help scaling a lot, and is in fact exactly how many centralized platforms scale (by creating a bunch of read-only copies of content).
As long as we can distribute the Lemmy userbase between different instances (and avoid creating one or two centralized super-instances), we can take a lot of advantage of this mirroring and the scaling will be quite good!
@sunaurus what about when those users like/boost/reply?
In those cases, the action will need to propagate back to the home server (that’s where the “hosts of popular communities will get more traffic” comes from), but keep in mind - people usually read at least one or two orders of magnitude more than they write.
@sunaurus there’s a lot of upvoting happening on popular subreddits
Absolutely, but a user will only upvote a post once, while they will read it on every reload of their page. (By “read” I mean “fetch it from their local mirror”)
Hmm, you could probably extend the protocol to do eventual consistency across instances if that ever becomes a problem, remote instances could keep their own counts and only send aggregated updates.
@schizanon @technology@lemmy.ml @technology@beehaw.org Yep. I expect that long-term a lot of instances will limit the ability to create groups (as Beehaw does) or place restrictions on the size a group is allowed to grow before asking that group to move to another/their own instance or ask for finances to help with the costs.
So there is no way to horizontally scale?
The network can actually scale quite well thanks to the fact that other instances will act as mirrors of communities!
If it’s just a temporary outage, whatever the mirror has received prior to the outage will be shown to users on that other instance but only local interactions for that instance will update it, when it comes back up, things like votes and comments will be synchronized again across all of the instances.
For permanent outages, the community will just need to be started again on a new instance.
But they could pick up where the now defunct community left off, right? Like, the cached copy from another server could be imported on a new server elsewhere?
That functionality doesn’t currently exist, but migration of communities is something that’s being actively talked about for development.
Did you post this from Mastodon? I wish I could tell where this came from.
Basically if I understand this right, if you have an instance with a very popular community on it. It is likely that it will need some massive infrastructure scaling if it wants to handle the enormous amount of world wide traffic?
I wish I could tell where this came from.
Isn’t that what this colourful icon in Lemmy is for? It appears to link to the original source of the post or comment:
Yes. If you run the server, then you are the source of truth of that community. All other servers that federate your community query your server to access the community and show it to their users.
So if you run a server and a community explodes there, you might only have 500 users on your instance, but you might have 50k users reading that community and interacting with it from other Lemmy instances, thus your server needs to scale to 50k users worth.
And ever more essential, your server is the source of truth of that community. So if your server is hacked or corrupted or deleted, that community is gone. Other instances don’t mirror it (except for temporary caching), so the Lemmy network essentially is a trust network of other people maintaining servers long term (and each inventing a monetary system to pay for it). I still think the network might be better than a centralized system like reddit, but it definitely has a lot of growing and policies that need to be sorted out very soon
So are these other servers just routing requests from their users to your server’s community? Or are they actually copying everything over every so often (caching) and serving up the requests themselves? How real time is it, I guess is what I’m asking?
