Technical solution - a distributed server

[Winterflaw]

Well non-time sensitive things like loot and drops can be high latency actions that all other nodes can agree on first. However, issues in agreeing on damage players took or damage monsters took is significantly more difficult because high latency there is unacceptable PLUS the game world for every other players isn't going to look the same for everyone. Plus you'll need to implement a physics/world rewind mechanic so that when people receive this information they can TRY, emphasis on the try, to rewind to the state the world was seen in by the sender. Since latency between request and receiving on other nodes will be anywhere between 200-1000ms it's going to be difficult or impossible to deal with.

If consensus is too slow, then you need to have a master, but to have a master in a distributed system, that means the master can change at any time. So you need a master plus a couple of slaves, each of which keeps tightly synced with the master and can immediately take over (all clients with connections to the master for a given object must also have connections to the slaves).

One thing to keep in mind - players cannot affect the game world more than about 50 yards away from themeselves. World PvP is the worst case scenario, for player density in a small area.

However, if we imagine a master server PER OBJECT, or something like that, then that scenario I think doesn't really matter - what you actually care about instead are situations where a single player has a very large number of objects nearby, since he'll need a master server connection for each one.

So what you can try to do is keep physically proximate objects on the same server, for latency - but then you need a more capable server, if there are a lot of objects. However, it might be possible to have consensus ina timely manner between a small number of servers, and if so, you could distribute the load then over a set of master servers.

Blah blah etc talking off the top of my head - I've only ever looked at the server source code to figure out how it was rendering the OpenGL for a scene =-)

[Winterflaw]

Things will desync as you have issues with floating point precision, non-deterministic pathing and other things like that. Who is correct in this scenario if you try to resync too?

Another thought - a lot of actions in-game have a lot of warning that they're going to happen.

Spells which take three seconds to cast. Debuffs which run out in 30 seconds.

Only a subet of actions are immediate - player movement, instant cast spells.

Maybe we could imagine partitioning the game map up into zones, which vary in size, one per client connected to the game - like mobile phone zones. As a player reaches the edge of a zone, his client has already connected to the neighbouring zones for handover.

[Winterflaw]

I've also been thinking of various ways to overcome this, but another big issue is that there is always going to be at least one weak link in the chain who is going to abuse the system and corrupt the data in some way to try and get themselves better gear than he/she deserves.

Well, one thing at a time.

First, see if the basic idea is possible.

I can offhand imagine for example ensuring any objects owned by a player are never stored on his machine.

[Glader]

Things will desync as you have issues with floating point precision, non-deterministic pathing and other things like that. Who is correct in this scenario if you try to resync too?

Another thought - a lot of actions in-game have a lot of warning that they're going to happen.

Spells which take three seconds to cast. Debuffs which run out in 30 seconds.

Only a subet of actions are immediate - player movement, instant cast spells.

Maybe we could imagine partitioning the game map up into zones, which vary in size, one per client connected to the game - like mobile phone zones. As a player reaches the edge of a zone, his client has already connected to the neighbouring zones for handover.

Hmmm, I would think partitioning up the various services the serverside provides more than just zone server would be a better idea. Since many players computer are incredibly low-grade they won't be able to participate in the system that demands they handle zone logic like: AI, quest events, Enemy pathing and computing interest for players/actors. That's too much for many peoples computers imo.

However, many people could participate in the system if they could elect to fulfill service specific roles such as: chat, quest requirements verification, agreeing on trades, agreeing on loot dropped from a creature, agreeing on players moving from worldserver to instances and etc.

I think those services would require considerably less computation than zone servers. Surely zone servers NEED to be run but who can? Who could possibly handle Sithilus during AQ?

I think we'll need to significantly refactor Nostarlius' codebase, when released, to seperate many of these services into a more modular form. At that point they could then be run authoritatively or we could build some sort of p2p layer under it for users to run.

Seems like a considerable amount of work, especially from my perspective as I don't have WoW emu experience nor reverse engineering experience, and may be highly likely to fail as a project.

I guess an attempt couldn't hurt. For awhile now I've wanted to rewrite some of the open-source WoW projects in a way that allows for better scaling and load-balancing. Guess this would just be another good reason to try.

[wowmad]

.

[Winterflaw]

Hmmm, I would think partitioning up the various services the serverside provides more than just zone server would be a better idea. Since many players computer are incredibly low-grade they won't be able to participate in the system that demands they handle zone logic like: AI, quest events, Enemy pathing and computing interest for players/actors. That's too much for many peoples computers imo.

Oooof. I profoundly disagree. The computational demand of those services is almost zero. Computers today are extraordinarily powerful and WoW was released back in 2004 - twelve years ago.

The main issue I see is network bandwidth.

However, many people could participate in the system if they could elect to fulfill service specific roles such: chat, quest requirements verification, agreeing on trades, agreeing on loot dropped from a creature, agreeing on players moving from worldserver to instances and etc.

It's harder to design a reliable system when hosts are asymetric.

One thought though is obviously to avoid a big-bang approach. So, instead, start with the existing central server, and then work to decentralize one function - like chat. Getting that going would be a huge step, since it would require addressing the majority of the changes which have to be made, but it would be in a non-vital system which would permit then the initial creation of the client (server) cloud, and allow it to be run in and made stable and viable. No one likes big-bang, in fact, you only do it if you're an idiot (hello all Government IT projects everywhere or if your hand is forced by circumstannce.

I think those services would require considerably less computation than zone servers. Surely zone servers NEED to be run but who can? Who could possibly handle Sithilus during AQ?

Mmm - there's no need for one server to handle an entire zone. The size of the zone handled by each computer should vary, and must vary in fact as the number of clients in the system varies.

I think perhaps your original comment I replied to above is based on the idea a client (server) would handle an entire zone. This was not the proposal - rather, each client would handle a small part of the world.

Seems like a considerable of work, especially from my perspective as I don't have WoW emu experience nor reverse engineering experience, and may be highly likely to fail as a project.

Mangos is sadly written in C++, i.e. the language of the beached whale.

[Winterflaw]

Winterflaw,

As a suggestion, I ask if you (or anyone) can keep updated a resume with the main ideas on initial post, that we should have an opinion (please also indicate a smile like a status). This way, we can focus to debate only the things that matters.

For example:

1. Idea A (New proposal)
2. Idea B (Resused)
3. Idea C (To debate)
4. Idea D (Accepted as OK)
5. Idea E (Possible, but won't work)

Mmm, I don't think this is really how design works. I mean, it's design by commiitee. It'll end in an Irradiated Horror

If you want a real design, someone who knows how to design such systems needs to go off and think up a complete solution and come back and deliver it. Doing so in this case will require at least a good overview of all the functions of the server. TBH, this is something the current devs can do, I think - I expect they have all the necessary knowledge.

I'll obviously help if I can, but I don't see I can without learning the current code base.

Writing it will probably take a year, too. It's not a short-term solution. OTOH, it's a *permanent* solution.

FWIW (not much, since I don't know enough about the situation), it seems to me the best short term solution if the C&D is to the host, is to host in another country.

If rehosting can occur, then a gradual move over to a decentralized design could occur while the traditional server runs - gradually moving services over from the traditional server to the decentralized cloud. That would be something where a few months of up-front work would have to occur to get, say, chat working - but you'd have the normal server running fine at the same time.

Also of course, if you decentralize like this, your cloud hosting and hardware costs drop to zero.

[Glader]

Oooof. I profoundly disagree. The computational demand of those services is almost zero. Computers today are extraordinarily powerful and WoW was released back in 2004 - twelve years ago.

The main issue I see is network bandwidth.

Sure, hardware is significantly more powerful today but that doesn't not mean people actually have today's hardware. For reference here is some info on the blades they sold at auction awhile back: https://news.ycombinator.com/item?id=8154515

I personally use a 2009 Dell lol and this is still likely to be better than a lot of people's computers that play old versions of WoW. I know people rocking laptops around that time trying to play.

Sure, you have thousands of these machines that could be running this stuff. I guess that's more important than individual specifications.

Bandwidth would probably be an issue yea. Especially if you have many peers who have to broadcast to a group of other peers. Some of the bandwidth a lot of people have is barley enough for them to watch a video on Youtube.

It's hard to design a reliable system when hosts are asymetric.

Hmmm, why is that? Where would the unreliability come into play?

Mmm - there's no need for one server to handle an entire zone. The size of the zone handled by each computer should vary, and must vary in fact as the number of clients in the system varies.

That could work, that seems more reasonable. Edges of zones has always been an issue in MOGs though. Handling cases where users stand or move back and forth between the edges of a managed section presents difficulties I think.

I think perhaps your original comment I replied to above is based on the idea a client (server) would handle an entire zone. This was not the proposal - rather, each client would handle a small part of the world.

Yea, I thought you meant each client would be managing entire zones and working with others who managed that same zone.

Mangos is sadly written in C++, i.e. the language of the beached whale.

Yea, I'm pretty anti-C/C++. If you're interested in rewriting the cmangos fork Nost has in a managed language like C# I'd try to contribute.

I'm a pretty big fan of C#, MS is going in the right direction with it I think especially with their massive open-source initiative.

[Winterflaw]

Bandwidth would probably be an issue yea. Especially if you have many peers who have to broadcast to a group of other peers. Some of the bandwidth a lot of people have is barley enough for them to watch a video on Youtube.

Yups, and it's a damned if you do, damned if you don't problem - consider; in the naive solution for chat, each client must emit to every other client. That won't work. What's the alternative? every client needs to know where every other client is in the game world, so I can only emit to those who can read what is written - but to do that, every client needs to tell every other client where they are in the game world...

Prolly you can combine the two though. Every client knows where IT is in the game world, and so IT can filter out stuff it shouldn't hear. So the emitter can afford to error generously.

Another thought is that you have something like a write combining tree. Each chat emit enters the tree, then fans out along matching nodes - i.e. retransmission occurs, like NTP. It's not just the job of the emitting node to tell everyone else. You get more latency this way of course.

It's hard to design a reliable system when hosts are asymetric.

Hmmm, why is that? Where would the unreliability come into play?

I edited the original, after your reply but before I saw it, to say it's *harder*, rather than hard.

It's because symetric designs are easier to reason about.

That could work, that seems more reasonable. Edges of zones has always been an issue in MOGs though. Handling cases where users stand or move back and forth between the edges of a managed section presents difficulties I think.

Yeah, I had the same thought.

Mangos is sadly written in C++, i.e. the language of the beached whale.

Yea, I'm pretty anti-C/C++. If you're interested in rewriting the cmangos fork Nost has in a managed language like C# I'd try to contribute.

*grin* I differentiate between C and C++ THANKYOU VERY MUCH

I am a hardcore C programmer. C++? pah!

I'm a pretty big fan of C#, MS is going in the right direction with it I think especially with their massive open-source initiative.

C#??! double pah!!

[wowmad]

.