Technical solution - a distributed server

Discussion forum related to Nostalrius Begins in general.

Technical solution - a distributed server

by Winterflaw » Sat Apr 09, 2016 11:53 am

What we need to do is create a server which is immune to shut-down orders.

For this to be possible, it must be decentralized.

This implies a "server" which is actually distributed over and composed of all the machines who are currently logged into the game (and perhaps some longer term permanent machines - but these are optional conveniences).

When a player connects, he receives the current list of the IP addresses of everyone in the game.

The server distributes multple copies of all objects across all machines, such that a complete copy of everything is always available.

Computation occurs on the machines holding the objects being manipulated, which then exchange results.

Another possibility also is to run the server in the Tor network, using .onion addresses. No DNS requirements, and new .onion addresses can be instantly created. The WoW client can be externally routed into the Tor network using a local proxy.

Latency will increase, which is the issue, but the question is by how much?
Winterflaw
Sergeant Major
Sergeant Major
 

Re: Technical solution - a distributed server

by justrambling » Sat Apr 09, 2016 1:01 pm

I was thinking something similar the other day.

Use a master behind tor to keep copies of all character data, then distribute a server client and an user client.

let user clients connect to tor and from there get ips of the distributed servers. If you can get enough redundancy you should be able to get to a point where its "safe".

Still a ton of work. though. And it relies on the community providing hosting.

The fact is that actively dodging blizzard shutdown attempts may very well scare players away. As soon as you start accepting that what you're doing is piracy or against the law, a lot of players will stay away on principle.
justrambling
Tester
 

Re: Technical solution - a distributed server

by wowmad » Sat Apr 09, 2016 1:32 pm

.
Last edited by wowmad on Sun Apr 17, 2016 5:30 pm, edited 2 times in total.
WowMad
Against any intolerance in the form of confrontation over the beautiful people
User avatar
wowmad
Senior Sergeant
Senior Sergeant
 

Re: Technical solution - a distributed server

by r00ty » Sat Apr 09, 2016 4:06 pm

I don't think the WoW protocol allows such a setup. It would allow you to distribute wow servers, even potentially support failover and overflow onto a new "world server" as they demo'd in the video. But, there always needs to be a central social server, to handle chat, guild info, player lists, and the shunting between servers. Plus, the database always has to be in one place.

To me, the best solution would be to produce the tools to export the data as they say they'd like to. Encrypted per player and only decipherable when you can supply some info about your account.

They would also need to make those tools open source and available (if the encryption is good, and based on good data only the real player would know, it's safe to be open source). That way any time there is a take-down the current database can be made available in the same way and the torch is passed on.

On that subject, I'd suggest writing a set of SQL scripts to extract everything about a player into a single file (XML, or .SQL it matters not). So, auth data, character, guild, all the item table data for them etc. Then of course another script to import that (noting that there will need to be some mass renumbering done). It's possible but a bit of work is involved. Then, each file should be encrypted with a key that is either derived or protected by a combination of the user ID and password (and maybe email address too).

The main problem is, once this information is made available, there will be a LOT of servers pop up, and it'll be a VHS vs Beta x 1000. Resolving it to 1 (or even a handful) of successors will be tough.

As for the source. I'd like to see the well coded stuff absorbed into whatever project it was forked from, or at least a new git open source project created that can be the basis of a community run effort. But, time will tell if they'll even release either.
Casual and proud.
User avatar
r00ty
Sergeant
Sergeant
 

Re: Technical solution - a distributed server

by wowmad » Sat Apr 09, 2016 5:37 pm

.
Last edited by wowmad on Sun Apr 17, 2016 5:30 pm, edited 2 times in total.
WowMad
Against any intolerance in the form of confrontation over the beautiful people
User avatar
wowmad
Senior Sergeant
Senior Sergeant
 

Re: Technical solution - a distributed server

by Glader » Sat Apr 09, 2016 6:25 pm

I was thinking about this yesterday since we're getting to the point where something like this would be the only safe way to handle things.

The issues I see is your part about "exchanging results" could be significantly slower and higher latency than if there were a central server. It might also take some time for these changes to actually ripple through the rest of the distributed system to update their game state. This introduces headaches where either the entire system doesn't move forward until everyone agrees OR the entire system is out of sync and every client see the world differently.

I just don't see it as possible. You can't have every player simulate the world and expect every client to be seeing the same thing. Things will desync as you have issues with floating point precision, non-deterministic pathing and other things like that. Who is correct in this scenario if you try to resync too?

It's like you'd have to rebuild a significant part of the game to support something like this.
Glader
Sergeant
Sergeant
 

Re: Technical solution - a distributed server

by Creep » Sat Apr 09, 2016 6:29 pm

I've also been thinking of various ways to overcome this, but another big issue is that there is always going to be at least one weak link in the chain who is going to abuse the system and corrupt the data in some way to try and get themselves better gear than he/she deserves.
Creep
Private
Private
 

Re: Technical solution - a distributed server

by Glader » Sat Apr 09, 2016 6:39 pm

Creep wrote:I've also been thinking of various ways to overcome this, but another big issue is that there is always going to be at least one weak link in the chain who is going to abuse the system and corrupt the data in some way to try and get themselves better gear than he/she deserves.


Well non-time sensitive things like loot and drops can be high latency actions that all other nodes can agree on first. However, issues in agreeing on damage players took or damage monsters took is significantly more difficult because high latency there is unacceptable PLUS the game world for every other players isn't going to look the same for everyone. Plus you'll need to implement a physics/world rewind mechanic so that when people receive this information they can TRY, emphasis on the try, to rewind to the state the world was seen in by the sender. Since latency between request and receiving on other nodes will be anywhere between 200-1000ms it's going to be difficult or impossible to deal with.
Glader
Sergeant
Sergeant
 

Re: Technical solution - a distributed server

by wowmad » Sat Apr 09, 2016 6:59 pm

.
Last edited by wowmad on Sun Apr 17, 2016 5:31 pm, edited 3 times in total.
WowMad
Against any intolerance in the form of confrontation over the beautiful people
User avatar
wowmad
Senior Sergeant
Senior Sergeant
 

Re: Technical solution - a distributed server

by Winterflaw » Sat Apr 09, 2016 7:07 pm

Glader wrote:You can't have every player simulate the world


I bet the game database is only some gigabytes big. I wasn't thinking of full database replication on each client, but why not?

Things will desync as you have issues with floating point precision, non-deterministic pathing and other things like that. Who is correct in this scenario if you try to resync too?


There's a couple of approaches to this. One is timestamp based - however, you have a problem then synchronizing clocks. Google made an advance in technique with this method a year or three ago; each timestamp comes with error bars, so you can know a *range* of times where an event could have occurred. Another way is to use the Paxos protocol, lthough it's not so well defined and I'm not sure what the latency is - but the big newSQL databases use Paxos, where multiple servers can concurrently receive conflicting updates.

Latency remains the question.

It's like you'd have to rebuild a significant part of the game to support something like this.


In the server, yes. The client would see no change, as it would connect through a proxy - the proxy, which would run on the client, would convert client-emitted requests into a form sensible to a distributed server.

(This forum BTW sucks - it keeps logging me out by the time I post, and when I log in, the post I've written is lost. It's a good thing I've enough experience with ballsed up web-forms to always make a copy of the content before I try to submit.)
Winterflaw
Sergeant Major
Sergeant Major
 

Next

Return to General discussion