Forums » Bugs

You might want to revise your updating procedure

Aug 07, 2010 blood.thirsty link
ok guys, we know that you're all kind busy and on a tight schedule to provide us crazy stuff and awesome in game contents.
But, could you wait until the next morning to push up updates,and not doing it at night.
This way you will be fully awake to fix problems and not leave us an unplayable game with unwanted disconnections every ten minutes like this night.
Thank you.
Aug 07, 2010 Router6 link
+1E24
Aug 07, 2010 incarnate link
You are playing a game on this thing called "the Internet", which is not inherently reliable. Just because there was a release 5 hours before does not mean it was related.

It is more useful to report the issues themselves, rather than assumptions.
Aug 07, 2010 blood.thirsty link
Very tempted to link the "be nice" to this reply, but that would get old.

You want issues, fair enough, here there are:
Most of the players at that time experienced disconnections. The ones i know were from different European countries Uk Russia, France.
Effects varied from my side, from chat lag, invincible moth (even to flare spams) ultra fast convoys jumping a sector, loading screen delayed.
When i logged i went to do some movie streaming and it was just fine; I don't know for the others, i don't live with em :p

I ain't expecting a reply, so do what you want with this.
Next time i'll just get lost and do something more interesting of my time.
Aug 07, 2010 slime73 link
You have a few bugs for only a couple hours and whine about it as well as being passive-aggressive towards the developers? It sounds like your issue isn't with the ingame problems you had but with something else, as I can't understand why you would be so mad over a few temporary issues that probably weren't even related to the update.
More often than not, lag issues such as you describe are actually caused by problems upstream of the servers' connections. And even if the lag issues originated at the servers, why are you so sure that the update caused them?
Aug 07, 2010 incarnate link
Most of the players at that time experienced disconnections. The ones i know were from different European countries Uk Russia, France.
Effects varied from my side, from chat lag, invincible moth (even to flare spams) ultra fast convoys jumping a sector, loading screen delayed.
When i logged i went to do some movie streaming and it was just fine; I don't know for the others, i don't live with em :p


Yes.. that's how the Internet works. At any given time, links go down, things break, people experience lag, but only on particular routes.

A lot of movies and other content of that sort is streamed by proxy services like Akamai, a type of distributed content hosting service that can spread one-way content all over the globe, so you're always most likely to pull it from a server farm in close proximity to you. This reduces the chances of outages, and improves performance for customers. YouTube and all other media hosters do this (Google has their own farms and does it internally, but it's effectively the same thing).

This is not possible for a realtime, two-way game. Or any realtime, two-way communication, like VoIP (phone over the internet, like Skype, etc). In that case, the packets still have to traverse the required distance, like across the ocean or whatever, so you can't locally cache "everything" next to the remote user like you can with a movie.

Our game, along with every other remotely realtime online game ever made, has been plagued by outages large and small since we created it. It's a fact of life, that will never entirely go away. It is not useful to say "I can get to X, but not to Y! Therefore Y is at fault!", because chances are it's something way back at like.. C, in your particular route to Y, that is to blame. We are not responsible for the Internet, and we cannot control what happens. We made the game as resilient to bandwidth and latency changes as we possibly could, and extensively tested it on a protocol level, but there is no cure for "packets stop getting through because a router rebooted in Maryland".

Moreover, when large upstream links experience issues, it usually impacts large numbers of people. It is not at all surprising that a lot of European players should experience problems at the same time. (On a side note, it's actually a credit to our game that it still worked at all, our protocol is actually pretty robust to these issues.. as robust as the laws of physics permit).

Don't get me wrong, I would like to have the best possible performance for our players, and we have a top-class hosting datacenter with many redundant connections in the gigabit range (literal gigs of actual transit pipe, not just gigE wirespeed). But, that's about as much as we can do; and even if we owned our own billion-dollar ISP with our own fiber going everywhere in the world, there would still be outages in localized regions that were beyond our control.

What I would like to do is create a server-side feature that will detect when people are disconnected, while maintaining active ping measurements to our upstream providers, and when those users reconnect.. explain to them that the issue they just experience had nothing whatsoever to do with us, and was somewhere in the global path between our servers and them. That would at least cut down on the support time we spend on issues like this, which people are prone to blame on us, or our release, or our servers.

Someday, if we have a lot of players, maybe we'll be able to split off a dedicated European server, which will be more reliable for European players. But then, they will not longer be able to play with their current North American friends, so it's really a big tradeoff. We're a single-universe game, and we're hosted in the US.. as long as we remain a single-universe game, these issues will occur. I actually spent a lot of time thinking of wild self-healing schemes to make it work as well as possible, but in the end.. in PvP with two people in locations remote from one another, there is nothing one can do about the experience of an outage between their locations.

..and even if we did have a European cluster, it would still have periodic outages, because again, at any given time something is breaking somewhere on the 'net. Whether it's in Maryland or Chicago or Germany or Shanghai. Speaking as one of the people who helped architect the mess that is the Internet (as a backbone engineer at a Tier-1/Tier-2 provider in the 90s), I speak from experience.

I did not intend to be rude, or not-nice with my response. I was curt because it is unfortunate that despite the amount of time we spend on testing releases with our small team, we still get blamed for issues that are probably not within a thousand miles of us. But, from the perspectives of our players online at the time, they must be Our Fault. But, that's life. C'est la vie.

I do appreciate the reports, those are useful, but please don't jump to conclusions on who is to "blame".
Aug 07, 2010 vskye link
Well said inc.

Seriously, I used to work at a ISP, and sometimes shit just happens. Backhoe, blown out switch, router, acts from god, whatever. And to top it all off, 99% of the time it wasn't even any of your own equipment.

Silly BT, ya Internet newb plant.