Forums » Bugs
OK, I think this deserves a bitch session...
Half a dozen players gathered in B-8 for pewpew, but the sector lags so bad it's not worth playing.
WTF???
Faille.
Half a dozen players gathered in B-8 for pewpew, but the sector lags so bad it's not worth playing.
WTF???
Faille.
Yeah we were all landing hits and nobody was doing damage.
Is this a game limitation or is it because the servers are not equipped to handle lots of people
Is this a game limitation or is it because the servers are not equipped to handle lots of people
I got a video of it for the devs to watch on request. if they need it.
Was really really annoying.
methinks its moms convoys, because before them, none of this happened.
Was really really annoying.
methinks its moms convoys, because before them, none of this happened.
Could have just been a net poblem - The VO servers usually handle large scale battles very well I dont know what the record is but I have been in sectors with 30 + and no lag was evident.
This happens to me as well. Now I hate station sectors because of the fake voys. On Friday, the tablet crashed at least 5 times. I run 7 plugins total but I did that number before momerath added the voys, and the tablet NEVER crashed.
Occasional sector-wide inability for anybody to damage anyone has been an issue for years, so I wouldn't blame the fake-convoys. Most of the times I've seen it were during buswars at Sedina D-14.
It's occurred numerous times during NW as well.
This also runs rampant during conq station assaults. It's not lag in the sense that there's network problems, but more like the server isn't "confirming" hits for any shots. It happens on a per-sector basis and affects everyone, including NPCs.
If the servers have not been restarted in a long time they are well known for having issues.... I think I did mention the devs need to have an automated method of restarting VO servers that will not disrupt everyone or they need to set a regular restart schedual when usage is at its lowest - it also should restart anytime player count drops to 0....
Anyhooo back to the bitchin
Anyhooo back to the bitchin
I've been having more lag and client freeze-ups that usual since about 3 updates ago (2 weeks).
More bork city in h2, we had a trident battle and it got to the point where we were unable to damage either tridents.
When is this going to get fixed?
When is this going to get fixed?
I am seeing lots of lag also and im only botting must be worse for pvp
Well You see VO runs in part inside Johns brain and he took some cold medicine......
I don't know what's happening. A few points:
- Server OS and game uptime have absolutely nothing to do with sector lag, and never-ever have. This is a player-invented fiction that has been repeated enough that everyone believes it.
- Network weather is a very real cause of these kinds of issues. A peering point will get overloaded, and everyone who traverses that point will be affected. This doesn't mean like "people in Maryland" this means like "people who come in via Level3", which will end up being like half the players online. Thus everyone goes "Rar!@#, the servers!@#", even if I'm online at the time, in the same sector, and playing perfectly because my Wisconsin home location means I'm not traversing the same overloaded peering point at MAE East.
Of course, people still say stuff like "It's not lag in the sense that there's network problems, but more like the server isn't "confirming" hits for any shots.", where, yes, it can be the network, because that's what actually has to be traversed for your shot confirmation to work.
- In practice, 90% of the times when people complain that the "servers are overloaded", it isn't actually related to the servers or bandwidth being overloaded. It's usually a client-side and intervening network issue (and when people say "but it happened to all of us!".. yes, things like that are going to become more evident when you have a situation where impacted players are all together). BUT, there can be rare instances where a sector will suddenly spike up in usage on a given cluster machine, and degrade the cpu availability to other sectors already running on that same cluster machine. We avoid this by profiling high-CPU sectors, like Deneb battles, and running them on hardware that is pretty much dedicated to "high-load" gameplay. But, in general, we have enough hardware to support at least one or two orders of magnitude (10x-20x) more concurrent players than we have.
The cloud test is more about dealing with situations where we get like.. a hundred orders of magnitude (1000x) in a short period of time.
--------------
That all being said, there might be some other.. thing.. going on, that doesn't show up in any of our graphs or whatever, that we need to try testing for. I don't have any answer right now, we have no development hours available. When we do, we'll try to improve this.. as obviously large group-warfare stuff is a huge part of this game and something that's only intended to become more common.
But the most challenging aspect is how to tell people "it's not us, it's you". Because judging by history, most of the time it is not us. And 10 people who live on the east coast finding their connection to the midwest sucking at the same time is not unusual on "the internet".
- Server OS and game uptime have absolutely nothing to do with sector lag, and never-ever have. This is a player-invented fiction that has been repeated enough that everyone believes it.
- Network weather is a very real cause of these kinds of issues. A peering point will get overloaded, and everyone who traverses that point will be affected. This doesn't mean like "people in Maryland" this means like "people who come in via Level3", which will end up being like half the players online. Thus everyone goes "Rar!@#, the servers!@#", even if I'm online at the time, in the same sector, and playing perfectly because my Wisconsin home location means I'm not traversing the same overloaded peering point at MAE East.
Of course, people still say stuff like "It's not lag in the sense that there's network problems, but more like the server isn't "confirming" hits for any shots.", where, yes, it can be the network, because that's what actually has to be traversed for your shot confirmation to work.
- In practice, 90% of the times when people complain that the "servers are overloaded", it isn't actually related to the servers or bandwidth being overloaded. It's usually a client-side and intervening network issue (and when people say "but it happened to all of us!".. yes, things like that are going to become more evident when you have a situation where impacted players are all together). BUT, there can be rare instances where a sector will suddenly spike up in usage on a given cluster machine, and degrade the cpu availability to other sectors already running on that same cluster machine. We avoid this by profiling high-CPU sectors, like Deneb battles, and running them on hardware that is pretty much dedicated to "high-load" gameplay. But, in general, we have enough hardware to support at least one or two orders of magnitude (10x-20x) more concurrent players than we have.
The cloud test is more about dealing with situations where we get like.. a hundred orders of magnitude (1000x) in a short period of time.
--------------
That all being said, there might be some other.. thing.. going on, that doesn't show up in any of our graphs or whatever, that we need to try testing for. I don't have any answer right now, we have no development hours available. When we do, we'll try to improve this.. as obviously large group-warfare stuff is a huge part of this game and something that's only intended to become more common.
But the most challenging aspect is how to tell people "it's not us, it's you". Because judging by history, most of the time it is not us. And 10 people who live on the east coast finding their connection to the midwest sucking at the same time is not unusual on "the internet".
Thanks Inc!
The issue seems to be more coincidental than that though, every time someone brings a trident out and starts with the massive missile turret spam this happens after a while. It paints the picture that trident turrets are the problem
The issue seems to be more coincidental than that though, every time someone brings a trident out and starts with the massive missile turret spam this happens after a while. It paints the picture that trident turrets are the problem
Yeah, I understand that. And we'll check it out. But it probably isn't "what you think", so to speak.
For instance, it's possible that the missile spam gets to a point where it exceeds the outbound packet queue length per-user and you start to see induced latency on network traffic. That isn't by accident, if for instance someone were to do a megabit of crazy missile crap, with 20 people in the sector, we'd have to pump out 20megabits of data, which is unsustainable and un-scalable (imagine with 40 ships. a hundred?).
However, we can still take a look at exactly when and how that is happening.
But keep in mind that, given how relatively-well the client does interpolation and handles packet dropouts and things.. if you are in a bad "network weather" situation, you're only going to really notice once you start pushing the limits harder. Which would imply causation to people in the situation, and totally NOT be true. Ie, the old rule of "correlation is not causation".
For instance, it's possible that the missile spam gets to a point where it exceeds the outbound packet queue length per-user and you start to see induced latency on network traffic. That isn't by accident, if for instance someone were to do a megabit of crazy missile crap, with 20 people in the sector, we'd have to pump out 20megabits of data, which is unsustainable and un-scalable (imagine with 40 ships. a hundred?).
However, we can still take a look at exactly when and how that is happening.
But keep in mind that, given how relatively-well the client does interpolation and handles packet dropouts and things.. if you are in a bad "network weather" situation, you're only going to really notice once you start pushing the limits harder. Which would imply causation to people in the situation, and totally NOT be true. Ie, the old rule of "correlation is not causation".
Of course, people still say stuff like "It's not lag in the sense that there's network problems, but more like the server isn't "confirming" hits for any shots.", where, yes, it can be the network, because that's what actually has to be traversed for your shot confirmation to work.
Inc, people say that stuff because other things that also traverse the network like chat, ship positions, shots being fired, rockets exploding still work perfectly whereas the only things suffering from this are hit confirmations (no beeps) and damage actually being dealt. When a sector does this, we can get right in each-others faces, empty all of our rockets and watch them explode, then laugh about it on sector chat without taking any damage. Go to another sector and things work just fine.
When there's a problem with the network, ship positions, chat, weapons fire, sector jumps, taking missions, etc are all affected. I can cause this whenever I want by starting up a torrent client, sitting next to the microwave while on wifi, or in the case of my phone, just fire all 134 missiles from a Ragnarok on LTE and watch the lag fly.
Inc, people say that stuff because other things that also traverse the network like chat, ship positions, shots being fired, rockets exploding still work perfectly whereas the only things suffering from this are hit confirmations (no beeps) and damage actually being dealt. When a sector does this, we can get right in each-others faces, empty all of our rockets and watch them explode, then laugh about it on sector chat without taking any damage. Go to another sector and things work just fine.
When there's a problem with the network, ship positions, chat, weapons fire, sector jumps, taking missions, etc are all affected. I can cause this whenever I want by starting up a torrent client, sitting next to the microwave while on wifi, or in the case of my phone, just fire all 134 missiles from a Ragnarok on LTE and watch the lag fly.
Inc, people say that stuff because other things that also traverse the network like chat, ship positions
I understand what you believe to be the case, and what you are correlating. But your specific example is actually a great illustration of why it probably isn't what you think.
But, I can't really write another big post right now.. we will check into this and see what we can do to improve the situation (in network-code land, it's less about "fixing" a problem as it is re-working the architecture to make the broken network less apparent; but if there's a capacity/spike issue too, we'll check that out as well).
I understand what you believe to be the case, and what you are correlating. But your specific example is actually a great illustration of why it probably isn't what you think.
But, I can't really write another big post right now.. we will check into this and see what we can do to improve the situation (in network-code land, it's less about "fixing" a problem as it is re-working the architecture to make the broken network less apparent; but if there's a capacity/spike issue too, we'll check that out as well).
Inc, in addition to making it less apparent when there's network issues, how about also including some kind of indicator in the game client that there IS a network problem that it's trying to work around. This way the game can tell the player "Hey, I'm having trouble talking to the server right now. Go check your network." Some other games do this and it's helpful. (Guildwars and Half Life 2 come to mind)
All we have right now to tell is to use outside tools (ping,traceroute,mtr,etc) and /verbose 6. Most people aren't going to know how to use the network tools and they don't accurately represent how the actual game's protocol is treated by all of the routers in between.
Vendetta's /verbose 6 isn't very useful to most (any?) of the playerbase because it gives you no frame of reference for how many retransmits are acceptable.
All we have right now to tell is to use outside tools (ping,traceroute,mtr,etc) and /verbose 6. Most people aren't going to know how to use the network tools and they don't accurately represent how the actual game's protocol is treated by all of the routers in between.
Vendetta's /verbose 6 isn't very useful to most (any?) of the playerbase because it gives you no frame of reference for how many retransmits are acceptable.