Forums » General

Website went down?

Mar 13, 2005 macguy link
What was that all about? Somebody knock over the server box?
Mar 13, 2005 sloanesky link
hmm? It didnt go down for me.
Mar 13, 2005 macguy link
It was down for awhile. I know some of you did experince it being down.
Mar 13, 2005 roguelazer link
It was down all morning until about 12:30 in the afternoon for me.
Mar 13, 2005 paedric link
Started after midnight for me.
Mar 13, 2005 epadafunk link
Mar 13, 2005 KixKizzle link
This was after he said the fiber splice was over epadafunk.
Mar 13, 2005 paedric link
As Kix said, this was well after the splice. I started having problems accessing the forums about 12:30 am CST. This lasted through late morning or early afternoonn today. May have been DNS problems but have no info regarding the actual problem.
Mar 14, 2005 incarnate link
The website outage was unrelated to our ISP.

We had swapped out an ethernet card in our webserver, due to some esoteric APIC bugs in our kernel with the particular type of gigabit card we were using. Unfortunately, we forgot that our webserver software (Litespeed) authenticates its "registered" (ie, purchased) state against the mac address of the first ethernet interface in the machine.. in this case, that card which we had swapped out. It complained to logs for a couple of days, and then died. When I realized the webserver was down, I restarted it and saw the "unregistered!" type message, and got a new reg code for us, based on the new gigE board.

So anyway, sorry about that. It's just a dumb demonstration of how we need more internal monitoring and watchdogging on our servers (which we used to have, but when we redid the servers for game launch.. yeah.. crazy time, we had other priorities). One more thing on my to-do list..
Mar 14, 2005 Veadro link
Anything special about that webserver product?
Mar 14, 2005 incarnate link
Well, yes. With a given machine, you can serve a lot more webpages with it than with say.. Apache. Not ragging on Apache at all, I love Apache, but it was designed more as a "do everything" than as a "do a few things really really fast". In more technical terms, Litespeed uses a kqueue (FreeBSD) or epoll (Linux) based tight-loop design, similar in concept to Zeus, thttpd, Boa, and various other select() driven webservers. It also does a lot of intelligent caching and various things to keep I/O non-blocking and such.

Also supports chroot(), hardware SSL offload, and other things. You can find a perma-link at the bottom of this page (the "powered by litespeed" icon.

We originally looked at Apache alternatives when we were concerned about getting swamped by Slashdot. We didn't have the money for a lot of server hardware, so we needed to get as much out of what we had as possible. The site itself is designed pretty intelligently, but we also wanted a really lightweight httpd. Considered writing one ourselves (more time taken from game development), poked around at a couple of others, then ended up with Litespeed.
Mar 14, 2005 Beolach link
Dude. You considered writing your own httpd, just to host the website for your real project, the game?

And here I thought I was a nerd.
Mar 15, 2005 incarnate link
Well, if you're running everything off of random old hardware from your basement, and are worried about serving say.. a few hundred thousand visitors a day off of a PII 400 with 512mb of ram (our webserver for most of the Test era, since upgraded), you start to look at any possible options for getting the most out of what you have. Any yeah, we have a tendency to roll our own.. everything. Because we always wonder if we could make it better and faster and cooler than whatever the standard thingie may be. We considered making our own database, too, at one point, but ended up using MySQL. Being open to doing everything yourself is good, in that you're self-sufficient and can optimize everything to your needs.. but it can also be bad if you spread yourself too thin.
Mar 15, 2005 Beolach link
4 guys, creating a full fledged MMO, a database server, and a web server would be spread too thin? Nah, you could probably even roll your own compiler, too!

You guys are just incredible. I am constantly awed at how much you get done.