Full of Fail

They say you shouldn't blog angry. You can add that to the long list of seemingly good advice that I don't heed.

So I've been in crunch for the last couple months which in general makes me crankier than normal. At some point in the last week or so I picked up the Supreme Commander expansion. You also might recall that I like Supreme Commander. It is, after all, being the spiritual succesor of one of my all-time favoritest games.

I verily took said expansion, installed it, ran it, patched it (multiple times) and for my $39.99 at that hive of scum and villainy otherwise known as Best Buy, I got this:



That's it. No indication of what's wrong in any kind of end-user-meaningful way. No clear recourse as to how to fix the issue whatsoever. Smashing. I'm going to guess that this has something to do with SecureROM. Ya know what? If my reward for buying the game is not being able to play it then maybe I shouldn't be buying your games! There's probably a workaround but the days of me having to fiddle around with that kind of shit should be way over. It isn't 1997 anymore.

Guys, I know software is hard, but would it be too much to let me play the game I just bought?

Oh man...

Crap. Double crap.

So sad; I just bought one of these so I can't justify it (even to myself). But they're soooo coooool!

Stupid technology addiction.

What You Should Know About Load Times

(cleverly abbreviated as WYSKALT)

If I have time to get pissed off and say "I can't help but notice that I'm not playing your game RIGHT NOW," then it means your load times suck and I'm probably wondering why I bought your game in the first place.

This seems bad!

Loading your game off media isn't rocket science. I'm going to assume that we're loading off some kind of disc with a 'c' media like in a console game rather than your hard drive like on your PC of choice. However, anything applied to optimize load times on disc-with-a-c will also optimize disk-as-in-hard-drive as well. I'm not going to cover everything with load times in this post, but this is the absolute bare minimum of stuff you need to know but I couldn't fit it in quite as catchy an acronym. There's lots more stuff that will make load times even better which I might rant about at a later date.

So at any rate, here we go.

Opening Files Sucks
Most times we don't notice this. Occasionally you run into something where you need to load N files which are kinda small but where N is something gigantic (like 1000). Then you notice--even if you have an uber fast machine. On this generation of console, N is usually on the order of thousands for a given level load for those who might be curious.

Fixing this is uber-easy: pack all your files into one giant uber-file so you only ever need to have one or at most a small number of file handles open. We often know of these as zips, wads, paks, pigs, etc. (The one I use for BEER is "keg". Clever, no?)

This makes load times suck less which is the goal. Sadly, they still suck.

Loading Non-Sequential Data Sucks
So now you've got your data all archived and it's time to load it. Surprise! It still takes a long time because you're constantly seeking around in your uber file. I can hear you smug Win32 people scoffing--but remember that your file mapping eats virtual address space and I know that since you're still using WinXP that you only get two gigs of VM. Ha!

This is pretty easy conceptually: stick all the data in your archive in order. The bad news is that depending on your particular data set and load time trickery, this might be really hard in practice. Chances are good that it leans toward the really hard side because you might only load some subset of your data and probably don't have a fixed order for it. If you can make your loading respect a fixed order then the problem moves pretty quickly to the easy side again.

I've found that in most cases the best we can hope for is that some subset can be made to lean toward "most" for fixed sequentiality. Sometimes you can change the loading code to do nicer things; sometimes you can't for <fill in the blank> reason that fits your situation. In either case you need to generate a sequence list for some set of your data then stick all the data in your archive in that order as best you can. Neither of these are particularly difficult assuming you have source code and enough time to do stuff and an archive builder that doesn't totally blow chunks.

Why does this work? Once file open issues are dealt with, seek time tends to be the dominant factor in unruly data sets. On discs-with-a-c, each seek is going to weigh in at roughly between 50 and 150 milliseconds depending on your particular hardware. So, if you're loading 10MB of data but seeking 100 times, your load bandwidth for that data can be no better than 1MB/s in the best case (assuming a flat 100ms/seek which is normal). If you try to load a gig of data with that bandwidth, then, well, I'll not be alone in returning your game.

So: if you can significantly reduce the number of seeks your game needs to do, you can significantly reduce the load time of your game. I've seen particularly poorly loading data sets extend the load time of a 100MB data set more than five minutes. For those in the audience reaching for calculators, that's a bandwidth of less than 1MB/3seconds. Glacial.

But Our Game Is On the Hard Drive!
Don't care. These will still make your loads better. Your seeks are still on the order of 8ms and your bandwidth isn't free. Wait till your users start fragmenting the crap out out their drives. Galactic Civilizations is a reasonably fun game but its load times are absolutely brutal. Why? Tons of tiny files none of which seem to be in easy-to-load formats just sittin' there on the disk. Lots of games are like this for the PC and it's a damned shame.

Oh! You did that for your modders, did you? Guess what; you can still put most of your stuff in order in archive. You can even give modders your packing tools so their stuff loads fast too! Sure, you have to design for it but you're already designing extra stuff in for modders, right?

Odds, Ends, and Disclaimers
As mentioned atop, this is the bare minimum you need to know about load times. There are lots of other things you can optimize to make your loads suck less but I'll save that for another rant.

This has been a public service announcement paid for in part by the "Your Game Loads Too Slowly" foundation of Maryland and made possible by viewers like you!

Script-fu

Not terribly long ago, I was a self-proclaimed scripting hater. "Oh sure," I said, "You can do some neat stuff in script-land with your not-real languages and your dynamic types. But wow, my C++ can do all of that and uber-more!" Yeah. So it's time to fess up and admit that I might have, ever so slightly, been somewhat mistaken on some 'a that.

I've had this completely irrational thing for Ruby for, oh, about a year now which mostly manifested itself as not doing Ruby. This (irrational thing for Ruby) is mostly the fault of the blogosphere, notably one Steve Yegge who, despite often being somewhat long winded with a propensity for ten cent words and compilers, is also quite often terribly amusing and a lot of the time spot on. He's got loads of stuff trumpeting the awesome that is Ruby. So I bought a book and read some online tutorials and then got a new job and moved cross country and got tied up in godawful amounts of stuff that's not ruby and wow is that a pretty laptop I should totally get one of those while not doing Ruby.

I used to know perl in much the same way that most people who knew (past tense) perl once did. That is to say, I have absolutely no ability to do perl now because a) it's very obtuse, b) it looks like line noise, and c) it's so unbelievably obtuse that sane human beings flee in terror when confronted with it. Have I mentioned its obtusitude?* Some have described Ruby as "perl with the suck removed" and as I mentioned before, I have a thing for Ruby.

<yet another entry for the worst segue ever category>

I'm now in charge of making load times not suck on my current project. I'd built a couple tools to build giant archive files and to parse in-game spew to do various things that are reasonable and useful toward that goal. I'd built them in C++. This in and of itself is not odd since by and large, I'm a C++ programmer. What I did notice is how godawful difficult seemingly simple things are in C++. This also is not odd since by and large, I'd been noticing that on and off for the last, say, decade. So moving from "hey, I think I can fix our load times" to "holy sweet moby jebus, I have to fix our load times!" I decided that I needed better tools.

Enter Ruby, stage left.

So I used it in a real "have to get this crap working right now kind of situation...and it was good.

So now I gotta take back some of the mean things I said about scripting languages and the people who promote them. Uncle already. I may have been a bit hasty with all of that. Ruby has made the otherwise unbelievably daunting data mangling tasks I've got slightly less unbelievably daunting because now at my very fingertips I have the power to mangle data like never before! Data the world around, tremble before my gemstone inspired might!

I'll save some gory details and horribly contrived examples from Teh Real World(TM) for a later date. So, if you're like me and only really ever looked at your primary statically-typed language for solving all manner of problems, you might go and pick up a scripting language just cause. It might save you some unbelievably daunting.

*obtusitude is not a real word. I made that up, honest.

HL2

One of the (many) things I missed out on during my two year long stint in WoW is Half Life 2. Despite being told how uber-awesome it is, I never got it since at the time I didn't have enough machine to really play it. So beta for the Orange Box rolls around and my co-workers are playing TF2 and I hop on the bandwagon...just in time for them to lose interest (bastards). TF2 is plenty of fun on its own but it also gave me a chance to catch up on otherwise missed gaming (brilliant move by the Valve guys).

I wasn't prepared. Half Life 2 rocked my socks. There are definitely things about it that I didn't like but overall I thought it was an excellent game. Add to that the two free-to-me episodes 1 and 2 and it ended up being even more excellent. I missed the whole "better than sliced bread" launch so I had to judge it on its own merits and I still think it's by far the best single player game I've played in the better part of two years.

If you missed this one, aren't totally put off by jumping puzzles, and enjoy a good action FPS, I highly recommend it. Can't wait for Ep3.