Fall from Heaven is a dark fantasy Civilization IV mod, built by a team headed by Derek Paxton. The first version was released on December 16th, 2005 and last month – exactly three years later – the “gold” version was uploaded. The project is the most successful Civ4 mod yet created, with hundreds of thousands of downloads and positive nods from critics like Tom Chick. I recently had an opportunity to interview Derek and his team on the final release of his the mod, which is available for download here. (Note: Fall requires a fully patch version of Beyond the Sword to play.)

Soren: How do you guys handle communication and source control? Are there private development forums? Have any of the members met or talked in real-time?

Derek: Thunderfall, admin of CivFanatics, was good enough to give us access to a public forum and a private team forum.  To date, the team forum has over 16,000 posts and the public forum has about 150,000 posts.

Although I may exchange occasional private messages or emails with a few team members, 99% of everything we do is done in the team forum.  It allows people to participate in the discussion as their schedule allows and for everyone to openly talk or discuss ideas.  Since all the messages are retained, it makes it really easy to go back and pull out ideas that may not have worked well at the time (okay, I’ll admit it, to go back and get ideas that I didn’t like at the time but finally realized that the team knew better than I did).

Source control isn’t a problem since I’m the only one with with access to the source code.  Art and writing assets are posted, and I cut them into the mod.

I don’t think any of the members have met in real-time.  I did talk to Woodelf on the phone once and took remote control of his computer to check out a problem he was having on Vista. I also sent everyone on the team a copy of “The Ultimate History of Mythology” and “History of Ancient Civilizations” books for Christmas one year.  As they correctly figured out, it was actually a homework assignment to help generate ideas.

William: Stephan and I were on the verge of meeting when we were in England at the same time a year ago, but unfortunately it didn’t pan out. I suspect it might be easier for us European contributors to meet as we’re all concentrated in Northern Europe. (Sto is in France, but – with the motorways – that shouldn’t be a problem.)

Soren: How do you keep a group this large organized? I noticed that there are twice as many “content creators” (artists, writers) as “system creators” (programmers, designers). Do you think that this type of ratio is a sign that getting programmers to work together harmoniously is a lot harder than getting artists to do so (who can work on separate assets in parallel)?

Derek: The second greatest risk to a mod is the “unfixable” error (the first being abandonment by the creators who burn out or go on to other things).  A lot of how we work is based around trying to mitigate that risk.  One way we do that is by forcing all code to go through me.  It’s not that I am any kind of standard bearer for proper programming – Chalid, Talchas and Sto are all better programmers than I am – but it’s important that one person understand all the changes in the mod.  That understanding has to be at the code level.  The reason is that issues will come up.  Issues that occur in a given function are easy to troubleshoot and fix.  Issues that occur between functions (the way two separate functions interact with each other) are much more difficult.  Unless a common person understands both of those functions, it multiplies the difficulty, especially when one of the functions has no apparent relation to the other.

Occasionally, I’ll even ask Talchas or Sto to provide simpler versions of their code before it’s added.  Or I’ll rewrite it in a form that I understand but may not be as efficient or full featured as their version.  They gradually teach me more advanced programming, so I improve, and the code for the mod stays relatively simple, which is one of the reasons I think others like to mod it.

Artists and writers are much easier.  I keep an updated to-do list for them, and they produce assets as it appeals to them.  Though they do a lot of work maintaining internal consistency either in the background or in the art style, it isn’t as critical as it is with code.

Soren: One of the biggest benefits of having all of Civ4‘s game/AI code in a separate project for the SDK was that it was always very clear where the line was between my code and the rest of the team’s code. With the exception of some of the utility stuff – like the XML parsing and the python glue – I wrote every line in the SDK, which gave the code a certain internal consistency that cut down on a lot of bugs. It looks like Fall has a similar philosophy in that all code changes need to pass by a single set of eyes. This level of consistency is very important for modders, I believe, as the code develops its own type of distinct grammar. When you pursued this strategy, were you just trying to keep the project manageable, or did you hope to build your own modding platform to enable “mod mods” based on Fall?

Derek: Just trying to keep it manageable.  I wanted to minimize the cost of losing any member.  There are too many mods that have great ideas but wait for a programmer, get one and do a bunch of good work until the programmer goes on to something else and leaves the project in limbo.  At that point, it’s hard for a new programmer to step in (or if he does you start to run into compatibility issues), and all the effort that went into getting the mod to that point is largely wasted.

I was surprised to see people using Fall as a base for their own mods.  People ran into problems using it because we had hard-coded so many things in the source code.  So for example if we wanted something special to happen when a unit entered a plot with an Ancient Forest in it, we had that right in the code.  It was quick and easy to program and worked fine, until a mod mod removed the Ancient Forest from their version.  At that point, they would get crashes.

When Beyond the Sword was released, we decided to rewrite Fall from the ground up.  I didn’t just copy-and-paste the code over; I started with an empty mod and began adding features from scratch.  I had learned a lot about programming since the project began, and this time we did it the right way.  As we progressed, we removed all hard coding from the mod.

Soren: So, in essence, Fall went through a similar transformation as Civ did, in that much of the open platform philosophy we adopted with Civ4 came at the request of our modding community for more openness. Where do you think modders will take Fall in the future? (For example, I wasn’t entirely surprised that a fantasy RPG mod became one of the most successful Civ4 mods as the XP/levelling system we added to the base game for promotions certainly suggests a classic RPG structure.)

Derek: I suspect “more of” mods to be created.  More units, more civilizations, more of everything.  Fall is particuarly rich for this because the development has been open, so people have access to all the mechanics and ideas we tried.  This makes it particularly easy for them to reimplement things we cut, especially if they can implement them in a way that solves the reason we cut them.  I also wouldn’t be surprised to see modmods based on popular fantasy sources, like Tolkien.

There is a lot of backstory in Fall, and I’d be excited to see if people developed mods based on the Second Age (age of magic) or the Fifth Age (that comes after Fall‘s timeline).  It would be cool to see others take on those time periods.

Soren: By the way, the XML parsing and python glue were late additions to the SDK. I found myself fiddling with that code a lot, so I figured modders would need to as well. How important were those chunks of code? What did we not expose that you wish we did? Were you able to get some of these with Warlords and Beyond the Sword?

Derek: The great thing about releasing that part of the source code is that you allowed us expose things ourselves.  So we created our own XML files, and we exposed our own python functions.  We weren’t limited to just the things you planned for us to be able to mod.

However, the source code doesn’t include everything.  The only parts that I often want to mess with and don’t have control over are in the graphics engine.  Alex Mantzaris, lead developer of Beyond the Sword, was good enough to add civilization-specific unit art to BtS.  Before that, since an art definition was applied directly to a unit, we were carrying hundreds of units in the mod just for their art changes (if, for example, we wanted a Lanun scout to look different than a Malakim scout). He also added a function for the unit art in the source code for BtS version 3.13 so that I could add some code to have promotions effect the units model.  Thus, an orc warrior would look differently than a human warrior regardless of what civilization it belonged to.

I would still like to have more control of the height maps (for creating canyons and cliffs), lighting effects, dynamic ground fog, dynamic scaling and a bunch of other graphic interface functions that aren’t in the source code.  That’s about the only area where I wish we had more access.

Soren: Let’s talk about some actual work figures here, so people can get a sense of what it takes to work on a successful mod. How much time, weekly, do you spend working on Fall? How much time – on average – would you estimate that the rest of the team spends working? How has your team been able to find this time for a fully voluntary project?

Randy: Does playing count as work? Probably, as we are often the first line of defense as far as play-testing goes. I wouldn’t call most of what I do as work, compared with scouring code for hard-to-find bugs or implementing new features as Derek does. But between playing, discussing things on the forums, and creating content, I probably average about five hours a week over the time I’ve been contributing – although oftentimes, that much in a day as well.

Tom: I’d hate to add up how much time I “worked” on this mod.  In the beginning, I played 20, 30, maybe 40 hours a week.  I still playtest over 10 hours a week on average and will continue doing so even after we wrap up Ice.  Who knows how much time I spend on the forums or daydreaming about FfH.

Derek: I spend about 15-20 hours a week on Fall from Heaven.  Multiply that by 3 years and you have a little over 3000 hours.

The initial investment of time was easy because my work sent me to Europe for 3 months to work on a project.  Away from my family and friends, I spent most of my free time working on Fall from Heaven and made significant headway during that period.  Our first version of Fall from Heaven II was released at the end of those 3 months.  Outside of that, even though 15-20 hours a week is a lot of time, I really try to make those hours as productive as possible.  My first major project for the mod back during those 3 months in Europe was in creating an editor that allowed me to easily manage the XML assets of the mod.  Although it took a lot of time to initially setup, that has been a great time saver in the long run.

We also have to be careful about where we spend our time.  There are lots of things we could do, but each has to viewed against the time to create.  Some very good ideas have to be left behind because their value in comparison to the implementation time just isn’t worth it.  I try to be even more conservative when I’m asking for other peoples’ time, such as a cool idea that would require the art team to create a bunch of new unit models.

Soren: Can you go into more detail about what your XML editor lets you do? Have you released it for other people to use? If so, have other people expanded it? I find it interesting to hear that you made the upfront investment to create one because we had a on-going internal debate trying to decide whether or not to build one for our modders. For Civ3, we did build an editor which handled everything that could be modded, but it became a major resource bottleneck because everything we needed to expose required editor/UI work, which ultimately limited the number of things we could expose to modders. More importantly – and this is something we couldn’t really admit in public – we didn’t think we could build as good an editor as our modders could because we would be build for a theoretical spec whereas they would be building an editor to meet their specific and concrete needs.

Derek: Its an XML spreadsheet with Visual Basic macros written to write the xml files based on the spreadsheet data.  I did release a version to the community that doesn’t have any Fall from Heaven attributes in it (http://forums.civfanatics.com/showthread.php?t=231917).  It worked out well for some people, but it doesn’t make modding Civ4 easy, which is what a lot of people want.

This is the dilemma between providing powerful modding capabilities with your game, or making it easy to mod.  Those are two very different goals.  To be easy to mod, the game needs to make most of the assumptions for the modder, allowing the modder to change a few thing and have the game be smart enough to work out all the details.  Each of those assumptions the game makes limits the modder.  The most powerful modding tools don’t make any assumptions and allow the modder to handle all of the specifics on his/her own.  That way, they can do things the game developers never intended, but they have to really understand all of the nuts and bolts of the gaming engine to do it because they have to put together all the pieces themselves.

An editor like you describe would have allowed people an easy first step to get into modding.  A front end that allows them to tweak and customize a bunch of set parameters.  It would have been a great tool for “casual modders,”  but it wouldn’t have been any value to total conversion mods like Fall because of how much we customized the game.  I know that the first step into modding is a big one, so I can’t say how much a Firaxis-created utility would have helped grow the community.  Even though modders can create editing utilities on their own, the problem is that most modding begins in isolation – someone playing around with their own game.  A typical modder won’t go out and grab modding tools for a game he doesn’t know how to mod; he will stop when he can’t find a way to do it himself.  An editor may have been a good first step to allow someone to create a basic mod, then come into the community to find out everything else that is out there

Soren: By the way, what is your day job? And do they know about your extra-curricular activities?

Derek: I work for a business software company in support.  I’m assigned to a few major accounts, and it’s my job to quickly resolve problems they have and to make sure problems don’t happen in the first place.  I work from home, or occasionally on-site, if someone has a specific need, and I’m on call at all times.  My schedule is largely determined by my customers needs – if an issue comes up, I may be working on it non-stop for that day or over the course of several days.  That makes it difficult to plan my life, but it’s the perfect job for modding.  I need activities I can tinker with when I have a few hours free and then drop if I get a call.  I could never play online games where other people are expecting my help at specific times or pursue a career as a costumed crimefighter with any sort of elaborate costume that I would have to get into and out of if I got a call.

Because of my job I have a very support-focused mindset to modding.  My code is not complex, partially because I’m just a beginning programmer, but also because my job is to help my customers remove the risk and complexity from their systems.  One of the amazing things about the Civ4 source code is that it is so simple and readable, even for a guy with no programming experience.  Simple systems working together in complex ways is the foundation of Civilization, and that was reflected in the code as well.  I tried to maintain that in our additions.

A few people know I made a mod for Civ4.  Everyone has their hobbies though, to be honest, since I don’t go into an office, I don’t have the watercooler moments like I had in former jobs.  So, it’s a pretty small group, but I occasionally hear from people I know in real life who ask me if I’m the Derek Paxton from Fall from Heaven.  Usually, as a result of them googling my name and finding the IGN interview about Age of Ice.  In real life (as compared to my virtual Fall from Heaven teammates), my family is very supportive and has endured endless hours of brainstorming and ideas over meals, walks or drives.  To their credit, they always offer good ideas and really think about what I’m saying, which is probably better than I do when my wife starts talking about something in which she is interested (note to self: be nicer to my wife).

Soren: It’s interesting to read that a couple members are using Fall as a pathway into the “real” game industry. How much of a motivation is that for modders in general?

Stephan: Right now, I am writing my diploma thesis in Chemistry, but a year ago, I have been considering finding a job in the gaming industry as an artist. I was seduced because I discovered this passion for 3D art, painting textures, etc. and everything felt so easy to learn. However, I think it is a bit dangerous because if there had been an offer from someone, I would have dropped my studies (3 years of chemistry) and moved.

While modding, you have endless freedom, which makes it so much fun. With a mod like Fall, you also have something like “semi-professional” status, with fans, publicity, etc., which gives the whole work another level of excitement. But you can´t compare this to the pressure one has to withstand in a professional company, influenced by publishers, monetary thoughts, etc. Most skills a modder have are self-taught, which is amazing if you think about it. However, to keep up in speed and quality with someone who had professional training and a degree in arts/programming is a challenge.

So I think hiring a modder will give you certain benefits – he has already proven that he is willing to learn a lot of stuff by himself to achieve what he wants. He will come with a variety of other attributes and knowledge not related to gaming (because he usually does gaming/modding in his free time, not as a primary course of study). He will most likely show a good degree of creative thinking and self-organizing skills. The major drawback is that he is lacking professional education and might be overwhelmed with new tasks, which require a profound understanding of all the details taught in professional 3D modelling/programming classes, and the routine developed in the course of working the whole day in this field (and not in a lab, an office, etc.).

Tom:  From the moment I played Fall, I wanted to try and contribute graphically somehow.  I had never touched a 3D modeling program, or GIMP for that matter, but that didn’t stop me from ‘skinning’ some ugly hellhounds and turning Great Prophets into wizards.  With the help of some very talented members of CFC, I finally learned how to model, texture and put my own custom work into Civ4.  Without Fall, I don’t think that ever would have happened.

I don’t have any plans on quitting my day job to join the gaming industry, but it is nice creating stuff and instantaneously being able to see it used in-game.

Soren: I’m curious where you guys got some of the static assets for the game, such as the leader portraits or some of the extra music. How much stuff have your “plundered” vs. built yourself?

Derek: Almost all of the 2D art and music is copyrighted material that we are using without the artists permission.  If any artist ever disliked how we used their work, I would remove it from the mod.  These artists are more than filler, they inspired the D&D game it is based on, and the art of Justin Sweet, or the music of Cirque du Soleli, inspired the Mod.  So I have great respect for them, and I hope the mod highlights their talent.

The artists, like Jason Engle, that have returned our requests, or that I’ve had the pleasure to talk to, have allowed us to use their work.

For music, we’ve drawn the line at only including small samples (10-15 seconds) of copyrighted music.  Anything longer will only be included if the artists gives us their permission or the artist makes it freely available on their website.  We’ve been particularly lucky that Joseph Vargo, leader of the group Nox Arcana, has been kind enough to allow us to use their music.  It’s an amazing mix of Gothic fantasy music that fits Fall well.

See also Part I and Part III…