Sprinting

I didn't get anything done last week. Nothing at all. Most of my days were spent reading random stuff on the internet, making minor tweaks to Blogabond, and obsessing over traffic stats for Twiddla (why did 5000 people suddenly show up from StumbleUpon in one day???)

This week, on the other hand, I've been on fire. In the last 4 days, here's what I've accomplished:

• Built a Reddit clone from the ground up for Rootdown's soon-to-be-live Clinical Pearls section.
• Built a Google-Maps powered acupuncture chart, cut up 3000 tiles for it, incorporated effective Lat/Lon coordinates into Rootdown's database, and built a little Ajax data entry tool to drag & drop acupuncture points and meridians onto the chart.
• Reworked the Photo uploading and Photo management pieces of Blogabond.
• Tore out and streamlined the installation process for Regressor.NET
• Wrote this lame article

Trust me, that's a lot of stuff.

I've noticed this same pattern happening over and over again. I think of it as Sprinting, and I think I'm getting better at harnessing it. There are a few factors that play into it, but I think the key is knowing that I have an entire day to Sprint on whatever it is that I want to do. Knowing Absolutely that the door to my office won't open, the phone won't ring, and no little IM popups will bother me for the Next Twelve Hours. Knowing that I'm free to get as deep into what I'm doing as I need to get whatever I'm doing Done and Done for good. Those are the days that I get the most accomplished.

Sprinting for a different reason:
Team Expat, Running with the Bulls in 1998

Another thing that seems to help, at least for me, is to have more than one ball in the air at a time. When I've only got one project, I seem more content to move slowly, check my email, read the occasional blog, and essentially stuff my productivity. But when I've got 3 things the Need to Get Done, and there just aren't enough hours in the day to do it all, I find that I work a lot faster.

Better still is to have something ELSE that I should really be doing. You should SEE the days I spend blowing off paid work to Sprint on a side project.

But it more than just blowing off one project for another. The real advantage of having more than one project going at a time is that if I get blocked for any reason, I can switch over to another project and continue Sprinting at the same pace. If I'm motivated to move fast, it doesn't really matter all that much what I'm moving on, provided I'm moving. If I can ignore the little bottlenecks and keep Sprinting until the inspiration fades, I can get a lot more done overall. If I only had a single thing to work on, any minor distraction, such as missing graphics from a screen designer, could derail me and send me off to check Reddit (and thus get stuck there for six hours.)

I don't think that any of this is new. It's common knowledge that developers tend to work in bursts. I guess the difference for me is that I'm starting to work on ways to facilitate those bursts. To keep them going once they get started. To finally look up and find it's dark out, and I haven't eaten for 16 hours and wow, did I really get all that done in a single day??? That's where I want to be. That's Sprinting.

Getting Your Priorities Straight. Scalability and Performance are the Least of your Worries.

Back in my Contractor days, I would occasionally take a job bringing a bunch of C++ guys up to speed in C# and ASP.NET. Invariably, I would have to break them of old habits that they had picked up back in the days when memory and hard drive space were expensive, and applications had to run in real time. Most of these little battles were quickly won, so flat files were replaced by relational databases, bit masks gave way to association tables, and data access code was pulled out into its own layer.

But one thing never went over well. Performance. Speed is largely irrelevant for a web application. Sure, it's important that your thing run fast, but there are a half dozen other things that are more important for a big web application. This is difficult to hear if your major skill is writing inline assembly for critical routines, but it's still the truth. Readability, Debugability, Maintainability and Development Pace are much more important than raw speed.

To deal with this rift, I would ask the developers to list out the most important qualities of a piece of software, and to rank those qualities in order. I've hinted at my answer above, but I'll take a few minutes to list them out below. Everything you see in the list is important, but the things toward the top are relatively more important than the ones towards the bottom. For what it's worth, we're talking about Web Applications here, so clearly this list does not apply to Game Development or even Windows Apps. Here goes:

Readability

In my mind, this is the single most important quality of a piece of software. Assuming your thing is going to be around for a while, you're always going to need to return to a given piece of code from time to time and make modifications. The faster you can read and understand what's going on, the sooner you will be able to start making modifications and adding new functionality. Better still, if you can quickly figure out what the code is doing and why, you'll be less likely to break anything in the process.

Debugability

Your code is going to break. Often. That's how it goes, so you'd better structure things so that it's easy to step through and figure out what's going on. That means declaring variables instead of stringing together 17 object methods on a single line. That means using real IF/THEN/ELSE blocks with squiggly brackets instead of inlined immediate if's. And it means thinking twice before committing to some automagically generated database framework that sniffs out all your column names, writes its own SQL, and keeps your data in ArrayLists of ArrayLists.

Keep your design simple enough that any exception will drop you into the debugger looking at a single line of code that does a single thing. Even if it turns out it's doing that single thing wrong, at least you'll be able to find and fix it.

Maintainability

Over time, new features are going to get added and old features are going to get dropped. Some of those new features will be stupid ones, with dorky business logic that rubs the fur the wrong way in your elegantly designed class structure. You want to be able to make those changes quickly, without breaking anything else. This means you need unit testing. You'll also want to refactor large sections of your backend to work in ways you had never anticipated, and you'll need to propagate those changes all the way out to the client code. For that, you'll need even more unit tests (and some good tools), but also you'll need an architecture that doesn't fall apart when you rip chunks out of it.

Development Pace

Modern applications are big and complicated. It doesn't matter how nicely written your thing is or how many simultaneous users it can support if you never manage to get it out the door. If you want to get your application shipped, you're going to need to put out a ton of code in a hurry. That means you're going to need the best tools available, and the most productive environment that you can find.

Side Note: PHP might seem fast if you've never seen the alternatives, but let's see how many Ex-Ruby-on-Rails and Ex-ASP.NET guys you can find doing PHP development by choice.

Keeping the above points in mind, you're going to want a development framework of some description. Here at Expat, we've rolled our own specifically to keep us fast without sacrificing Readability, Debugability, or Maintainability. I'd recommend doing the same, but there are any number of 3rd party frameworks out there that might fit the bill. Just make sure you keep those three qualities in mind when you are evaluating any new framework.

Scalability

At some point, your thing is going to get popular. Actually, chances are it won't, but you shouldn't architect your thing to preclude the possibility that people might start using it in the Millions. So how do you pull that off without undoing all those Important Things further up this list? Simple. Just be aware that one day you might need the ability to add more database and web servers to the mix. Add a few little abstractions such as a Database Connection Factory, and a Session wrapper that you can replace someday with something BEEFY. For now, they don't have to do anything fancier than wrapping the existing stuff in whatever framework you're using. But if you're diligent in using these wherever you would normally use the framework components, you might end up saving yourself a lot of headache down the road.

For the most part though, don't worry too much about scalability. Having a million people that want to use your thing on a single day is a good thing. If you've done a little homework, you'll work things out when the time comes.

Performance

Computers are fast. Seriously, computers are faster than you think. If you try to imagine which piece of your application is slow, you're probably wrong. I once worked with a developer who spent the better part of 6 months hand optimizing an algorithm to do fast fuzzy string comparisons. It turned out that the server doing the text processing was only spending about 10% of its time actually processing text (even with a simplified, non-optimized algorithm), and 90% of its time battling database locks to get the results put away. He could have figured this out in one day with a profiler, and then spent a few hours tweaking database indices and optimizing queries. Instead, he spent half a year solving the wrong problem.

So yeah, keep a profiler handy, and if you see something that is obviously taking a lot of extra time, go ahead and fix it. But don't spend too much time sweating performance issues. At least, wait until they present themselves as issues before you start sweating them!

Life imitates Rant...

As I write this, Blogabond (one of my diversions from real work) is starting to show its first signs of scaling pain. Every once in a while, a misbehaved crawler will swing by and hit it 500 times in a second, causing SQL Server to time out on a specific long-running query. This is a good thing in my mind, as it gives me a chance to tackle a potential bottleneck before it starts affecting real users.

Still, Blogabond has been up and running for almost two years now, and it is only now that I'm having to think about performance at all. Those other qualities though: Readability, Debugability, Maintainability, Development Pace. I'm seeing benefits from them every day.