Programmer's Toolbox

I’ve been working on a web project that has gone through multiple iterations, and many hands, before landing on my desk. Over that time (and while it’s been in my care, as well as before), the stylesheets have acquired quite a few geological layers. The primary CSS file has had over 2,000 lines, with more than 600 selectors spread over almost a dozen stylesheets. Since the site has gone through multiple revisions and adjustments–at least five major versions in the month or so I’ve been working on it–a lot of these selectors aren’t being used anymore.

This is a classic case of the Lava Flow anti-pattern:

When code just kind of spews forth and becomes permanent, it becomes an architectural feature of the archeological variety. Things are built atop the structure without question and without hope of changing what is beneath them. The existing code is seen as an historical curiosity.

The best thing to do with lava is to blast it away, leaving just the useful bedrock. But removing unused CSS selectors is a tedious and error-prone undertaking; my Java IDE can tell me which methods and variables are no longer in use, and profile an application for redundancies and deprecated components, but I didn’t have a comparable development tool for styles. Luckily, my core development philosophy–somebody smarter than me has already solved this problem, and has probably put the information out on the Internet–panned out, and I discovered the Dust-Me Selectors Firefox add-on.

Dust-Me Selectors can look at an individual page, or at an entire site, and tell you which selectors are actually in use. It generates a nice report, broken down by stylesheet file and including line numbers, that will direct your blasting caps and jackhammers at the problem areas. After running it through my pages, I was able to reduce that main CSS file to just over 1,000 lines (it’s a complex site), with less than 200 selectors defined.

The tool can be run against individual pages or against a sitemap; for a large site, a sitemap is by far the best approach. If you don’t have a sitemap.xml file yet, it’s relatively easy to generate one, either with your development tools or CMS, by hand for a short list of pages, or with a simple script. I found that it was less reliable in single-page mode: your coverage won’t be as good, and things like Ajax windows are going to get in the way of accurate results. A comprehensive sitemap ensures coverage.

It’s also a good idea to use caution when pruning the styles; I went through the list of flagged selectors and disabled them with comments, visually testing the site as I went along. There were a handful of cases where Dust-Me flagged styles that were actually in use, primarily within jQuery code that was manipulating element classes on the fly. After I verified that my leaner, meaner CSS files were working, I moved all of the commented code out into a new file just in case some of those selectors need to be resurrected in the future.

There are a few reasons to tidy up your CSS files. Download performance, though not as much of an issue in the broadband world, is a factor: my old core file was about 41KB, and my new one is 17KB. Browser performance, too, is a factor: parsing unneeded CSS code may be a very tiny drag on performance for faster machines and browsers, but every millisecond you save on unnecessary parsing is time you can put the browser to work doing something more interesting and useful.

The best argument for cleaning up the lava flow, though, is maintainability. Scrolling through those 2,000 lines of code in search of an errant style, or trying to debug a rendering issue when looking across a dozen files for a class or ID, is a drag on productivity. In the heat of battle, it’s far too easy just to add another selector to the bottom of the file, which only contributes to the chaos. Pruning the excess weight early in the development process makes it much easier to maintain clean code as the project moves along.

I’ve found that putting together a very rough architectural sketch is the best way to start when you’re building an application from scratch. While it’s a lot more fun to write code and see it executed, you save yourself a lot of grief, and create a more robust and flexible system, if you think things through up front and draw some boxes and arrows.

For this project, I knew that I would have at least three tiers: the user interface, which would initially run on WebSphere Application Server but should be portable to WebSphere Portal Server; the business logic layer, which would do the heavy lifting of taking the user’s input and sending it to the right back end screens; and the back end screen interface itself, built on an existing Java bean architecture.

I also knew, from painful previous experience, that keeping those tiers separated was critical. Java makes n-tiered development easy, with the ability to control scope and encapsulate functions, but it also makes breaking a clean separation of tiers very easy, too. Sloppiness early on, in making too many methods public rather than protected or giving one class too much access to things outside its logical scope, can doom your application. So that initial architectural sketch should include an attempt to block out the domains of interest and identify what limited interfaces should exist between different parts of the application.

I ended up with something roughly like this:

The service and client layers are implemented as separate web applications: though they are packaged in the same enterprise application for ease of deployment, they are independent of each other and could be deployed as individual WAR files. Each domain surrounded by dashed lines in the table above represents at least one Java package, with most methods set to protected rather than public: this limits the points where changes in one layer can affect the code in another, and makes swapping out different services (caching, configuration, etc.) relatively easy. Building clear walls between application layers in the beginning will help you in building out and maintaining the application in the future.

In the original version of the application, developed on Eclipse and running on Tomcat (I didn’t yet have access to the licensed versions of IBM’s tools), the interaction between the service and client was handled through Axis, a Jakarta web service infrastructure. In the final version, running on WebSphere, the interface is handled through JAX-RPC, the default mode for WebSphere. My first attempt to deploy the application to WebSphere was, in fact, botched by the use of Axis: WebSphere wants to do JAX-RPC when it does web services between WebSphere servers, and was very unhappy about Axis. Luckily, both Eclipse and Rational (IBM’s version of Eclipse) have handy wizards for generating the web service bindings, so switching the web service layer was relatively painless and, since I started with the practice of separating the application into distinct domains, required no code changes.

In developing a SOA application, a lot of attention should be paid up front to designing your service interfaces. Keeping your changes to the service interfaces to a minimum will make your development a lot smoother–no need to re-generate the service and client bindings if you keep method signatures in place from version to version–so it pays to do as much analysis up front as possible.

For this application, I knew that I would be passing a user-generated data bean from the client layer to the service, sending that bean through various permutations in the service layer to do all of the back office work, and then sending that bean back up to the client to show the user the fruits of their labor (and any error messages that might have been picked up along the way). So the first order of business was designing the data bean, which would act as the primary transfer object, and the methods that would accept it.

The data object had to be very flexible, since the back end system is so denormalized: multiple fields and value types would have to be carried by the bean, not all of them well known at design time. The solution was to make that bean into the carrier of a collection of field objects, which have name, value, and type (since passing raw Objects over web services is forbidden) attributes. This allows the bean to pick up whatever fields it needs to feed the back office system, without having to change the code any time a new field is discovered. With simple getField(String fieldName) and setField(FieldObject field) methods, any value can be put on the bean.

Another consideration in developing your service interfaces is data typing. As noted, you can’t send a plain Object over the wire and expect to be able to cast it to the specific type you want. Any object that is transferred over the service needs to be mapped through XML, and so should consist of serializable or primitive members. I’ve found that sticking to Strings, ints, longs, decimals, and such is by far the safest way to work with web services; more complex objects can be generated from the primitive transfer objects after they’ve reached their destination.

Once the service interfaces were in place, and the demarcations within each application were defined, it was time to start wiring things together. And that’s the fun part!

Since September, I’ve been developing a Java application for a client in St. Paul. As noted in an earlier post, this has been a very successful project; I’m quite proud of the application, and with the quality of the work that we’ve done as a team. It’s unusual for a corporate IT project to be on time, under budget, and to deliver real functionality, so being involved in a project that hits all three is quite a treat.

The code in this project was bought and paid for by the client, for use in an application that runs behind the firewall, so I won’t be sharing any of the source here. But I think there’s value in sharing some of the patterns and principals, the architectural strategies, and the methodologies that have led to our success, and also in calling out some of the open source projects from which we’ve borrowed. In this series of posts, I’ll provide highlights of the project that have made it especially fun.

My role on the project is winding down at the end of the month, and I’ve started searching for a new gig. I can only hope that my next project will be half as enjoyable and successful as this one has been.

The Problem: So Many Screens, So Little Time

The application is used by a business unit responsible for setting up the licensing, maintenance agreements, and billing for customers. Before this application went into production, they used a series of browser screens that expose a 4GL application (extended, I believe, from a terminal screen application). To set up a customer order, the users would have to visit a dozen or more screens, often entering the same information into each one. Behind the screens is a highly de-normalized database.

Though the 4GL application contains some validation rules, there are other business rules that were really more like uncodified traditions: the users could very easily enter “incorrect” data into the screens, because there were no program controls in place to control their input. Data entered in one screen is dependent on data entered in other screens, resulting in a lot of back-and-forth in the system to gather up all the information that needs to be submitted. Training in a new user on the Byzantine ways of the process was difficult, and errors were frequent and painful to correct.

The primary goal for the application I’ve been working on, then, is to increase the users’ productivity and accuracy by giving them one form to complete. All of the manual lookup steps, all of the rules for deriving certain values and codes, needed to be automated to simplify and streamline the process.

Despite its limitations, though, the 4GL system is of great value to the business processes surrounding the data entry. It’s used to generate reports and billing, it serves as a repository of customer contact and relationship information, and it feeds into the business’s bottom line. Replacing it was not an option; the new solution would have to work with the 4GL layer, and be compatible with the rest of the business processes.

Another requirement going into the project was the separation of the business logic and UI layers. There was a strong interest in eventually using WebSphere Portal Server as the front end (not yet implemented, alas), but in the interest of meeting the schedule the first version of the application would be delivered as a stand-alone web application running on WebSphere Application Server. To avoid reinventing the wheel and introducing a lot of regression errors, we started off with a service-oriented architecture: the business logic would be implemented as a web service, with the front-end web application driven entirely by the web service layer.

The biggest challenge in getting this application started–the communication with the 4GL layer–had luckily been handled in a different project, so I was able to build on a successful set of APIs instead of developing my own methods for interacting with the back end screens. The 4GL programs expressed by the screens have an XML-over-HTTP interface, and a developer had already built a very nice tool for converting the XML into Java beans. The challenge of wiring together all of those screens, then, fell to me. And that’s the real heart of this application: marshalling a collection of disparate Java beans into an orchestrated process that does complex things using simple tools.

I’ve worked on a variety of projects in my IT career, from small Lotus Notes applications where I’ve been the business analyst, developer, tester, system administrator, and support engineer, to multi-year enterprise initiatives with far-flung teams and huge project management systems where my role has been very narrowly defined. I’ll admit to a preference for the former–I like to write code, and waiting weeks or months for the first solid requirements to trickle in is painfully dull. But all of the projects, like most IT projects, have been plagued by the usual disconnects, missed opportunities, and frustrations with delivering what the users really want on time and on budget.

My current project, though, has been surprisingly successul. We released our first version in November, after two months of development and testing on top of about six months of thorough analysis (most of which happened before I joined the project), and since then we’ve released new and improved versions on a monthly schedule. The customer has been pleased, the application has been solid, and we continue to meet the users’ expectations.

What’s the secret?

To a great extent, it’s due to a very talented team of developers, testers, project managers, analysts, and business users. We work together well, have open and honest communication, and set up realistic and reachable goals for each release. The problem with talent, though, is that it’s not necessarily reproducible; you can’t bank on having good people in every role, or even on having good people at the top of their game most days. A project that relies on talent alone is bound to fail eventually.

What has really worked for this project is a philosophy of continual improvement. Our driving principal has been, to borrow a line from Jeff Atwood, version 1 sucks; ship it anyway.

My current workplace doesn’t have a formal “methodology” for development, no waterfall gate-checks or SCRUM masters, at least that I’ve encountered. There are rudimentary project controls and such to meet corporate governance requirements, but development teams are left largely to organize their own efforts. As a result, we’ve landed on some practices that borrow heavily from various flavors of “agile” development without professing the full “agile” theology; the guidelines that I’ve found work best on this project, and that may be reproducible on other projects, are pragmatic and contingent, flexibly implemented within a loose framework. This may not work everyplace, on every project, and it doubtless has some scalability issues, but for a mid-sized project with an aggressive schedule, these are some practices that have worked for us:

Manage the requirements to the schedule: hit the dates by containing the enhancements

We have a huge list of things we’d like the application to do, ranging from simple tweaks to pipe-dream fantasies. They’re all good requirements, all worth meeting because they represent what the users really want. But they’re not all going to go into the first, second, or third release.

Instead, we’ve promised a monthly release with at least one major system enhancement, and as many smaller enhancements as can be realistically squeezed into the time frame. Like the big rocks fable suggests, we focus on the one big thing first, and then categorize the other requirements as hard, challenging, or low-hanging fruit. Once the big requirement for the next release is ready, we knock off the smaller requirements as time permits, always mindful that no small enhancement should jeopardize the big one. It sucks to leave low fruit on the branch, but we keep our spirits up in the knowledge that we’ll have a long harvest season if we keep the customer happy.

A little spice and sizzle helps, though

The “one big rock” is usually a meat-and-potatoes affair, and it’s always filling and nutritious. But we’re also sure to include a little spice among the smaller enhancements. Refreshing the style sheet, adding a more attractive screen layout, or providing an extra screen of administrative information on the application’s performance is often cheap, easy, and low risk, but it’s very useul for maintaining customer satisfaction. The users may not notice that you’ve shaved an average two seconds off the web service response time and implemented a really nifty sorting algorithm–indeed, you’d better hope they don’t notice those things, because their only evidence should be when they fail–but they’ll ooh and ahh over a nicer interface.

Track every requirement, no matter how small

Indeed, make your requirement-tracking as granular as possible. Break the big requirements up into bite-sized chunks, and build good estimates for them (this is where something like the Pomodoro Technique can really shine). You don’t know which rocks are big and which are small unless you track them, and you don’t know if you need to scale back your release features unless you do estimates.

Open up the black box and let everyone see the work list

Having a good requirements and bug-tracking system is critical to managing in a progressive-enhancement environment. We’re using FogBugz, but other tools–Roundup and Bugzilla come to mind–are also useful. Even a shared spreadsheet is better than nothing. The key requirement is that everything is on the table and visible to the entire project team; having the project progress available at a glance, and maintained in real time, is the only way to keep everyone honest and ensure that releases happen on schedule.

Build plenty of testing time into the schedule

I’ve known thin-skinned developers who don’t like testers. Personally, I’d rather have someone on my team find my bugs before a customer does: it’s easier and cheaper to fix problems before your release date, and a good, thorough tester can be the difference between a product that people love and one that makes their jobs harder. In our current project schedule, we have a code cut-off date a week before the release date, after about three weeks of development; this should be adjusted for larger and more complicated projects, with even more time dedicated to serious testing.

Release early and often

That “three weeks of development” in our project is really three weeks of development and testing. As soon as you have something to show, even if you know it’s not ready for release, get it out there for your testing team to break. If you’ve got users who can spend time looking at things that are in development, so much the better: unvarnished responses to early iterations can flesh out requirements and ensure that you’re meeting the customer’s needs. There’s nothing worse than releasing something that’s been carefully developed, thoroughly tested, and still misses the customer’s core requirements. During the last two weeks of development on my current project, I’m deploying something to the shared development environment nearly every day (and if I had an automated build and deploy system, I’d be checking in updates even more often).

Build a solid architecure in the beginning, and build out modularly

My current project lends itself well to continual improvement, because it was architected from the beginning to be modular. It’s a service-oriented architecture that uses the Apache Commons Configuration framework to abstract the business logic into XML documents. It’s developed to Java interfaces and abstract classes as much as possible, with an eye toward identifiying and reusing patterns; if something can be accomplished through XML rather than code, that’s the direction we go.

SOA is a good fit for continual enhancements because the application layers can be clearly separated from each other; you’re less likely to break something if you don’t have to touch it. But the same principals apply to any development platform: make code small, abstract, and reusable, and avoid great big tangles of spaghetti. If you can’t see the whole method on your screen without scrolling, don’t adjust your monitor resolution: break the code up and look for the patterns.

The next release will be better

Whether the customer is pleased as punch, or grinding their teeth in angst, that’s the appropriate response: the next release will be better. The requirements we missed in this release are first on the docket for the next; the new requirements that have emerged from the testing rounds have been captured and scheduled for future deployments; there’s nowhere to go but up.

Provided that you set a pattern of actually delivering on this promise, the customer will be willing to accept that they won’t have the perfect system out of the gate. And if the customer is involved at every stage, and has a hand in the requirements triage and testing, they’ll be happy to play along with incremental enhancements. Something is almost always better than nothing, and unless they’ve got the self-control of toddlers they’ll be willing to defer some of their gratification, especially if they get a little taste of real improvement at each release.

I can imagine projects where these guidelines would fail to deliver: big enterprise initiatives with lots of interrelated parts are hard to release quickly in small pieces. At the same time, though, this may be as much a matter of perspective as of scale: if the project is too big for continual enhancement, maybe it’s really two or three or ten projects that need to be broken up and managed independently, with a longer test and integration period set aside to mesh the components. If it’s possible to deliver a little bit more often, rather than a lot after a really long time, my bias is toward the former: it keeps everyone working, ensures that real requirements are identified early, and shines some light into IT’s darkest boxes.

I’m embarking on a little experiment with netbook operating systems; there’s a lot of interesting activity in the lightweight OS world–Ubuntu, Jolicloud, Chrome, Android, etc.–and I’m curious how the different systems stack up. Each weekend from last weekend until I run out of operating systems, patience, or time (or land on the ideal OS paradise and never want to leave), I’ll be installing a new platform on my Dell Mini and rating it on usability, stability, features, and other criteria specific to the netbook space.

One of the things that has kept me from making big OS changes on my netbook in the past is that getting Firefox reconfigured is such a hassle. While I’ve been using the Xmarks plugin for bookmark and password synchronization between home, work, and my netbook since the plugin was called Foxmarks, getting all of the other plugins installed and configured after wiping out the netbook has been a tedious chore. About 90% of what I use the netbook for is browser-based, so this is a relatively big deal for a little computer.

The solution that I’ve landed on is actually pretty simple, and uses two nice utilities in concert.

First, there’s FEBE, the “Firefox Environment Backup Extension,” a nice Firefox plugin. FEBE will create backups of whatever Firefox components you choose–plugins, themes, bookmarks, cookies, etc.–and restore them. You can set it up to do scheduled backups, restore settings into a new profile, and manage selective backup configurations.

And then there’s Dropbox, an online file storage and synchronization service. I’ve been using it to easily synchronize writing projects between my Windows PC and netbook, and it works like a charm: silently synchronizes the files that I place into its directories, and seamlessly integrates with the file systems on both my Windows and Linux computers.

Before I uninstalled Ubuntu on my netbook, I ran a full backup of Firefox from FEBE to a directory under Dropbox’s control. Then when I installed Jolicloud, I added the FEBE plugin and installed Dropbox. In just a few clicks, I had all of my other plugins plus bookmarks, passwords, and other browser settings back in place.

I admit, it was a little disconcerting to be suddenly confronted with more than a dozen Firefox tabs for each installed plugin after the FEBE restore ran. But it was a lot easier to close tabs than it would have been to reinstall all of those plugins.

The same concept could, of course, be used with other combinations of tools. FEBE natively supports Box.net, for example, and there are some other tools for doing Firefox backups (I’ve used MozBackup before, which handles the whole Mozilla suite, but it’s a Windows-only utility and therefore not terribly helpful on my netbook).

When I move on from Jolicloud in a few days, I’ll be going through the same steps again, perhaps with a few refinements. Simple is good.

In keeping with my recent interest in the Pomodoro Technique, I’ve been on the lookout for other dead simple techniques to make my work life easier. This time, again via a couple of podcasts (the New York Times Book Review and Lucy Kellaway of The Financial Times), I’ve discovered the power of check lists.

In The Checklist Manifesto, Atul Gawande explores how the lowly check list can save lives in the operating room. By having everyone on the operating team follow a straightforward list of pre-conditions and procedures, the World Health Organization was able to increase survival rates among surgical procedures by 47%. The problem, as Dr. Gawande explains it, is one of extreme complexity: there are so many steps required, so many places where one error can cause severe failure, and the human brain simply is ill-equipped to handle such complexity by itself.

I don’t have nearly as complex a job as a surgeon or an airline pilot, and I’m very happy to say that no lives are held in the balance by my ability to execute my tasks without error. But there are a lot of details in my work, and a lot of places where things can go terribly wrong (on a small scale), and though no lives will be lost, there’s a good chance that my day (or the day of my customer) will be ruined by failure. I speak from bitter experience of overlooking a simple detail and seeing things spin badly out of control.

As a test of the manifesto’s thesis, I put together a check list for my application build process. It’s a terribly manual process, with lots of chances to make a bad build. Yes, it should be done on a build server directly from the source repository (at my last job, we used Hudson: I highly recommend it to anyone looking for a clean, simple, flexible build infrastructure), but as a contractor I’m a guest here and can’t really demand that kind of change. And yes, I could make things easier for myself with an ANT script, but I could also make things easier for myself with DTDs for my XML, more consistent use of jQuery, and some Eclipse plugins for my configuration system, but somehow I have to find time to implement features too; we must choose our battles, and I’m willing to defer this one. We’re running the development of this project on an aggressively iterative time line, so in the midst of development I may have one or more daily builds to release to the customer for review, and monthly deployments to production. That’s a lot of opportunity to miss a step and waste several hours of time.

Originally, inspired by the Pomodoro Technique’s paper system, I thought I’d print out a check list for each deployment. Having a physical object to mark the steps on seemed like a good way to ensure focus. But one of the hallmarks of a good check list is its flexibility and adaptability: my first list was obsolete after the second time I used it.

I looked at a few on-line tools–Toodledo and Remember the Milk seemed like the hot properties in to-do lists–but they didn’t quite fit the bill. The thing that makes a check list different from a to-do list is that it lives in multiple instances: the same steps repeated over and over, rather than tasks that are assigned and executed once. One of my requirements was that my check list be cloneable, while at the same time offering flexibility in adding and removing steps as the process changed.

I finally ended up at Checkvist, which met all my needs:

1. I can create a master check list and copy it any number of times as needed
2. I can add and remove steps on the master template, and can also modify the working copies
3. Tasks can be arranged, rearranged, and put into hierarchies
4. The interface is quick, clean, and simple, and a joy to use
5. It’s free (though there is a paid version, which supports encryption and secure sharing of lists)

I have a master build check list that I’ve used for 10 builds so far; and so far, none of my builds have failed. The list is largely obvious, possibly even pedantic, but there’s a tale of woe behind almost every line: a build that failed because the application.xml file was corrupt, or because an external properties file was for the wrong environment, or because a network hiccup caused the compiled application to copy incorrectly to the deployment server. Most of the steps take only a few seconds to perform, but can save hours of effort to correct.

Dr. Gawande found that surgeons were initially resistant to using a check list. They consider themselves (perhaps rightly) above such things: they are as much artists as they are technicians, and going through a check list seems anathema to their skills and knowledge. And I had a little resistance to overcome myself in giving it a try; I think of myself less as a software engineer than a software sculptor, shaping the formless bits and bytes of data and code into a beautiful machine. But after giving the check list a chance, I found that it actually freed me up to think more about the creative parts of my work and less about the mundane but critical details. The benefits have included:

1. More consistent success in getting a successful build released.
2. Less anxiety: I’m both absent-minded and anxious, the sort of person who returns home after getting half way to work to make sure I really did unplug the coffee maker and lock the doors. The check list gives me confirmation that I did in fact take care of the details.
3. Less time thinking about the details: I don’t have to remember where in the configuration settings I have pieces to switch for deployments to dev vs. cert, or which files point to service endpoints. I’ve spelled out the specifics in the check list, and I update the check list when the details change.
4. Better development practices overall: I’ve been using the Apache Commons Configuration API to control the application’s setup, and the check list encourages me to put environment-specific information into consistent spots in the application. I am also far less tempted to hard-code things might have to change at deployment if I know that they need to go into the check list; externalization is good for maintainability, and also good for making a speedy list.
5. A concrete log of my work: Checkvist lets me archive my check lists after I’ve finished with them, so I can go back over them and verify that each step was completed, and also see how the process has changed over time.
6. A better understanding of the process: when I do get around to using ANT to simplify things, I’ll have a template to work from; steps that could benefit from improvement stand out in the list, and can be addressed before they become entrenched.

Using a check list for this particular set of tasks has encouraged me to identify other things that can benefit from this discipline. Adding configurable components to the application, branching the project in the repository, organizing requirements for future releases: all repeatable processes with the possibility of failure if a detail is missed. And all candidates for freeing up my brain to think of more interesting tings while the check list serves as my memory.

Task management is the bane of my existence. I’m easily distracted by shiny objects, prone to flights of fancy, and generally unreliable about estimating the time required to complete a project. In this, I don’t think I’m much different from the average software developer; we tend to be overly-optimistic about how easy a job will be, forgetting how many blind alleys we wander down before finding the “easy” road to success.

Recently, though, while listening to Rob Long’s Martini Shot podcast¹, I discovered a deceptively simple and surprisingly effective approach to task management: The Pomodoro Technique.

The Pomodoro Technique (named after the whimsical tomato-shaped kitchen timer with which Francesco Cirillo perfected it) boils down to a very simple strategy:

• Make a list of the things you need to finish.
• Set your timer for 25 minutes and do one thing–only one thing, and only that thing–until the timer rings.
• Take a 5 minute break.
• Repeat.
• After four of these 25-minute sessions (the unit is called a “pomodoro”), take a 15 minute break.
• Repeat.

Simple, but surprisingly hard to do.

Multitasking has become the standard mode for most of us in this technologically-connected, ever-accelerating world. If we’re not doing two or more things at once, we feel like we’re not getting anything done. But a great deal of new research suggests that multitasking doesn’t really make us more productive; indeed, it can make us less productive.

Humans, it turns out, don’t run multi-core processors. We’re more like the old Windows 3.1 “multitasking” model: fast at switching between tasks, but really not capable of truly doing more than one thing at a time. As a result, the multiple things that we do suffer when we switch contexts, and we end up doing several things poorly instead of one thing well.

The Pomodoro Technique is explicitly anti-multitasking. During the “pomodoro” period, you just work on your task list: no checking e-mail, taking calls, surfing the web, talking to co-workers. It’s a heads-down, fully-focused sprint to the end of the task, with a much deserved rest at the end. And for someone who’s used to “busy-ness,” it’s exhausting.

My first few days trying the technique were frustrating and tiring. I felt the oppression of the ticking clock, the twitching desire to check my e-mail once or twice just in case, and an exhausted relief at the end of a session. But then I started to internalize the “pomodoro” time unit; I found myself adapting to the rhythm quite naturally, and the time allotted to a task seemed to expand into an ample amount. Indeed, I’ve begun to feel time slow down in a surprising way: if I glance at my timer now and see that I still have five minutes left, I know that I can still get quite a bit done.

The technique also helps me be a little smarter in how I do my job. Before, when I wasn’t letting the timer set my pace, I was prone to investigate many more blind alleys. I might end up losing an entire day to one bad decision, backing out hours of effort. But knowing that I have a limited amount of time in which to finish my task, I now opt for the simpler approach. Rather than re-architect a huge chunk of code, I’ll stop and think things through, and usually find that a simple, elegant, and easily-implemented solution is available.

That five-minute break at the end is just as important as the twenty-five minutes before. It’s the time to take care of the coffee cup and the restroom, to stretch and crack knuckles and read the news, but it’s also time to switch off the task-oriented part of the brain and let the unconscious burble up a bit. If a “pomodoro” ended in frustration, with a task unfinished and more tasks piling up, I’ll often find this five-minute break from the project lets me come back refreshed and more likely to see my way out.

The Pomodoro Technique works very well in some phases of software development, and is easily incorporated into Agile and other iterative methodologies. When I have a nice collection of features to implement with clear requirements, or when I’ve got a set of bugs that the testing team has sent my way, the Pomodoro Technique excels: my tasks are clearly defined, easy to organize, and I can put my head down and get to work. And I could imagine a development team adopting the technique very effectively (especially if they use a timer in meetings: meetings that end when the buzzer goes off! There’s a concept worth implementing … ).

I’m not sure, though, that it’s as easy to apply to every technology job. I’ve had positions in the past that were much more reactive, where my daily work was largely driven by incoming requests and messages from customers and co-workers. In a culture that insists on “urgency” as a core value, the Pomodoro Technique’s managing of interruptions and “protection” of the “pomodoro” could be problematic. Of course, there are more problems than just task management in a culture like that …

I’m also unsure that it is as easy to apply in the more nebulous “design” phases of a project. There’s a good deal of exploration, guess-work, and fiddling about that goes into discovering an architecture for a reasonably-sized application. When I’m developing something from scratch, with sketchy requirements and an unfamiliar environment, it’s difficult to identify the kinds of clear-cut tasks that the Pomodoro Technique demands. And fuzzy, indeterminate tasks are exactly the kind of thing that lead one down the rabbit hole of multitasking chaos.

The other trap that I find myself trying to avoid is biting off less than I can chew. Given the time constraints, and the focus on completing a task (or set of clearly-defined tasks) before moving on, I sometimes defer larger jobs that might span multiple “pomodoros.” I feel a bit like a member of a millenarian cult, unsure if I start darning my socks because the Lord might return while I’m in the middle of it.

All in all, though, I’ve found it to be a productive technique. By turning off the multitasking trap and setting boundaries around my work, I’ve managed to become far more productive in a few weeks than I would have expected. I only hope that no one notices and starts to expect it to continue …

I should note a few of the tools I’ve found useful in exploring the Pomodoro Technique:

• Task list worksheets (and a free download of the book) are available at the official Pomodoro Technique website; I use the paper worksheets and a pen rather than any fancy-schmancy technology, though there are quite a few programs based on the method (even an iPhone app or two if that’s your thing).
• A simple timer program, Egg Timer Plus 3.12, from Sardine Software. Though available in a free version, I recommend spending the $5 to get the license: I created three pre-set timers (25, 5, and 15 minutes), and set different sounds to go off at the end of each. It was about the same price as a real egg timer, and less likely to get lost.
• This whimsical introductory slideshare from Staffan Nöteberg makes the case for the technique, and offers a good thumbnail sketch of its methods.

All in all, a pretty cheap investment for some very good initial returns.

¹I highly recommend the Martini Shot podcast to anyone who works in a technical or creative role. Mr. Long is a veteran sitcom writer who made his mark with “Cheers”; his brief spots are largely about the plight of Hollywood writers, which may seem a far cry from the life of the code monkey but is actually quite applicable. Sitcom writers get “notes,” we get bug reports; they have producers, we have project managers; they produce their work in lonely seclusion only to face the ignorant and capricious criticism of the suits in the corner office, and we … well, we do the same thing. Almost every week I find some little nugget of wisdom and insight that, if it doesn’t improve my work, at least gives me a wry and knowing chuckle.

Google Reader has been the core of my on-line news reading life for a couple of years now. It’s a great tool for keeping up with a whole range of news sources that use RSS: I organize my feeds into categories (News, Literature, Bicycling, etc.), skim the lists of news articles from any computer since it’s all stored in “the cloud,” and use the “star” feature to keep a list of things that deserve more than a cursory read.

For all its back end power, though, Google Reader’s interface is a tad utilitarian. (It’s the sort of interface I would probably have designed myself–as my project manager says of my current work (the UI has, thankfully, been handed over to a more competent designer), “it takes a mother to love it.”) The design owes much to the e-mail model, with folders for each topic and a bold unread count. You can see at a glance what new articles are available in each category, and expand the folder to see what’s available for each feed in that category. Articles are listed in the right pane in chronological order, though you can choose to sort by “magic,” which uses an algorithm that takes personal habits in sharing/starring and article popularity into account.

The problem with this interface, at least for me, is that it invokes a sort of anxiety when I see large numbers in the unread count. It’s very much like an e-mail interface, and I dread a full e-mail inbox. I’ve occasionally sat up far later than I should browsing Google Reader articles in an attempt to get the count down to zero.

I recently discovered Feedly, a web site and browser plug-in alternative to Google Reader, via Adria Richards (whose “But You’re a Girl” site has been in my Minneapolis-area feeds category since she broke the story of the Norm Coleman campaign’s shoddy computer security practices, though she should really be in my technology category). For a couple weeks now I’ve been using Feedly as my primary RSS reader, and I’ve been very pleased with it.

Feedly is built on the Google Reader backbone; it uses your Google Reader subscriptions in generating its pages. It also adds some content of its own–Twitter feeds, YouTube links, Amazon shilling associate links–and organizes content into attractive and easy-to-read layouts. You can choose different layouts–a “magazine” layout with featured articles at the top, picture and video grids, or more utilitarian summaries–for each category or feed, allowing you to customize your reading to fit your content.

Reading content through Feedly is easy and enjoyable. The headlines are rendered as links; clicking on a link opens a modal window within the page that displays the item’s summary content (including embedded media, which can be played within the page if the embedding technology supports it); the window is dismissed with a “minimize” link. Feedly offers “Cover”, “Digest”, and “Latest” views that show the most recent content (using an algorithm that utilzes Google Reader’s “magic” to surface personalized features), and presents a view of each of your categories; customization options let you modify the layout, colors, and optional components (Twitter, Amazon, Flickr, and others). There’s even a “Feedly mini” option, which provides a floating toolbar on other browser pages that surfaces related content from Feedly, and provides tools for sharing pages on Twitter, Facebook, and Google Reader.

Best of all, the Feedly layout reduces that anxiety of a giant list of unread articles. It still provides you with a count of unread articles, overall and by category or feed, but it’s not as prominent as in Google Reader. Instead, the layout is geared toward bringing the most recent articles to the top, along with articles that are ranked high in the “magic” algorithm, so the most interesting and useful pieces are the most visible. I currently have some 3,000 unread articles in my collection of RSS feeds, but I’m not bothered at all by it because the information that I see on the page is so much more compelling.

The only drawback to that I can see to the Feedly service is that it’s delivered as a browser plugin. It’s available for Firefox, and in a limited fashion for Safari and Chrome, but not for Internet Explorer or Opera. Because it’s a plugin, it’s less “cloud-friendly” than Google Reader alone: I can’t access my Feedly page at a public terminal, for example, though I am able to use it seamlessly across the three computers (Ubuntu Notebook, Windows 7 laptop, and Windows XP laptop) that I use most often. In those rare cases where I do have to use a non-Firefox browser on a system I don’t control, Google Reader continues to be my choice for reading the news.

Spam–the junk message type, not the tasty luncheon meat from Hormel (pan fried and served on rice, it really can’t be beat…)–is a significant annoyance on all the “Web 2.0″ services, but nowhere more so than on Twitter. Twitter meets all the criteria for an easy spam target:

• Low barriers to entry: scores of free accounts can be easily and automatically created through the Twitter API
• Large audience: some 19.2 million users in October 2009, according to ComScore
• A pervasive culture of implied trust and promiscuous sharing

If spam were a viral infection vector, Twitter would be an over-crowded, fetid swamp where not nearly enough people wash their hands.

The Stop Twitter Spam site maintains a list of tools for combating spam, and also a list of feature proposals (many of which have been implemented) that could help mitigate the problem. And while these are all good tools and useful suggestions, they’re not a panacea: the Twitter spam werewolf is impervious to silver bullets, because the open nature of the service makes spamming so cheap and easy.

Short of increasing the barriers to entry, limiting the ease of communication, or restricting the number of messages that can be sent–all things that would cripple Twitter in its current incarnation–there’s not much that can be done in a centralized fashion to solve the problem. Instead, the cost of fighting the spam lands largely on the non-spamming users. And there’s where the battle will ultimately be won or lost.

Our best weapon against Twitter spam (and other Internet spam sources) is healthy suspicion, just short of paranoia. It’s far too easy to be lulled by the fluid back-and-forth of Twitter communication into forgetting to ask, “Did this person wash their hands? What nasty germs might be lurking behind that URL?” We need to be vigilant about Twitter spam, which can come in direct messages, @replies, or follows, in order to block its spread; since much of the spam comes from accounts hijacked through “phishing” scams, making sure our own accounts are clean is a good defense against infection.

My own guidelines in assessing followers, links, and other Twitter content include:

• The follower/following ratio and number of tweets: now a part of the “new follower” notification, this is the easiest way to catch the spambots. Though Twitter tends to be asymmetrical, with most users following more people than follow them, a ratio that is too heavily weighted toward following–say, following 20 people for every one that follows them–is a clear indication that something is amiss. If that account is also not doing much tweeting–one or two messages, especially about making your teeth whiter or earning $325.42 a day or finding love in all the wrong places–then you’ve almost certainly been targeted by a spambot. Use the “Report as Spam” button to block the account and move the issue up the chain, and then get on with your life.
• The profile picture: If the account is using the default bird icon, they may be a spambot, or they may just be someone who hasn’t updated their profile yet. If the account is using a picture of a hot babe, shown from chin to bellybutton, then the odds are approaching certainty that this account is up to no good. If the account has the hot-babe profile and tweets links to weight loss miracles, the odds need not be calculated; “Report as Spam” and proceed.
• Grammar and content: Spambots don’t do so well on standardized tests of written communication. If the tweet has a lot of misspellings and webby abbreviations, the related link is suspect. If the message is “LOL u gots to c thisss” then it’s certainly spam. Don’t click.
• Context and consistency: Know the people you follow, and who follow you. If you’re following someone because of their pithy observations about Romanian poetry, or their scathing reviews of trends in script-based computer language design, and they suddenly send you a link to a Cialis offer, something is likely amiss. Out of courtesy, let them know, so they can take steps to resolve the problem; if they continue to send spam, then they’re either not taking the problem seriously, not being careful about their account, or have gone over to the Dark Side; unfollow at least, block if you must, report if you suspect a permanent hijacking.
• Look before you link: The 140-character limit of tweets is a great enabler of scary links. With the average URL length coming in at 34 characters (at least according to Kelvin Tan’s calculations), and with many news and blog sites’ URLs even longer, there’s been a proliferation of URL shortening services. Alas, it’s far too easy for malicious content to hide behind a generic shortened URL. (And this assumes that the shortener itself isn’t compromised in some way; the security implications of third-party URL shortening are really quite scary…) LongURL (available as a web site service, Firefox plugin, Greasemonkey script, or jQuery plugin), expandurl (web site and API), and ExpandMyURL (web site and bookmarklet) help by letting you preview the original URL before loading its content into your browser. If you’re not using one of these services, or a similar tool for previewing shortened URLs, you’re putting yourself and others at risk.
• Don’t be part of the problem: Keep your own account from passing spam around, or giving the impression of spamminess. Don’t give out your Twitter credentials to any service that you don’t completely trust; shorten URLs only when truly necessary; provide context for your links (“Great discussion of Hegelian epistemology” rather than “U shd c this,” though you’ll probably need to use that URL shortener after all); be as trustworthy and consistent in your tweeting as you strive to be in the physical world.
• Follow smart people: I don’t get a lot of spammy or suspicious junk on Twitter, largely because I’m careful to filter for quality up front. It’s not hard to spot the garbage when so much of the content is high quality, and smart people are less likely (I hope) to be drawn into a phishing scam, or will at least do something about it if their accounts are compromised. Be ruthless about unfollowing accounts that are frequently hijacked; it may seem cruel, but quarantine can be a useful tool in fighting virtual diseases too.

It’s a shame that the cost of combating Twitter spam falls back on the users, but that’s the paradox of the free and open model that makes Twitter so valuable to begin with. As in a free society, where we occasionally trade comfort and security for liberty, keeping Twitter from being overrun with spam is one of the responsibilities of its users.

As I noted earlier, some ongoing problems with the Diigo Firefox add-on and general annoyance at the graffiti public annotations feature prompted me to switch my transient bookmarking to Delicious. Some great support from the Diigo team resolved the add-on problem, at least for now, so I may be giving the service a second chance.

If you’re having the problem I was having with the Diigo add-on–a missing “Add to a List” menu option when bookmarking a page–these steps may save you some grief:

1. Right click on the firefox toolbar area
2. Click Customize…
3. Click Restore default set.
4. Sign out of Diigo and sign-in again

You may have to repeat these steps; I had to do it twice for the change to stick. But it’s a lot easier than the earlier troubleshooting steps, which involved creating a fresh Firefox profile and adding, one at a time, the dozen or so add-ins I use to see if there were conflicts with the Diigo toolbar.

Note that this does not, as I had feared it might, reset your Firefox to its default settings. But it’s not a bad idea to back up your Firefox settings, especially if you’ve made a lot of tweaks; take a look at the MozBackup tool for a nice and easy way to do this.

I’ve turned off the public annotations–from the Diigo button, choose “Show all annotations” and toggle to “Don’t show annotation”–to banish the yellow sticky notes left on Google, Wikipedia, and other sites by a handful of vandals. For the next few days I’ll be alternating between Diigo and Delicious before making a final decision on which service I’ll stick with long term.

A big thanks again to the Diigo support team; their prompt assistance certainly helps their service in my weighing of options. There’s no overestimating the value of solid technical support, whether a problem is big or small.