Debunking Some Start-up & Software Myths

I saw a great article by Tim Ferriss quoting Rob Mee of Pivotal Labs thanks to someone I follow on Twitter, and thought I should mention it here. It's rare that I read an article on a topic like this and agree with it 100%, but it happened in this case. I encourage you to check out the post itself, but in order to whet your appetite, here are the 7 Myths he describes:

1. You have to hire "ninjas."
2. Programmers need to work in quiet, without interruption.
3. Start-ups run hot, so we're just gonna have to burn everyone out.
4. Looming deadlines necessitate shortcuts.
5. Developers should take ownership of their code.
6. You need a quirky hiring process.
7. Specialization is essential.

As I say, it's a very good read and excellent food for thought!

Agile Metrics

I'm only partway through the video (it's about 2 hrs long) but so far I'm really enjoying and agreeing with what Dave Nicolette had to say about Agile Metrics at Agile 2009. If this is a topic that's of interest to you - especially if you're a Project Manager! - then I recommend you give this video a watch.

The First Retrospective

Last week I spent Thursday morning facilitating a project Retrospective for a local development group who'd never held a Retro before. As part of my engagement, I was also asked to do some "training the trainers" in the sense of working with a couple of people there who were interested in taking on the facilitation role in the future. The thinking on the part of the executive sponsor was that they wanted to bring an expert in for an early Retrospective (or more) but not be reliant on him forever, which is a sentiment that I totally agree with and was happy to support.

The preparation for the session started several days beforehand, as I e-mailed questions to the sponsor and also had a phone conversation with him. I'd sketched out a possible structure for the Retro prior to that, and then the two of us hammered out the final details during that call. One of the most important factors we talked about was duration, which ultimately ended up being 2.5 hours. I would have loved to have scheduled it for longer since it was covering an entire 6- or 7-month project. However, the flip side of the coin was that this was going to be everyone's first Retro, and there was concern that being subjected to a full-day session first time out would be too much, too soon. That led to the decision for a meeting running from 8:30 to 11:00 a.m.

When I got on-site Thursday morning, I met with the future-facilitators for an hour and described what would be happening. The three of us got the room ready by putting up some large sticky notes (a mostly-blank timeline, ground rules and the agenda) and I provided a few thoughts on what I believe the facilitator's role to entail:

• keeping the group focused on events and learning, not on people and blaming
• warming up 'cold' rooms and cooling down 'hot' ones
• nipping any personal attacks in the bud
• avoiding ruts in the proceedings by keeping things moving forward
• making each of the attendees feel comfortable with the process itself

I'd planned a warm-up activity that went well (got people talking, generated some light-hearted laughter, felt like fun) and then we drove headfirst into the agenda.

Most of the attendees were slow to open up very much, as you'd expect from people in their first Retrospective. Nothing too contentious came out until more than halfway through, and even then it was offered up fairly tentatively. By the 2.5-hour mark, though, things were getting much more interesting... just as time was up! As one of the trainee facilitators pointed out, it was all we could do to get several of the attendees out of the room even half an hour after it was over! And these were very busy folks, some of whom were missing other commitments in order to stay a little later and chat with us about what they`d just gone through. That sort of thing is really rewarding to see!

The results of the 1-page survey that I handed out at the end were very positive (scores averaged between 8.2 and 9.0 out of 10). In our 1-hour debrief after the session (half of which was chewed up by the hanger-on-ers!), the two trainees did a great job cleaning up the room (while I was waylaid with questions) and then talked about how interesting it had been for them. They were concerned about being able to do what I had done, but I assured them it was easier than it looked! In fact, the toughest part for me was that I was on my feet for nearly the entire two and a half hours, an amount of standing well beyond anything that I've done in the past several months. I'd had the presence of mind to bring a big bottle of water with me so I wouldn't get parched, and yet I didn't think to sit down for more than about 2 minutes over the course of the meeting!

I don't know yet whether or not I'll be invited back for additional work, but this first contract was very enjoyable, indeed. The participants were committed to the process and willing to open up in front of each other and a stranger, and you can't ask much more than that!

The Software Development Cycle

With my recent contract now complete, I thought I'd post a few non-specific details about what I did during it. While it's not completely germane to Agile, you'll see a few places where there was some overlap, at least.

There were basically three parts to complete as part of the contract:

• review the organization's current software development process
• compare that picture to what a more standard development process would look like
• provide recommendations on changes or enhancements that could be implemented to make things even better

Initially there had been some thought of doing a fourth part, as well: implement some of the recommendations, alongside the development team. However, virtually none of that came to be, through a combination of factors the most relevant of which was that the developers were in the middle of a high priority project that couldn't be delayed at the moment.

In order to get a clear idea of how development happened in this company, I read over a fair number of documents that related either to standards, or to recent projects, within the organization. From that exercise, along with a few strategically-timed interviews with personnel there, I was able to figure out what type of artifacts were produced, as well as which practices were followed. I'd describe what I saw as a fairly lightweight model, which is what I'd expected to find, given the age (fairly young), size (6 to 8 developers) and position within the company (an Information Technology group within an industry that's far removed from being an IT provider). In some areas, it was really only rigour that was missing; in others, there was simply a lack of experience with "best practices for large-scale software development", let's say. And that's the sort of thing you bring a consultant in for, anyway.

For the comparison aspect, my boss suggested a Gap Analysis in the form of a spreadsheet, which I thought was quite appropriate. The U.S. arm of the company had previously brought in-house a very comprehensive System Development Life Cycle model for the American IT groups (which are considerably larger than the Canadian one located here), and I was happy to use that as my baseline for the comparison. This model was very thorough (and very gated, as befits a Waterfall approach) and so it provided an excellent "ideal" for me to measure their setup against. There were roughly 150 artifacts or activities included in it, and so I was able to evaluate my clients' methodology in terms of which of those items they included, and (in a few cases, at least) how their versions measured up to the baseline.

Once the Gap Analysis was complete, I identified what I believed to be the highest priority gaps. This ended up being just over 20 items. For each, I then wrote up a Recommendation, in the following format:

Recommendation: xxx
Benefits: xxx
Approach: xxx
Effectiveness Metrics: xxx

In other words, I provided the what, the why, the how, and some notion of how to measure the results to make sure that they weren't wasting their time making the change.

That Gap Analysis spreadsheet, which included the 20-some Recommendations, was the primary deliverable from my contract. However, in talks with my boss we decided that a couple other concrete artifacts would be desirable to come out of this, as well. The first was a Microsoft Project Plan that was essentially a skeleton Plan that included all of the steps from the "ideal" SDLC model, customized somewhat to their environment, in order to make it easier for all of the steps to be remembered in the future. Also, because I considered a thorough and comprehensive Release Plan to be a prime example of where some additional rigour would help them out, I provided a template for that, too.

Those three documents were reviewed with most of the development staff on my last day there, and were extremely well-received. I had tried to incorporate as much feedback from the staff as I could in the content of my deliverables, and I think they appreciated that. One example was the inclusion of a recommendation that features be planned in a way that allowed them to be completed more serially than they had in the past, so as to allow at least some testing of some of the features before being passed on to the business area (who, in that environment, act as "QA"). That, and some of my unit testing suggestions, definitely benefited from my two years as Agile Manager!

All in all, it was a good learning opportunity for me, and I think that the clients received good value from my 22 years of IT experience. I think that if the timing had been different, such that the people there who needed to play a bigger part had had more time available, then it could have gone better and more immediate gains might have been realized. But as the poker enthusiasts say: you have to play the hand you're dealt, and that's what I did.

The Story Point Workshop

I've been meaning to write up a description of the Story Points Workshop that I designed and delivered in 2007 and 2008, but hadn't gotten around to it... until now. I figure that there's always a chance someone will stumble across this blog and be interested in getting some training on Story Points, and so I really should have something here by way of a sales pitch. If you're reading this and would like to know more, please contact me at AgileMan@sympatico.ca and we can discuss it further.

The workshop that I created is intended to be appropriate for anyone involved in the development of software: programmers, testers, integrators, project/program managers, product owners, team coaches and even executives. In other words, you have to possess at least a basic understanding of the process of software development in order to be able to complete the exercises. Note, however, that you don't have to be of a technical bent, as there's nothing in the workshop that requires you to design, program, or even debug any software.

My goal in providing the workshop has always been to increase the attendees' understanding of, and comfort with, Story Point estimates. At the end of the day, I want everyone who took part to feel confident that they can use Story Points themselves - either as producers of them, or consumers - as part of the release planning process.

To get a large group of people, often with disparate skills and perspectives, all the way to that goal, I use a format that's proven to be both effective and fun (based on the results of feedback surveys that I've done at the conclusion of each session). Basically, the day-long workshop goes as follows:

Part 1: Introduction of Story Points

Here I cover the concepts of relative sizing (as compared to absolute) and what Mike Cohn characterizes as "Estimate size; derive duration." I do this through a combination of slide presentation and interactive exercises, the latter of which allow the group to do some estimating on things not at all related to software (eg. figuratively moving piles of dirt). This 60- to 90-minute section ensures that everyone has a solid foundation of understanding around the conceptual portion of Story Points before we move on to applying that knowledge.

Part 2: Estimating in Story Points

For this part of the day, which takes up the majority of the workshop, I introduce a new application called "the Meal Planner." This is a Java program that I wrote years ago, but the attendees don't really see it or any of the code for it. Instead, what they see is a description of what it is, as well as a set of additional features for it. Because the Meal Planner is a simple and fun concept that anyone can grasp within a few minutes, it provides a perfect backdrop to use in understanding how to Story Point items. No one in attendance has any history with it, and therefore it's unlikely that anyone would dominate the proceedings based on any real or imagined "expertise" with it.

The attendees are then broken up into smaller groups of 5 to 7 people and physically separated (by team) as much as possible. Each team is tasked with coming up with Story Points for each of the items on the Meal Planner's product backlog. I serve as the Product Owner for each group, running back and forth between them in order to answer questions or offer clarifications to the requirements that they've been given. This exercise typically takes between two and three hours to complete, as the teams often get off to very slow starts while they go through the "forming/storming/norming/performing" cycle.

Part 3: Comparing the Results

When all of the teams have produced their Story Points for the complete set of backlog items, I bring everyone back together, post their results up on the walls and encourage each team to review what the others came up with. Then I have a representative from each team briefly go through their rationale for each estimate, allowing the other teams to comment or question them. Sometimes the teams will have arrived at similar estimates; often they'll be quite different. We discuss what sort of factors contributed to those results, and build up a list of "What Worked and What Didn't" that help the attendees understand what sort of things contribute to better estimating versus what works against that goal. I usually will also provide my own Story Point estimates (after everyone else has), based on having written the application. All told, this section usually requires about an hour to get through.

There's a slight variation on the above that I've done in one instance, which proved quite insightful for all involved. I had 2 groups in that case, and so I decided to spend most of my time with one, at the expense of the other. Also, I'd planned it so that the "AgileMan-starved" group got a different set of requirements than the other group, with slightly less details in it. The results of this disparity were striking, to say the least: in many cases, the "starved" team produced estimates that were completely at odds with what both the "better-fed" group came up with as well as the "expert's Story Points" that I provided. I think this sort of thing is probably only appropriate if the workshop is being set up for an environment where the product people are struggling to fulfill their responsibilities (as was the case where I did it).

It's possible to get all of the above done in as little as about four and a half hours, although that's awfully rushed and not very conducive to learning. I prefer to schedule it for either five or six hours, in order to allow for regular breaks and to give people the opportunity to reflect on developments as we go along. Something like a 9:30 start, with lunch brought in, and a wrap-up by 4:00, works best. I've done the workshop with as few as a dozen people, and as many as 25 attendees, and both extremes seem to work fine. With fewer than 10 to 12 people, it's difficult to get 2 groups that can compare results. More than 25 would provide quite a logistical challenge for me to facilitate once the smaller groups are formed.

When I took this workshop on the road to a company in the San Francisco area in late 2007, I did 3 sessions there (one per day) and received an average score of 8.7 (out of 10) in terms of "overall quality." One of the company's Vice-Presidents recently recommended me to a colleague based on those workshops, saying that I had "received the most positive reviews from folks that any trainer has at our site, and we've had quite a few." So I like to think that it left a good impression!

"My Life As A PM Is Over: The Company Just Went Agile!"

Last night was my presentation on Agile Project Management to the local chapter of the Project Management Institute, and based on the electronic polling results that were collected right after I finished, I think I did fairly well. Each member of the audience was given a handheld device about the size of a small calculator and then asked to respond to a series of statements along the lines of: "The topic presented by tonight's speaker was informative and interesting" and "Tonight's speaker appeared to be well-prepared" by selecting, on their device, either "1" (strongly agree), "2" (agree), "3" (disagree) or "4" (strongly disagree). For those who were having trouble figuring out how to work their new toys (since this was the first time they had tried this approach), I helpfully offered the advice, "Just press the button labeled '1'!" In most cases, I received scores where the total of "1"s and "2"s put me at or above the 80% mark, which was gratifying to see.

As for the material itself, I had worried that I had too much to cover in 2 hours, but somehow it all worked out, even with a 15-minute break midway through. I spent the first half introducing Agile, and then called for a short break. However, I asked that, during the break, everyone write down how they thought they might react if they went into work on Monday and their boss walked up to them and announced, "Guess what? We're going Agile in a month!" Just before resuming, I collected their responses and read them out for everyone to hear. Here are some representative samples of what I got:

• "Agile? Can we get the Functional Managers to support it?"
• "Good, [but] first educate the clients."
• "Let's go!"
• "Can we still perform accurate Earned Value analysis?"
• "Great, but how do I co-locate with my global team?"
• "1 week cycles??"
• "But what's wrong with RUP?"
• "Great, let's try it. As long as we follow ALL of the steps, not only parts of it."
• "How do we break deliverables into iteration-sized chunks?"
• "Risk of scope creep."
• "How do you budget??"
• "What are the signs that we NEED Agile?"

and my personal favourite:

• "Do Agile? We don't even do Project Management!"

What impressed me most about these results, aside from the variety, was that it was pretty clear that everyone had been paying attention to what I'd been saying up to that point and attempting to rationalize it against their own experiences and work environments. Virtually no one was writing off the notion of "going Agile" as some crank thought experiment that they could laugh off or simply ignore. Also, I'd say that about 1/4 of the responses were of the "Let's go!" variety, showing that there was at least some buy-in among a group that was mostly hearing about Agile for the first time.

During the Q&A session at the end, many of those same concerns were expressed once again. One woman was sure that no budget would ever be approved without a guarantee that the work done would match what was being approved, to which I said, "If that sort of arrangement is working out for you right now, then I don't know why you'd ever want to move to Agile. The organizations that move to an Agile methodology are the ones where things change considerably between initial concept and final installation." Another fellow was understandably concerned about how to create automated tests to replace the extensive manual testing that they currently do at the end of each project, and I said that that was one area where you'd want to allocate time in order to research the new products and services that are coming out on the market for that purpose. "But in the end," I said, "you really don't have any choice: you need to have automated testing, or you'll never be able to do short cycles." I also got asked whether Agile was really only appropriate for "UI companies", to which I was able to describe some of the very-different types of organizations I've heard of - or seen! - doing Agile.

The moderator eventually had to call an end to the Q&A, as I think it might've gone on for quite a while longer otherwise. I received a $25 Chapters gift card for my troubles (which was nice, and unexpected) and eventually took off home to unwind. Sadly I didn't make any book sales, despite having several copies with me for just that reason. Maybe next time!

The Project Manager "Problem"

I've been invited to speak at the next scheduled meeting of the local chapter of the Project Management Institute. It was requested that I make my presentation "about Agile" but the rest of the direction was pretty much left to my imagination and questionable discretion. As such, I've come up with material under the banner of:

"My Life As A PM Is Over: The Company Just Went Agile!"

Now, that title is most certainly intended to be attention-grabbing, humourous and perhaps even a little ironic. That latter characteristic stems from the slightly-nebulous view in which some in the Agile community hold Project Managers. At one extreme fringe of perspectives on the topic is the one that says, essentially, "Now that you've gone Agile, you can get rid of all your PMs!" A more moderate stance shows up in some of the Scrum literature, which advocates for having most or all of your Project Managers transition into the Scrum Master role. All the way at the other edge of possibilities lies the approach that I saw undertaken while I was the Agile Manager in my last job: keep the PMs as PMs, and figure out how to make that predictive, deadline-focused paradigm work in an iterative, adaptive environment. (Those who've read my two AgileMan books will recall that, many good intentions notwithstanding, we struggled greatly in integrating those two very different world views.) Even with that attempt to keep things somewhat as they were in the PMO, though, I think all of the people involved would agree that the PM role changed considerably as we moved through our Agile transition. Therefore, in all cases, there's some bitter irony to saying, "My life as a PM is over..."

For my material, I'm starting with what I presented to the 3rd year Project Management course last November. There's quite a bit of overlap, it seems to me. In both cases, I need to start off by describing what Agile is, under the assumption that some won't have even heard of it. Then, I'd be foolish not to acknowledge right away that many of the principles and practices of Agile can, and probably should, strike some project management types as idealistic, at best, and heretical, at worst. And finally, I want attendees to get a sense of how the role can still add value and be rewarding within an Agile framework. The biggest difference, of course, is that next week I'll be talking to people who have presumably been out in the real world actually running projects for a number of years now. That should mean that they're even more cynical, or at least skeptical, about the practicality of an adaptive approach. Will customers/product areas ever really sign up for a system in which change is expected throughout the process? Won't scope creep simply cause projects to run on forever? How can you let those developing the software also test it? And so on.

In providing feedback to Mike Cohn on his next Agile book recently, one of the places where I really challenged him was around his treatment of what to do with Project Managers. I thought that the draft version I reviewed made too light of the problem, not acknowledging just how much of a leap of faith is required in several aspects of his recommendation. If a company's culture has a strong dependency on micro-tracking of deliverables, who's really going to take that up if the PMs are re-purposed to other duties? Will most PMs, some of whom have spent considerable time and money getting their PMP certification, really want to risk losing it by directing themselves to activities which won't keep them current as PMs? How many individuals in the PMO really want to become Scrum Masters / Agile Coaches? I think those are just some of the vital questions that need to be addressed on this topic, and I believe too many of us have glossed over them in the past. I may not know the answers, but at least now I'm aware that the discussions have to happen.

It should be interesting to see how all of this goes over with the group next week! If you never hear from me afterward, you can draw your own conclusions...

Bugs Happen!

A friend of a friend (who actually follows this blog, I believe) e-mailed me yesterday, asking if I had any advice on how to handle a situation that his Agile team kept running into. Apparently they'd had several instances in the recent past where high priority production bugs had come to the team in the middle of a (Scrum) Sprint. In each case, it was considered unacceptable that the defect resolution simply wait for the next Sprint, and so they had tried a couple of different approaches to address the urgency of the situation:

1. Include the bug-fixing activity in with another feature in that Sprint that was somewhat related
2. Add the bug fix as a deliverable for the Sprint, with a higher priority than everything else

I was told that neither of those had worked all that well, either in terms of successful completion or from a morale point of view.

As I said in my e-mail response, both of those were reasonable ways with which to try to accommodate a difficult and frustrating scenario. However, I could easily imagine - even without knowing any of the people involved - how each might've gone awry. So instead I offered a different tack to consider next time, and it's the one that most Agile purists would likely always push for. But before I get to that, a few thoughts on the general topic of "bugs from the field."

Since I don't personally believe in "defect-free software" (having never produced, used or purchased any examples of such a mythical creature of legend second only to the unicorn), I go about my day under the strong assumption that there are always going to be problems found in production that weren't caught prior to getting there. Of those, some will be of a truly urgent nature that need immediate attention, and many won't be (but may still be initially prioritized as if they were); most almost certainly could have been detected and corrected before getting out the door had enough time and energy been expended in the search for them, and a few probably couldn't have been. In other words, they come in all shapes and sizes, but regardless: they come when they come! So, for me, the important question to be asked of any bug when it arrives back at our doorstep is, "What can we do better in the future to make production defects less likely." And if you're getting a significant enough quantity of showstopper bugs following release, then clearly you're not asking or (more importantly) answering that question adequately. Some serious and ongoing "inspect and adapt" discussions and tasks need to be happening if "stop the presses!" types of live bugs are more than an occasional occurrence for you and your team. That's one particular problem, and it's discreet from the one that was being posed of me.

What I was being asked was what to do in the short term, potentially high-pressure situation of dealing with the here-and-now of a bug's arrival and its effect on the current Iteration or Sprint. And, of course, the by-the-book answer is that you either cancel the Sprint and plan a new one to include the bug fix, or (somewhat less drastically) get the Product Owner to remove something from the Sprint in order to make room for the bug fix. I prefer the second approach because I'm not a big fan of bringing everything to a halt simply to accommodate something new. It may be that the bug in question is going to suck up so many of the resources for so long that you'll determine, while sizing it, that you're going to have to effectively cancel everything else anyway, but I'd prefer that an outcome like that simply remain an option, rather than the default response. In either case, though, you're taking actions that clearly send the message that fixing this bug is "not nothing." And that's critical because, among other things, it can sometimes result in the bug fix suddenly not being deemed as important as it had been when it had no price tag associated with it. It's like the free coffee and pop at my last workplace, which a few misguided souls there tended to treat rather disrespectfully (open can, take sip, leave remainder there for cleaning staff to deal with). When something's thought to be "free" that isn't really, it often gets some poor assumptions made about it.

Now, one argument against what I've just written that sometimes gets made goes like this: "Since the team didn't find the bug in the first place (but should have), then it's up to them to figure out how to fix it without jeopardizing everything else that they've committed to in the Iteration/Sprint." Honk if you've ever heard that one before! (Honk! Honk! Honk! Honk!)

There are a number of reasons why that's a silly stance to take, and I couldn't possibly think of them all in one sitting. But among the most obvious problems with it are:

• it encourages teams to fix things in the most slap-dash manner possible, since every minute they spend on the resolution is taking time away from what they were actually expecting (and expected) to work on
• it drives home the belief that the team is going to be penalized for every mistake they make, thereby making experimentation less likely (which is appropriate if you're programming a system for the Shuttle or a pacemaker, but in most cases...?)
• it flies fully in the face of the reality that even software organizations with tens of thousands of employees ship code with bugs

So for those reasons and others, it's unreasonable to expect development teams to "suck it up" and absorb critical bug fixes without removing any other work (unless the team decides, of course, that it's a small enough effort and can be added in with no impact).

Getting back to the question of "how often does this happen?", I think that what I've outlined above works fine as long as the scenario we're talking about is the exception, and not the norm. However, if it's happening on a more frequent basis, then in addition to figuring out why (and doing something about it that should reduce the occurrences in the longer term) you may need to set aside a small percentage of the team's velocity every Iteration for critical bug fixes. As I said in my e-mail response, you really want to make sure that any time that's allocated for that use be limited to actually dealing with high priority bugs, or bringing work in (off the product backlog, in priority order) if no urgent defects come along by, say, the midway mark of the Iteration. It definitely should not become a "slush fund" that can be used to soften the blow related to under-estimating, get work in on pet projects or simply to provide a cushion for the team. (You may, of course, want to allow some Slack Time for the team to dream up process improvements or new tools, but that's a different discussion.) If you can do it right, the team will regard that extra time correctly, and no one outside the team will have any reason to resent it or try to eliminate it, because they'll see the value of it, either in terms of showstopper bugs being fixed quickly or additional features being delivered.

Of course, it's easy for me to offer advice when I'm not in the middle of the fray, but that's just the way the ball bounces at the moment. Maybe someday I'll be back in there, swinging for the fences...

Risk Assessment Is An Underappreciated Talent

Any project managers out there, upon encountering this blog post, are probably looking at the title and saying, "Yeah! You better believe it!" In many software development organizations (perhaps most of them), it officially falls to those in project management to worry about risk. In reality, though, risk identification and mitigation are two concepts that every one of the project participants should care deeply about.

To draw on a current real world example, look at the state of the financial industry in America right now. No one in their right mind would describe it as anything less than disastrous, no matter how much people may disagree on the exact root cause. One of the main contributors to the meltdown, we've been told many times, was unbridled optimism. Whether it was home owners taking out mortgages that they couldn't afford (but all of whom believed that they could), real estate speculators assuming that housing prices could only continue to go up, credit businesses refusing to consider the consequences of a market downturn, or the public in general failing to set aside much in the way of "rainy day savings," the result was the same: once the first domino fell, all of the rest were lined up in a perfect, catastrophic pattern behind it. In other words, a lot of folks misinterpreted a variety of "best case outcomes" as being virtual certainties.

In the world of software, we face a very similar problem. When we create a plan - whether it's a traditional Microsoft Project plan for Waterfall environments or an Iteration plan for an Agile team - I've too often seen what I can only describe as that self-same unbridled optimism. In one way, that's a commendable attribute: after all, it's indicative of a "can do" attitude that's much preferable to a bunch of nay-sayers who won't even try, for fear of failing. But it still has to be tempered with reality. At the very least, any group involved in this sort of planning activity has to spend some time discussing what might reasonably go wrong, and what the impact(s) for each would be.

That's the identification part of the equation, and I'd say (from my own experience) that programmers in general tend to grade somewhere around a C- at that. You'll get some who are really good at it, and others who apparently can't see anything at all in front of them except the Yellow Brick Road that leads directly to Oz. Team members with a testing background or responsibility are usually better at this, perhaps because their stock and trade is stuff not working the way it's supposed to. I'd say the average test-inclined person maybe scores a B- at risk identification, although, again, you'll certainly get all sorts of results. Some testers, for example, only think in terms of bugs in the code, rather than considering "bigger picture" hazards like environmental issues, requirement ambiguity or dependency failures.

When you bring that programming perspective together with the testing POV - either in a meeting, or as Agile tends to do, in one person or a close-knit team - I think the discussions often provide some very good risk identification. The two views feed off of each other, and pretty soon you'll hear some very creative ideas of "what could go wrong." This is more of a refreshingly-effective B+ effort, if you can make it happen. There can almost be a gallows humour aspect to those talks that may cause the unseasoned project manager/stakeholder to run for the Pepto-Bismol! (So, yeah, it can be fun!)

For risk mitigation, though, I think the software world in general really lets our PM friends down. I'd even go so far as to say that we probably earn a solid F in this particular area (or perhaps it's simply an "Incomplete"). In part, the bad performance here stems from the fact that mitigation is often as dependent on the corporate culture as it is on anything else. As an example, consider the classic case of an external dependency not being met. In some companies, such an event would be regarded as reason enough (some might say "excuse") for the greater initiative to fail. Therefore, there might be little stomach for planning any mitigative actions for that possibility, as the blame would be laid squarely at the feet of the non-deliverer. End of story. But in other environments, there can exist expectations - spoken or otherwise - that those involved in the local project can somehow work around unfulfilled external deliverables. Those two scenarios are very different, and it behooves everyone on the project to know which type they operate within. And that's something that makes this whole "what do we if this risk becomes reality?" question hard to answer, especially when you consider the unspoken expectations that often go along with the handling of these risks.

Another challenge to good risk mitigation is that old favourite of mine: empowerment. If a company has a consistent and well-understood empowerment model, then it's easier for a group of programmers/testers/product reps to tackle what to do about each potential hazard. In a top-down model, you may be able to simply kick the problems upward, and comfort yourself in the knowledge that your job is to identify the risks and then escalate them. Again: end of story. Where teams are more empowered than that, then they'll probably look for creative solutions that may involve spending money (buy additional hardware, set up network redundancies, increase bandwidth) or moving human resources around (get temporary expertise from elsewhere, increase headcount on the project), with some reasonable assumption that their time wasn't wasted in considering those options. Where the empowerment model is muddy - maybe one type gets lip service paid to it but another is found in practice - then the job of risk mitigation becomes that much more difficult.

As someone who worries a lot about risk - maybe too much, according to my wife! - I can't help but wish that it got more consideration in the minds of others. When things are going well, there's a natural tendency to expect that delightful state to continue forever. But it's our ability to imagine just what might come along and turn all of our plans to mush that gives us an advantage... if we remember to use it! Otherwise, we end up with situations like the cratering American financial sector.

Where's The Happy Medium Between Gold-Plating And Shoddy Work?

I think sometimes the line between "not good enough" and "better than it needs to be" is about the width of a razor blade (not that I know all that much about razors!). And that slim of a margin for error is not a very easy target to hit, especially amidst the chaos, pressure and distractions of most software projects. So what can we do to make the situation better?

Well, first I think that we need to examine what's really happening whenever accusations of "gold-plating" come up. I've seen it often enough in the past 22 years to at least have some ideas on it by this point in my career. The most common culprit always seems to be time, in the sense of there never being enough of it. When a customer, or project manager, or really any stakeholder with their ass on the line has their eye on the calendar, "finishing touches" can quite easily take on the appearance of "make work projects" or "endless tweaking" (and I don't mean the kind that requires amphetamines!).

To appreciate that perspective, just imagine that you've gone to pick up your car after a repair. You're in a hurry because someone's waiting for you and you told them that you'd be there by now, only to take longer than expected to get to the garage and then find out that the car's almost-but-not-quite ready. You then catch a glimpse of the mechanic busily doing what looks for all the world to you like a polishing job on your hubcaps! Of course you'd be frustrated at that scene, because as far as you're concerned you'd much rather get where you need to be right now with dirty hubcaps than arrive there later with 'caps that look brand new. But what if what was really going on was that the mechanic was checking out your tire pressure and using a rag to clear the tire well enough that he could read what the pressure should be? Most people would agree that having your tires checked is hardly "gold-plating," but you may not even know that that's what is actually happening from your vantage point. And on a more relaxed schedule, you'd simply trust that the mechanic knows what he's doing rather than jumping to any unfortunate conclusions. Start the clock ticking, though, and that whole willingness to extend trust may go right out the window... unless we make a conscious effort to be aware of that trap, and avoid it.

Now, that first point assumes that it's a simple lack of understanding that's creating the conflict. The software developer, in that case, is engaged in some activity that he or she has every reason to believe is required, in their expert opinion; meanwhile someone less involved leaps to the wrong conclusion about it being unnecessary. That's certainly not going to be the only scenario that can result in "gold-plating" charges being thrown around, though. Another possible cause is that there actually is work being done that is above and beyond what's needed to complete the job satisfactorily. So how do developers know when they've reached "good enough" with a piece of functionality?

One obvious way to answer that question is to ask another one: "What's the likely result if I don't do this next bit of work?" If that query can be posed and subsequently answered honestly, then I think we'd have far fewer confrontations of the sort discussed in this blog post. Most of the time, though, I suspect the question's not even being asked; and of those times when it is, then all too often the answer offered up is at least somewhat disingenuous. Why would a developer not honestly answer it? Because they may have an agenda (hidden or otherwise) that's not necessarily related to customer satisfaction. Some examples would be:

• "This code is too hard to work with unless I change it to look like what I'm used to."
• "The customer might ask for this functionality in the future so I need to put it in now."
• "I've thought of a case that will probably never happen, but if it did this code might possibly break so I'd better rewrite it."

That's not to say that any of those responses are categorically bad; just that they might not be reason enough to dictate that something's not ready, and therefore delay a release. And even a whiff of that sort of thing can cause the cries of "Gold-plating!" to go out.

So there we have two of the many ways in which this unfortunate situation can arise. To guard against them requires greater transparency, communication and honesty than we might naturally demonstrate if we just went about our business with our heads stuck in the sand. Living up that higher standard is a lot to ask of us all, to be sure; but as skilled professionals who take pride in our work, we can do it... can't we?

Book Publishing Running Ahead Of Schedule

I was very happy to see this week that Lulu, where I print my AgileMan books, was able to produce and deliver the print drafts of More Real-Life Adventures of AgileMan (Year 2: Easier Said Than Done) and The Complete Real-Life Adventures of AgileMan (Two Years of Lessons Learned in Going Agile) in less than a week! I ordered them Sunday afternoon and they arrived the following Friday! That's quite impressive and in fact exceeded my expectations by about 50%. (That's the good news; the bad news is that even their cheapest shipping option is now more expensive than it used to be, meaning that they've already started eating into my profit margin before I even sell the first copy!)

After Vicki and I checked over what had arrived, I made a few minor tweaks and then placed an order large enough to cover all of the requests that have come in to-date plus a handful of extra copies of each book. Based on the estimate provided by Lulu on the order, I would think that they might arrive sometime during the week of Dec 15th, or possibly earlier if I get really lucky. It's hard to predict just how much things will slow down in December because of holiday-related activity, though. It's still possible that the books won't arrive until after Christmas, which would be more in line with what I imagined when I planned out the self-publishing schedule leading up to this.

In any event, we're on the last leg of the journey for the 2nd AgileMan book, and those of you out there who are eagerly awaiting it (c'mon! stand up and be recognized, both of you!!) should know that it won't be much longer before you have it in your hot little hands. I still have to figure out, logistically, how I'm going to get copies to all of those people at my former workplace (I can't exactly book a meeting room for it, after all!), but I'm sure that we'll figure something out.

Let History Be Your Guide

I was asked an interesting question yesterday, thanks to the magic of Instant Messaging. One of my friends at the company that I used to work for (in my capacity as AgileMan) wondered if either my wife or I had ever heard of something called "empirical forecasting." We hadn't, but as he told me more about what he'd been reading (and even shared some of it with me), I realized that it was something that I'd been practicing during the final few months of my employment. It's also a topic that I wrote about briefly in More Real-Life Adventures of AgileMan (Year 2: Easier Said Than Done), although you'll just have to take my word for that until the book actually comes out! I just hadn't had a fancy name for it, until now!

Basically, that style of forecasting or planning involves using the empirical (or observed) data that you have, and only predicting what can be extrapolated from it. In other words, you only expect to get results that match what you've gotten in the past. That may sound obvious to some, but in my former company, it was rarely the way planning occurred. I sat in countless meetings in which statements like the following would be made:

"Our owners need ______ (some set of features) by ______ (some date in the future) so now we need to figure out how to deliver it."

Put another way, we'd start at a desired outcome and try to work our way backwards from it. I completely understand why we always felt that it was necessary to do things that way, but repeated failures of that approach should have incontrovertibly shown the folly of it at some point. Instead, it continued to happen.

What works much better, I believe, is to base your schedules on what has been proven in the past. For example, our development teams each had a certain average velocity (in Story Points) that they'd worked at for the past N Iterations (where N could reasonably be any integer between 3 and 6, let's say). Realistically, if that's what they did in the past, it's likely going to be close to what they'll do in the future. Therefore, you should expect about the same productivity going forward, and anything beyond that is just wishful thinking. If a team has 100 Story Points of work left, for example, and they work at an average velocity of 15 Story Points each Iteration, then you could reasonably assume they'd finish all of the features in roughly 7 more Iterations, with the caveat that the "remaining work" doesn't change dramatically in the meantime. What would happen all too often in my former workplace were frantic meetings in which endless discussion would ensue as to how to shorten that timeframe down to 3 or 4 Iterations. In the end, of course, it would still take 7 Iterations, or even possibly longer, if enough churn had happened in the process such that it actually made additional work for those involved!

In my conversation yesterday, the other person even talked about using simple line graphs to predict when the work remaining would hit zero, including accounting for features that have been added or removed. I think that sort of graphical representation can certainly help in such matters, as some people relate better to pictures than to words. In either case, though, your expectations are limited to an extrapolation of what's already happened, rather than relying on the time-honoured "... and then a miracle occurs" (as we all used to fall back on for those impossibly-hard university math exam proofs!).

In order to use this sort of technique, of course, you need development teams that can become fairly good at providing relative estimates (such as Story Points). That's no easy challenge, but it's also a big enough topic to warrant a separate blog post, and so I won't tackle it here!

Project Management In An Agile Environment

Last night, my wife Vicki and I traveled to our local university where we'd been invited to provide guest lectures to an evening class made up of third year Computer Science students, all of whom are enrolled in a course on Project Management.

Vicki wanted to go first, and so she took up the first forty-five minutes (of a 3-hour class) describing several of the jobs that she's had over the years that didn't, at first glance, appear to be related to Project Management... and yet were! The overall message of her talk was that there are more types of PM jobs out there than you might think, and the students seemed both interested and encouraged to hear that viewpoint! Although she was nervous going in, you'd never have known it to watch her... and she's now much more confident of her own ability to present on that topic.

My portion of the proceedings ended up taking close to 2 hours, in part because I had more material to cover (more on that in a moment) but also because I got asked a lot of questions! Although a handful of the inquiries were slightly skeptical in nature ("Could you really get a customer to approve spending money if you're not promising to deliver on time?"), most were more obviously related to them beginning to connect the dots. Which leads nicely into what my talk was on.

I broke it into 2 parts: What is Agile?, and What do PMs do in an Agile world? Because I had to assume that they had no knowledge of Agile prior to this (which turned out to be mostly accurate), it seemed critical to me when I was planning my lecture to give them a sufficient grounding in it before moving on to the Project Management angle. Based on how things proceeded, I'd say that was a wise choice.

Because it was a PM course, and I'd taken a look at their curriculum online, I decided that it was appropriate to frame Agile within the context of how common it is (and has been, for decades) for software projects to fail... a topic that I knew they were well familiar with. I provided some ideas of my own as to why the failure rate has always been so high, and then showed how - if done properly - Agile might address some of those factors. An actual PM from my former job was in the audience, and was nice enough to offer up some good observations to the class about areas where Agile had actually helped.

Before I moved into the second half of the material, I decided to get everyone up off their butts in a goofy little move where I had them stand up and give me an enthusiastic "but but but" as I launched into some of the more obvious reasons why Agile, as I'd described it up to that point, might seem like pure poison to a group of project management enthusiasts. You never know how that sort of thing is going to fly, but at least they played along and it kept them having to sit still for the entirety of the 2 hours!

I had 8 aspects of Project Management that I discussed, in turn, showing how a PM would add value in an Agile organization while doing the role in a manner different than what they might expect:

• Requirements Gathering
• Task Estimation and Tracking
• Project Planning
• Personnel "Management"
• Quality Assurance
• Customer Relations
• Status Reporting & Milestones
• Scope Management

Each of these prompted questions from the students, none of which were silly or indicative that they didn't understand the subject matter. They were simply absorbing it all, quite quickly, and beginning to draw conclusions or anticipate problems. Fielding all of their questions, including several by the course instructor himself, was definitely the most challenging part of the evening for me. That also, of course, made it the most rewarding!

Both of our presentations appeared to go over very well, and we came away feeling as if we'd done a good job. Those of you who know people in the course are free to seek out differing opinions, though!