Why Transparency Matters

Sometimes it astounds me just how much transparency exists in the world today, and how much benefit we derive from it. In some cases, it's the incredible access to information that we have, mostly thanks to the Internet, while in others, it's simply how willing people are to share what they're doing.

As examples of the former, I started making a mental this morning of some of the data that's now almost literally "at my fingertips" that would have, even a decade or so ago, been at least a phone call or trip somewhere away:

• animated radar images of the current precipitation patterns in my locale, so that I can plan my biking forays accordingly
• real-time, up-to-the-minute sports scores, including details like who's pitching, who assisted on what goals, and how many shots have been taken
• specifications for devices like pool heaters and DVD players that you may have long since lost the paper copies of
• forms, as well as forums, related to topics like Canadian Income Tax, immigration policy and travel destinations, just to name a few
• dictionary and encyclopedia content enabling anyone to write a factual, typo-free essay (or blog post) without having to even get up out of their chair
• transit system schedules so that you know exactly when to be where to catch what (bus/train/plane)
• and thousands and thousands more

Similarly, we're able to get much greater insight into, say, the creative process behind a television show, movie or comic book than we ever could in the past, thanks to creative folks who "put it all out there" for the world to see. Not that long ago, much of that would have been closely-guarded secrets, or simply considered too pedestrian to warrant the time to write it down and the cost to publish it. Now, however, it's just a few clicks away.

So what does any of this have to do with Agile, or software development in general? Well, I think that we're realizing many of the benefits of transparency in the 21st century (as touched on above), but that doesn't necessarily always get applied to how we run our development projects. In an Agile culture, we strive for transparency in some very big ways - Burndown Charts, strictly-prioritized Product Backlogs, Retrospectives - all of which are important and positive. If you compare that to a more traditional (Waterfall-style) software project, though, you can see that the same degree of transparency is seldom achieved there. Team members may seek to hide some of what they're doing out of fear that the work won't be approved if the bosses find out about it. Project coordinators may talk about, or even hope for some sort of post-mortem to be held after the product finally ships, but rarely do those mythical meetings actually happen. Status reports all too often end up being "green-shifted" by those responsible for providing the data, in order to keep the alarm bells from sounding early on. These are all instances where the lack of transparency hurts the success of the project.

Even in an Agile environment, however, there can still sometimes be a resistance to openness. I certainly encountered many situations in my two years as Agile Manager that spoke to this point. Whether it be shutting down conversations or blogs that are bringing to light unsavoury failings among management, or a reluctance to share with others the challenges that are being encountered (because it might be used against you later), it's no easy task to adopt a culture of transparency. Politics can get in the way, as can pride, fear and an exaggerated sense of self-preservation. None of those are trivial obstacles to overcome.

Unfortunately, in virtually every case this tendency toward hiding facts ends up doing more harm than good. Decisions end up being made based on faulty data, morale problems develop thanks to a cloak of secrecy that seems to always inspire conspiracy theories much worse than the reality of the situation, and people waste time going through hoops trying to find "the person with the answer." When I think about how efficiently I can use my time today, thanks to the wealth of information available to me, compared to what life was like when I first started working... it's like night and day. And that's because, given quick access to the necessary data, it's always easier to make a reasonable, informed decision. That perspective sometimes gets lost in the heat of the office battles, though. Those of us in the Information Technology industry need to remember that, as we go about our day-to-day business of cranking out top-notch software products, information is king. And the king belongs to everyone.

So if you're reading this from your desk at work right now, ask yourself: what bit of data are you holding on to right now, that would better serve the company if it were shared? Don't betray any confidences (personal or professional), but rather look for something that you've simply been lax in getting out there, or that you know would help someone else if you made the effort to tell them about it. Sometimes that's all it takes.

Learning May Actually Help Keep Us Young

I'm reading a book called The Brain That Changes Itself, by Norman Doidge, M.D. In it, the author describes many different examples of the human brain's neuroplasticity, which is to say: its ability to change itself based on various stimuli. For decades, the belief was that each part of the brain had a certain, unchangeable function to perform. This implied that an injury to any part of the brain meant the loss of that type of functionality, forever. More recently, though, neurologists have begun to uncover many examples where the organ inside our skulls is actually much more adaptive than that rigid model would allow for.

One of the topics that Dr Doidge covers which certainly applies to Agile is the importance of continuing to learn, even after completing the scholastic period of our lives. In studies that have been done with those in their sixties and beyond, there's clear evidence that elderly individuals who seek out ways to engage their mind - such as by learning a new language or dance, taking on a new hobby or even just visiting locales that they've not been to before - are less likely to suffer from dementia and other symptoms typically related to "the aging of the brain." New neuro-connections continue to form as we get older, but only if we exercise our mental facilities during that latter stage of life. As Dr Doidge points out, it's still possible that it may be a correlation rather than causation relationship, or even causal but in the other direction. In other words, those who are actively keeping their minds fine-tuned may only be able to do so exactly because they aren't losing their edge (rather than the activities being responsible for the sharp minds). But research is making that seem less and less likely, and so I'm inclined to believe that there is more of a positive cause and effect relationship at work in those scenarios, and certainly the rest of the book is a tour of amazing brain-alterations that have been observed by researchers.

So if we accept - optimistically, perhaps! - that learning really does help to fight off some of the effects of aging (such as senility and dementia), then how do we adopt a way of life that promotes learning? In my capacity as Agile Manager between 2006 and 2008, I encountered a lot of individuals who truly epitomized the answer to that question. For them, the focus was always on discovering new ways to make their product better. Often, the people around them (especially in management) were fixated on "hitting a date", which often translated to "cutting corners." The friction caused by the intersection of those two very different approaches was probably healthy, in the final analysis, although it rarely felt that way at the time.

In life, though, we're less likely to constantly run into schedule pressures in quite the same way as we do in our vocation as software craftspeople. Granted, there may never seem to be enough time in the day, but we still get to decide - within reasonable bounds, set by family obligations - just how to spend our spare time. Do we "veg out" and turn our brain off every chance we get, or do we look for ways to stimulate our mind and imagination? Are we always satisfied with the way things are, or do we look for paths to improvement? Have we locked ourselves into a comfortable pattern, or are we still growing in ways not related to our waistlines?

I obviously have a lot more free time now than I did just a year ago. Even so, I've made a conscious effort to spend significant portions of it reading, writing and generally trying to bring my overall understanding of the world around me to a higher level. I'm also finding that my "second career" as a Math tutor is requiring no end of learning on my part (how's that for irony?) as I discover that every child really is different, and therefore no one approach works for every situation.

Will any of that do any good? Am I still destined to end up senile (or, as AgileBoy would put it, "more senile")? I don't have any answers, but I think it's important that we all all keep asking questions.

How Can Productivity Really Be Increased?

As our current Recession continues to wreak havoc across the employment ranks, one topic that we'll all surely hear more of is "increasing productivity." It's the conjoined twin of "reducing costs," after all, which has already proven to be a popular refrain since last Fall. Once a company gets to the point where it doesn't believe that it can effectively reduce the workforce any further, the next logical step is to look for ways to get more out of whatever employee base remains. I suspect that that's already happening; if it isn't, then it probably soon will be. (This, too, shall pass... but maybe not anytime soon.)

I want to say right off the top: increasing productivity by making people work longer hours isn't the answer. It may work in the very short term, but it certainly doesn't over the long haul. All kinds of studies have proven this, to the point where I'm not even going to bother citing them here (this stance against sustained overtime should be accepted by now, I hope).

So how do people become more productive? Since this is an Agile blog, I'm talking here about the so-called "knowledge worker" industry, rather than something more manual-based, where productivity improvements often take the form of automation. It's much harder to replace a Computer Programmer, Project Manager or Business Analyst with a machine or a piece of software, so just how are productivity boons realized?

The answer that was already floating around when I first hit the industry back in the mid-80s was, "Work smarter." Now, it was also the case that not too many people had any notion of specific ways in which to accomplish that, but it was still a worthwhile goal. What I learned, over time, was that working smarter required a certain amount of reflection and adaptation, both of which fit quite nicely with the principles of Agile. Specifically, you could increase productivity if you were willing to root out, and then call out, parts of your process that were inefficient. The Toyota Lean Manufacturing approach essentially crystalized this mindset and represents the most pure application of it that I've yet encountered. It's one of those things that's easier said than done, though, and it requires a superior style of leadership that's quite frankly lacking in most organizations.

But even if you work in an environment that's not based around Lean, I think that you can still apply some of the same principles. Some of your success will be limited by how much empowerment exists, since making change in an empowerment vacuum is, if not impossible, at least ridiculously difficult. Therefore assessing the latitude with which you can effect change is probably the first step to take.

Assuming that you really do want to become more productive and you have at least some wiggle room within which to operate, then the next thing to do is look for obvious inefficiencies in your current processes. In Lean, I believe these are referred to as "waste." One way to spot these opportunities involves doing some sort of workflow analysis, paying particular attention to items that either introduce delays, or which seem to have limited value. There's more to it than that, but generally speaking you're taking a critical look at "how stuff gets done" and trying to identify those areas where the effort required isn't exceeded by the reward delivered.

As an example, imagine a process in which one step requires a meeting between representatives of several different departments, during which the various attendees provide feedback, and ultimately approval, on Business Requirements. It might be the case that setting up such a meeting often introduces a delay of up to several weeks, due to the lack of availability of the players. Therefore, you might actually be able to shave a project's timeline considerably if you could find a way to gather that same feedback and approval without requiring everyone to physically attend a meeting. Perhaps it could be done electronically, or in two or more smaller groups... regardless, the goal would be to see if it was possible to accomplish the same outcome without incurring the large elapsed-time cost.

In my capacity as Agile Manager, I saw something like this done with Code Reviews. The acquisition of a product called Code Collaborator enabled two improvements to occur: first, a dramatic increase in the amount of peer reviewing of code changes (thus, improving quality of the delivered product); and second, allowing developers and testers to review code at their desks, during breaks between other activities. The win-win nature of this change, whereby both the product's quality and the impact on the schedule were positively affected, made it an outstanding success story.

One of the biggest challenges to making changes toward improved productivity, of course, is that it involves change. As anyone who's ever moved to Agile knows all too well, change is hard. Some people resist it, and many of those individuals are quite adept at throwing up obstacles to any change that they feel threatened by. The introduction of Automated Tests, for example, may represent a huge peril to anyone who's built a career around manual testing. The trick, of course, is in helping those people see what sorts of operations Automated Testing is appropriate for, as well as re-purposing the human brains that used to do that work to something that can't be automated. It's not easy, but it can be done... and the end result is improved productivity.

Other types of change that may meet resistance involve switching up how others do their job. Even something as seemingly trivial as moving from a big meeting room setting to doing something electronically may cause some folks to recoil in disgust. They may actually like attending meetings, for one thing. But beyond that, they may have legitimate concerns about how effectively a set of requirements can be reviewed without the back-and-forth discussions that they're used to in those meeting rooms. How you handle those objections will go a long way toward determining how far you get toward your goal of productivity increases. As someone well acquainted with "inspect and adapt" these days, I recommend that you set up any alteration as an experiment, and then monitor the results so that everyone can see whether it's working out or not.

I think a basic question to be prepared to ask at every step is, "What value are we getting from this part of the process?" That can be an unpopular query to pose, especially when you get to the part where, for example, Bob's entire raison d'etre is called into question. You can bet your bottom dollar that Bob will have a handful of justifications at the ready for why filling out that form in triplicate (two of which get shredded) is a key component to the checks and balances of the entire company. Being able to handle situations like that require more tact than I could ever advise on, so don't be afraid to seek out your best diplomat in management if you find yourself down that path.

The bottom line, I guess, is that you can make productivity gains, if you put your mind to it. It just requires an analytical approach and a willingness to change. One of the alternatives, of course, is that all of your jobs get outsourced to a country where the knowledge industry works at less than half your rate... and nobody wants that, I'm sure!