The No.1 i-Technology Magazine in the World !
   
 
Timothy Fisher

Search Box

 

About me

Timothy Fisher
Flat Rock, MI USA
blog.timothyfisher.com

View Timothy Fisher's profile on LinkedIn



Add to Technorati Favorites

Subscribe



Subscribe with Bloglines

Mailing List

Visitor Locations

My Java Book

My Tumblr

Blog Status

  • 3 yrs 30 wks 3 days old
  • Updated: 3 Jun 2008
  • 22 entries
  • 74 comments

Hit Counter

Total: 259,302
since: 19 Jan 2005

Extreme Programming (XP) Unbiased

posted Friday, 29 April 2005
Extreme Programming (XP) has been a very hot topic over the past couple of years now. Unfortunately, while there has been much written about XP, most of it has been extremely slanted either in favor or in opposition. It is not easy to find an objective, unbiased review of the core practices espoused by XP. This article attempts to provide a neutral and unbiased review of XP, neither completely praising it, nor completely slamming it.

Recently, I spent a year working as part of an XP team. This gives me a perspective of XP that many of XP's critics lack. People will always question a critique of something that has not been tried.

At the heart of XP are its 12 core practices. These are the planning game, small releases, system metaphor, simple design, continuous testing, refactoring, pair programming, collective code ownership, continuous integration, 40-hour work week, on-site customer, and coding standards. We'll touch upon some of these in discussing the good and bad of XP in the remainder of this article. Those that we don't discuss will perhaps serve as material for a future follow-up article.


The Good

Unit testing has certainly been around for many, many years, but it's certainly not unreasonable to say that we owe a great deal of today's increased emphasis on unit testing to XP. Unit testing was promoted as one of XP's core practices. Along with the recognition of unit testing as a powerful aid to developers came the methodology of test-driven development. This is a development style in which unit tests are always written prior to the code which they are testing.

What XP refers to as the "planning game", is another good practice that can be carried over into other methodologies. XP does a very good job in maintaining schedules, plans, and keeping the plan very current. The iteration and build strategies advocated by XP are very good strategies for any project. Amongst these strategies, XP advocates frequent releases, short and well defined development iterations, frequent builds, and emphasis on build automation. Whether you choose to practice XP or not, these are strategies that add value to any methodology.

Small releases is also a very positive practice of XP. Planning for small releases has many benefits. Getting some functionality in front of a customer quickly and having frequent opportunities to get feedback on released code is certainly advantageous. Small releases can prevent a project from wandering too far off course, preventing the case in which you end up with a project that does not meet the customer's needs.


The not so good

Another core practice of XP asks you to keep a customer representative onsite with your development team so that you are able to quickly work out requirements confusion, get answers to questions, and get feedback on your progress. This sounds great, but it’s the practice I usually refer to as the fantasy practice. We'd all love to have James Gosling onsite also in case we run into any Java problems, but it's simply not realistic, just as having an onsite customer is not realistic in most cases. When you are able to achieve this, absolutely go for it. It's a grand idea, but not one that a software development methodology should rely on as a core practice.

XP also tends to be full of short catch phrases summarizing its core principles. One of those is "Do the Simplest thing Possible." Here they are advocating that you should always implement the simplest, least complicated solution to the task at hand. They specifically warn against coding for anticipated future extensibility and add-ons that may or may not actually happen. While in general, this may not be bad advice, I think making this a core principle of XP is going too far with this attitude. Often, it does make a great deal of sense to spend the time to implement something that, while not necessarily the simplest solution, is a smarter and better solution. Remember this; the simplest solution is not always the best solution. This tenet seems to be born out of programmers who are adverse to strategic design, analysis, and modeling and simply want to code.

Next, we come to the 40 hour work week. This is an admirable goal that I support. Unfortunately, I believe this becomes too engrained in XP programmers' heads, and even on projects in danger they do not feel the motivation to put in extra work hours. Just as expecting a development team to consistently sustain greater than 40 hour work weeks can be detrimental to a project, so can a rigorous standard of only 40 hour work weeks.

Finally, let's discuss one of the most controversial practices of XP, that is the practice of pair programming. This is the practice of having two developers sit side-by-side at a computer and develop a piece of code. Generally, one person is known as the "driver." This is the person controlling the keyboard. The other person provides instant code review, and deeper design input while the driver is writing the code. There have been some research studies of pair programming that claim pair programming is at least as productive as individual programming, and perhaps more so. The problem here is that when people talk about how great pair programming is, they are making a huge assumption about the personalities of those who might be placed into that environment. When you do a study on the productivity of pair programming without considering programmers personality types, you have a huge gaping hole in your research. From my own experience, I know there are certain personality types which are very common in the software development community, for which pair programming simply does not work well at all. Pair programming is not something that can be forced upon a team. Each and every team member must buy into it and be able to work well in that environment. If you go with the attitude that you'll only hire those programmers who are comfortable with pair programming then you are immediately eliminating a huge pool of very talented software developers. So, while two programmers may say pair programming is great, everyone should be doing it, what they really mean is that pair programmer works great for them. Given two other developers with different personalities you might have a disaster. Thus, without taking into account the specific team you have, you can not make any claims that pair programming is superior to individual programming, or even that it’s a good thing. To be fair, yes pair programming may work remarkably well for certain individuals. For those who enjoy pair programming, I believe it may be an option for producing good code to solve most of your typical programming challenges. However, what you'll lose out on with pair programming is your individual heroics that take you beyond just good programming to solve the most challenging problems. Unfortunately, this tenet above all others tends to dampen creativity and heroics in XP. Proponents of XP often say heroics are a negative thing anyhow, but then again, they often prepend their reference to heroics with the term "late-night." I guess they assume most heroics happen late at night.

Now having just read that paragraph, I am certain there will be some readers who are thinking, well those developers who don't have the personality type to be successful in a pair programming team are probably not very good in team environments and you don't want those types anyhow. Here again we have a misunderstanding of what it means to interact with a team and be successful in that environment as opposed to having someone in your personal space working side-by-side with you nearly 8 hours a day. Some of the very best team players would fail miserably in a pair programming environment.

While not an official XP practice, it is a common belief in the XP community that source code alone can serve as the developer documentation for a project. This may be fine for the developers currently working on the project, but have you ever tried joining a project in progress with a large code base and being told that the only documentation for what they've created so far is the source code? Believe it or not, many XPers would say that is perfectly fine, and this is how they document their projects. Again here, we have a failure to consider the future and what is strategically best for the organization as a whole.

XP also does not believe in the need for "experts". Their philosophy is that all programmers should be well balanced on all technologies employed by the application. While this sounds admirable as a goal, the reality of this is that you kill the enthusiasm of an expert, and encourage a mediocre understand of all technologies in all your programmers. It is simply not realistic for all programmers to become experts in all of a project's technologies. Individuals who want to gain expertise in a specific technology, such as security, GUI design, databases, etc. are discouraged.


Conclusion

The best way to view XP is as a collection of principles which may be applied to your software development project individually or in sets. You should not be dogmatic in your views about XP and believe that it is an all or nothing methodology. I'd go a step further even and refer to XP as a methodology toolbox, rather than a methodology. Many experts will tell you that XP is not compatible with large, complex projects and that it's ideally more suited to smaller scale projects with localized development teams. Actually, many of XP's practices are very beneficial to very large and complex projects. It's only when you try to take XP as an all-or-nothing strategy that you will get into trouble on large projects. XP is a very code-centric approach to software development. It lacks higher level controls necessary for larger projects. Pick those practices you want from XP and don't use those that your team is not comfortable with. Integrate those you choose into your overall project methodology and you have the greatest benefit of XP.

Until next time,
Timothy Fisher




1. a reader left...
Saturday, 30 April 2005 12:53 pm

I have also spent considerable time on an XP project, and I generally agree with your take on things. There are a couple of points that I would quibble with, though.

Regarding documentation: the ideas is that you create as little documentation as necessary. Assuming that you're talking about a project that is all contractors that will go away at the end, then you certainly do need more documentation so that someone can pick up the code later.

However, if the project is built with staff who will continue upgrades on an ongoing basis *and* you're doing pair programming, new staff can just be dropped in with the pairs and learn via the shared knowledge.

The documentation bit is pushback on the projects that spend months and months creating requirements docs and design docs without ever building software and learning what really works. There are still many circumstances where documentation is desirable and necessary and a successful XP Project will deliver on that.

Regarding experts: you don't want towers of knowledge on a project, because that causes grief if an expert is on vacation or leaves. But, without a doubt, people on a team will have different strengths and you will have people who are expert in some technologies but not others. So, you pair the experts with the nonexperts, and the nonexperts will learn the technology as they work with it. The expert does not become any less expert as a result, they're just helping the team.

Finally, doing the simplest thing possible is really a tricky phrase that will mean different things to different people. When you're done implementing a story, you should end up with clear, concise code and no duplication. Achieving that can require some thought and experimentation in approaching the problem. Doing the simplest thing possible does not mean shutting off your brain. It just means "don't invent future requirements". Here's a simple, contrived example:

Color background = Preferences.getPreference("color.background");

What if it's not a requirement to have user-adjustable background color? This is exactly the type of thing where a programmer might say "hey, it would be cool to make this adjustable". But, that kind of thing adds a level of indirection, and there's no reason to do it if the customer didn't ask for it.

A more likely scenario is that the programmer will create an interface for something, and then create a single implementation, assuming that somewhere down the line there will need to be another implementation (or worse, creating a second implementation that isn't needed at that stage!). That's exactly the type of thing to avoid. And, in order to do that, you really need to have a good test suite so that you can make sure things keep working when you refactor to add in that interface later, if it becomes necessary.

Lastly, I wouldn't describe XP as a "code-centric" approach. XP is a customer-value centric approach. The whole process is geared around discovering and implementing what the customer wants as quickly as possible.

Kevin Dangoor


2. Timothy Fisher left...
Monday, 2 May 2005 11:06 am

Kevin,

Thanks for leaving the comment. I always enjoy a 2-way dialogue. While I certainly respect your viewpoints, I'd like to call out some points you make that I disagree with.

With regards to justifying lack of documentation by assuming that your development team will be around to pass knowledge through to future developers, this is a really bad way to plan in my opinion. The reality of software projects is that you never know when you might lose your entire team, have the project transferred to another group, etc. Its a mistake to rely on a core of developers (pair programmers, or not) to communicate your documentation.

I understand your points about simplicity and those are points that I do not dispute. My point was that XP places too much dogma on that term "simplicity". I believe at the expense of good design in some cases.

I also stand by my reference to XP as a "code-centric" approach. Amongst the commonly accepted phases of a development project, analysis, design, code, and delivery, XP places most of its emphasis on the coding phase.

Timothy Fisher


3. a reader left...
Tuesday, 3 May 2005 2:26 pm

I too have been doing XP (light) for the last 3 years but in the cheminfomatics domain. We kept most of the practices but binned pair programming and simpless thing possible.

While he is right about pairwise programming not making experts less so, Kevin did not explain how you become an expert in this type of environment. Generally, to develop expertise you need to shut yourself in a room and concentrate on exploring the technology area.

As for the simplest thing possible I would refute this one totally. I agree with Kevin in never add unnecessary features but the phrase "do the simplest thing possible" is just a bad choice. For example we always designed our code for future flexibility adding interfaces where there was no current need and generally desinging in extension points pretty much everywhere. My experience is that we were 'never' wrong to adopt this approach and we always wound up saving time in the long term.

Overall, a good article that reflects my personal experience very closely.

Lewis Jardine


4. a reader left...
Tuesday, 3 May 2005 3:10 pm

Good job, I think this article is very common-sense. There are a lot of refinaments that could be done, like the notes about documentation. The point is where you stop documenting and actually coding? (XP is extreme in this regard) Where is that boundary which assumes that if you go further with the documentation is detrimental to the project as a whole? And the practice of pair programming, I find it unacceptable and unusable. Companies won't hire two persons to do the one's job. And what are you going to have? one person writing the for loop and the other one reading over the shoulder and making sure that "counter" is declared? Frankly I don't see this working at all.
Regarding kevin's note, I find funny statements like "you don't want towers of knowledge on a project..." ("because that causes grief if an expert is on vacation or leaves" !!!???) . Of course you want. Just like you don't want two 10 years old kids (working in a pair :-)) building your house. Or you want...?

ranjix


5. a reader left...
Tuesday, 3 May 2005 5:06 pm

A balanced article indeed. I have worked at a project where pair programming was being forced and I just said 'No'. From my perspective, PP works if project cycle allows enough time for 2 developers to work for less than 2 people independently...

Debu


6. a reader left...
Tuesday, 3 May 2005 7:42 pm

I also agree with the article. I have 'cherry picked' XP for years and have ended up with the same list of 'good'.

'Simplest' is simply to simple to be understood. That was intentionally obtuse. Einstein was reputed to say when asked for a simple solution to a promblem 'Simple! I don't have time to make it simple!'. I see simple as from that viewpoint. Simple is not adding layers over an algorithm to make it work. Simple is re-designing the code to work as required without unneeded complexity. I once worked with a developer for a full day. We reduced 1,000 lines of code into less than 1 active line. He was shocked, while I considered it to be the most perfect development effort ever!

Paul Marrington


7. a reader left...
Wednesday, 4 May 2005 8:11 pm

Ok, here's a different perspective - from a programming manager...we have attempted to have an agile/XP (?) environment for a while now, with some successes and some not-so-hot non-successes. My take on your negatives:

I agree with your assessment of the customer representative portion. What I have found is that if you don't have a customer, one person should represent that customer's needs/viewpoint/etc as best as he/she can. I found that this is necessary to keep stories honest and relevant.

The "do the simplest thing" one or YAGNI (you aint gonna need it): I think that you have to agree with the spirit that triggered this objective. Some developers/teams/companies have a culture of over-design, and do not reward simple designs. I have witnessed 9 month projects that executed 6 months of analysis, 2 months of coding...and no, they were not done in 9 months...so rather than be literal about the rule, be more general: don't overdesign, don't let design send you far away from the ultimate goal.

I agree with you on the 40-hr week wholeheartedly. That's a practice that can easily be abused. Again, its the spirit here that brings the value, not the literal sense of the rule.

On documentation, its another case of oversimplification. I think a better way of phrasing it is: make useful documentation. Yes, that can also be abused, and I've been witness to that. But the solution isn't a gargantual modeling and documentation exercise either. You have to simply resist the temptation that comes with the perception that you are paid by the hundreds of pages of mumbo-jumbo you produce. I liken this to some open-source projects. Some are sparsely documented, some are not. At some point, you come to the realization that the best way to explain a method to a Java coder is to see/show him/her the java code. Having said that, some bureocratic organizations would not have good things to say to you without any documentation. Striking a balance, and not generating 8,000 pages of docs is probably a good strategy.

XP does not undervalue experts, it values shared knowledge. The individual expert is useless if other people cannot benefit from his/her expertise. XP is trying to fight the "use of information as power" by an individual, when the team can benefit from it. I, for one, value vertical expertise, but horizontal knowledge and general expertise comes with it too. This is not incompatible with true XP.

Lastly, my thoughts on pair programming: I agree in principle that some personalities are not good for pair programming. In fact, my biggest concern with pair programming is that it tends to be more rewarding for the novice than the expert. Still, if given a hiring choice between a pairprogramming compatible person vs one that is not, expertise being equal, guess which one I would hire?

And no, the expert/scientist/savant/guru/merlin should not be assumed to be a poor pairprogrammer, because of his/her personality, or preference to work on his/her own. Programming aside, complex projects and tight deadlines require great communication and societal/team skills (as corny as that may sound). Such a guru is not a bad pairprogrammer, he/she is an inferior employee altogether.

value: $0.02.

Ed


8. Timothy Fisher left...
Thursday, 5 May 2005 10:31 am

Ed,

Thank you for your input. I value all feedback. For the most part, I have little disagreement with the first half of your post. Your merely echoing beliefs that I share, in more detail than I cared to go into. Where we part paths is towards the latter half of your posting.

I stand by my statement that XP does not value expertise... Its practices and culture discourage an individual from taking the initiative to gain expertise on a particular technology.

With regards to your preference to hire a pair-programming compatible person over a non pair-programming compatible person, I'd say thats certainly your personal choice, and I dont disagree with your logic if you are a fan of pair programming. But, and this is a big but, if you limit yourself to pair-programming compatible candidates, you are eliminating a very large pool of highly qualified candidates.

And finally, regarding your closing paragraph, I just simply think you are flat out wrong on that point. I've been in this field for a long time, and I've seen some of the very best communicators with excellent social and team skills who would fail miserably in a pair programming environment. I suggest you read this article for further insight - Pair Programming is not Always a Choice.


9. a reader left...
Thursday, 5 May 2005 7:01 pm

Timothy:

Valuing expertise: I partially agree and can respect your position here. However, I would not go as far as to say "discourage". Either developers by initiave, or managers, by suggestion, can accomodate or plan for spikes, which are nothing more than informal research exercises where you can deeply investigate, learn, or concentrate on a specific area. I realize that these are tactical, and that, for example, no deep expertise in "encryption" can be acquired with a single "encryption spike". But if I decide that it is important to have an expert in encryption, I can identify specific teammembers to participate in a progression of spikes across many releases and accomodate/reward/encourage such expertise. Point is: your point is valid, but it can be "worked with" and be addressed.

I also agree that there are technically qualified candidates that are not pairprogramming compatible. And that if a business values those technical skills over the other team/peer communications, then those would be good, contributing employees.

Regarding the last point, I did read the article you linked to, and I must say that I agree with the Loner definition and with the premiss that it is not a choice. I also agree that there are places/roles that such a loner could contribute at a high level in any organization, including one that emphasizes XP/agile processes. It is likely that we would find a role for that employee and try to both get value (for us) and fulfillment (for the individual). However, by part of the core definition, a Loner, in being "incompatible" with pairprogramming, will prove to be less effective, or not effective at all, when the objective is largely defined and influenced by the inherent benefits pairprogramming brings to a table.

One of the things that leads, managers, and indirectly most other team members have to address is risk mitigation and management. From the manager's perspective, an example of a risk is if employee X develops a subsystem in isolation, then we must have employee X document the system in such a way that if he/she, the only person that knows the innards of a system were to leave, someone else could operate/change/maintain the system without an expensive reverse engineering exercise. Ironically enough, this expects very good documentation, which a) seems to be contradictory to the "light documentation" rules of XP (it really isn't) and b) depends of the very subjective definition of "good documentation". If employee X is the only one that knows the subsystem, how can anyone else truly assess the quality of the documentation, in describing methods, structure, and algorithms? Are mounds of generated UML diagrams enough? The only way to answer that question with confidence is to have at least a second person assess the documentation from a perspective that will force the second person to learn the system. Hmmm. We got two people knowing the system after all. Nice.

Now, was that the most economical/productive way to get two brains to know the system?

Back to the risk perspective for a conclusion: more than one human being needs to know the system. More than one implies a minimum of two. As a risk mitigator, I want to ensure that this happens. Pair programming is one mechanism to attack the problem, collective learning is another. My scenario above is yet another one. All valid, in my opinion.

But if because of risk and other reasons I value pairprogrammers higher, then undoubtedly a pairprogramming compatible person will perform at a higher level if the policy were to use pairprogramming as part of the methods to mitigate risk. Hence, under those measures, the Loner would perform at a not-so-high, lower, etc level.

The loner can still provide value, and in that I agree with you. But by the measures and priorities I have presented, the other employee is better positioned to provide the value I expect.

There is not a right/wrong answer here. However, some arguments I read somehow try to defend the loner's way of doing things as the superior and sometimes necessary way of doing some things. In the same way the paired person is not the right, infallible answer in most circumstances, neither is the Loner's.

Ed

Ed


10. a reader left...
Wednesday, 11 May 2005 9:02 am

I have been managing XP-style development teams for a few companies now. Typicially, these XP-style teams are in standard waterfall geared organisations. I fully agree that the "pair programming" mandate of XP is a tough one to encourage/enforce. Even as a manager it's hard to believe in, as dealines stack up and features extend scope.

My re-interpretation of pair programming has become a setup where the "driver" writes the main code on one computer, while the "auxilary" writes the the unit tests for the same piece of code at the same time on a second computer. This allows for the checks-and-bounds that pair programming professes, while also leaving developers to work more independently (yet together). You also get the added bonus of more tangable output (in the way of unit test).

Obviously, to keep things interesting for both developers "driver" and "auxilary" swap roles throughout the life of a project. The act of swapping helps homogenise the code and in the end has produce quality output.

It still requires some hand picking of the developers but you can open your scope more that a traditional XP developer.

Martin

Martin