Agile Development On a Fast Track
Choosing iterative development over sequential methods brings advantages in global services delivery — but also trade-offs. Experts explain how to get beyond the hype and evaluate agile's business value
It's true... StarSoft Development Labs and one of our top clients have the privilege of being recognized in the January 2007 issue of Global Services Magazine. Being on the cover is important but more important is the fact that our focus on Agile development is gaining strong recognition in the industry.
StarSoft has made significant investments in this development approach over the last few years and several of our clients are reaping the rewards of this with industry-leading software, healthy project teams, best practices and recognition from the founding fathers of Agile, i.e. Jeff Sutherland and others.
To look at Ben Battle, you wouldn’t think he was an old-fashioned guy when it came to software development. The team helping his firm with software development is in Russia. That St. Petersburg-based group is a big proponent of agile programming — an increasingly popular method in which software developers analyze, design, implement and test simultaneously. The point of that methodology is to deliver pieces of code iteratively and get feedback from users constantly. It strikes some as a bit, well, chaotic.
Battle, on the other hand, likes structure. He spent the 1980s steeped in agile’s doppelganger — the waterfall method in which specifications precede programming. “I’ve always been driven by architecture and requirements,” says Battle, the vice president of product development for Boca Raton, Fla.-based ScriptLogic, a developer of network-administration software. “I started out as an engineer on large communications systems with thousands of distributed nodes. Specifications were necessary because the user view alone was insufficient.”
Battle was first introduced to iterative methods in the late 1990s. At ScriptLogic, he and his team have successfully melded one methodology that thrives on structure with the iterative advantages of agile processes. “I always thought structure was left out of the agile methodology, which is why we overlayed the Rational Unified Process, which explicitly defines [guidelines] for project planning and to produce initial builds of software.”
In a way, the aptly named Battle embodies the struggle among software developers when it comes to combining outsourcing with cutting-edge development methods represented variously by the terms extreme programming, agile development, and scrum (See Box Terms of Development). Even some experts who think agile is the perfect methodology — especially for a world in which requirements change as quickly as the competitive landscape — aren’t convinced that it’s suited for the context of outsourcing. Others maintain that it’s madness to outsource software development using any other methodology.
 |
“Agile is very tactical. When you look at two-week iterations, |
The truth is, you can screw up any development project, no matter where the development takes place or what methodology you use, if you don’t manage it properly. So can agile programming work for you and your offshoring team? We’ll look at both sides of the question and show what people like Ben Battle are doing to be successful in their efforts.
All In Favor
According to its proponents, an impressive list of global companies are using agile programming in some way: Allstate Insurance, BT, Capital One, Google, Intel, Fannie Mae, Sabre Holdings and Yahoo, among others. According to Forrester Research data, some 17% of companies in North America and Europe are using agile, up from 14% in 2005.
Set in the context of global business, the interest in agile is understandable. For instance, agile is designed to accommodate changing needs and changing minds as the project progresses, rather than adhering to a static list of specifications. “When you come to the end of a waterfall project, the customer says, maybe that’s what I said I needed, but that’s not what I need now,” says Robert Martin, CEO, Object Mentor, a Gurnee, Ill.-based development and consulting firm. “Even if the customer gives perfect requirements, in a few months the business environment has shifted to make the requirements invalid.”
When you fold the communication revolution into the mix, it’s easy to see the attraction in the offshoring context. “To some people, the idea of offshore agile development is an oxymoron,” admits Peter Vaihansky, a vice president of Cambridge, Mass.-based StarSoft Development Labs, whose St. Petersburg team is helping ScriptLogic. “The agile manifesto values interaction over documentation. People have to be in the same room and get things done fast without writing stuff down.” But the increasing availability of collaboration tools and decreasing costs of communication eliminate that obstacle.
Many people believe that well-managed offshoring projects interleave quite neatly with agile programming, because of the way its iterative nature dovetails with follow-the-sun work. Battle believes that agile is suited for a distributed development model. It works well for ScriptLogic in part because the StarSoft team has adjusted its hours to be more available more during Florida business hours. “On the flip side, they’re able to produce and test build before we start our business day,” says Battle. He also believes that it requires management discipline, especially with an offshoring team. “The team knows what they need to do when. In a distributed model that’s key.” He also stresses the need for collaborative tools for planning, defect tracking and communication.
Martin of Object Mentor asks, “If you are going to hire a group of developers halfway around the world in a time zone 12 hours away, how should you work with them? Should you give them a huge requirements document and wait to see what they give you in six months? Or should you ask for something every week?” When it’s difficult to communicate, you need to shorten the feedback cycle and measure results frequently.
That’s what makes using waterfall dangerous in offshoring, according to agile proponents. So named because work is done sequentially between the four phases of development — one cascades into the next — that very aspect can cause problems. “When you have a handoff, there’s always the opportunity for a defect to get injected into the code, particularly true when it’s difficult to communicate because of time zone and cultural differences,” says Scott Ambler, the practice leader for agile development at IBM Rational. “With handoffs, you need more documentation, which raises costs and slows you down.”
Because errors can perpetuate themselves, a waterfall project can go awry. “But you don’t realize it’s out of control until late in the process,” says Ambler. “You end up with a false sense of security because there aren’t any warning signs. The feedback cycle isn’t there.” But because the team is following the specifications, developers find it comfortable.
Additionally, because agile programming focuses on creating and integrating code on an iterative, ongoing basis, proponents insist it can accommodate the work of larger teams. Frequently putting more people on a project to achieve a goal — a common motivation for adding offshore teams — introduces flaws into software more quickly. But because code is integrated constantly, problems show up, and thus get rectified, sooner. Jeff Sutherland, Chief Technical Officer, PatientKeeper, a medical software developer as well as a consultant on agile programming, cites a Danish outsourcing firm he worked with that reduced its defect rate by 40%, its planning costs by 80% and increased its overall productivity by 50%. If you’re looking for scalability in team size, Sutherland insists, agile can give you not just incremental improvement, but linear improvement.
Nonetheless, proponents admit that just because you use agile programming doesn’t promise success. But using it helps you identify a doomed project earlier, and thus save money. As Ambler puts it, “I’d rather fail two months into a two-year project than three years into a two-year project.”
All Opposed
Even so, some agile proponents are still unconvinced about the method’s value for globally delivered projects. They cite communication obstacles, hidden costs, cultural issues, and inertia as reasons for their reticence.
Communication obstacles. Even with technological advancements, communication barriers can arise. Martin Fowler, CEO of ThoughtWorks, a Chicago-based global development firm, isn’t completely convinced about the value of offshoring. “I take the view that IT development is done pretty badly. If your communication is screwed up, then offshoring to India isn’t going to improve the situation. I suppose if you’re going to screw up anyway, you might as well pay less and screw up.” He also believes that the lack of time zone overlap between the U.S. and India is a problem, something that isn’t as true of the U.K. and India.
Terms of Development
• Agile. A method of developing software in which design, analysis and testing take place simultaneously within specific and short milestones (daily, weekly, bi-weekly) so that customers can see results and offer feedback or make changes throughout the process
•Extreme Programming. A method of developing software designed to accommodate changes in customer requirements, even if they’re requested late in the cycle; it emphasizes teamwork and communication
• Scrum. A method of developing software created by Jeff Sutherland in 1993 and designed for teams, so named because it resembles a rugby scrum
• Sprint. The time in which developers have to complete their tasks before the next milestone
• Waterfall. A method of developing software in which each step of the process takes place sequentially, so named because it resembles a set of waterfalls cascading into a series of pools.
“At the end of a waterfall project, the client says, maybe that’s what I said I needed, but that’s not what I need now”
Robert Martin, CEO, Object Mentor
Hidden costs. “You have to have people on this side who can interpret the work being done on the far side,” adds Bob Martin. “People have to look at the code, understand it and make sure it works the way it should.” If you decide to have the kind of person Martin describes working at the offshore location, you’re going to incur higher travel expenses to overcome communication difficulties.
Cultural issues. In Asia, there is a strong cultural inclination toward deferring to superiors, while agile programming requires developers to push back if they don’t think they can accomplish a task within the allotted timeframe. “At ThoughtWorks we encourage pushback and have an argue-with-anybody culture, which is different than it is in most IT shops,” says Flower He adds that he’s worked so hard to develop a culture that encourages anybody to make a fuss, ThoughtWorks’ developers in India have gotten a reputation for being difficult to manage. “We take that as a plus,” Fowler chuckles.
IBM Rational’s Ambler worries less about insincere agreement. “If they’re constantly saying yes to what they can do, they’ll just get caught two weeks later when they can’t deliver. You have to be smart about the way you manage the teams, and that’s what puts offshoring at risk. People on the client side either overmanage or undermanage, and that’s a serious problem.” Sometimes, he admits, the agile community focuses solely on development issues; with offshoring, you have to look at governance and project management as well.
StarSoft’s Vaihansky, not surprisingly, takes a different viewpoint, given that his teams are in St. Petersburg, Russia; Kazan, Russia; and Dnepropetrovsk, Ukraine. “The advantage of working with Russians is that you get smart, opinionated people. The disadvantage is that you have to deal with smart, opinionated people.” That’s why he believes in offshoring agile development — just not to India.
Inertia. In some cases, the use of waterfall development is simply self-fulfilling. It’s what companies have used for 30 years, and it’s what they’re accustomed to. “A lot of outsourcing companies expect you to give them specifications,” says Steve Mezak, CEO, Accelerance, a Half Moon Bay, Calif.-based consulting firm focused on global services delivery. “They’re more geared for that because it allows them to put more junior engineers on the project.” To work collaboratively, he adds, developers need to be both mature enough to know that they have to be inquisitive and ask for clarification. Those kinds of people aren’t so easy to find.
Carey Schwaber, a Forrester Research analyst (whose father Ken is coincidentally a key proponent of agile), spent time in India early in December 2006 specifically to gauge service providers’ capabilities for undertaking agile development. “Two years ago, they didn’t know what agile meant in India,” she says, “but there’s definitely an appetite for it. You have to figure out if your provider has enough agile expertise, and if not, can you teach them?”
Still Undecided?
Agile proponents acknowledge that while the concept of so-called extreme programming has been around for years, it’s only recently that momentum has been growing in its favor. The impetus? The growing need for fast innovation among companies, and the increasing importance of cutting-edge software and technology. That’s giving techniques such as agile programming a foothold in more companies, especially those willing to try anything to boost productivity.
The answer, in Battle’s mind, lies in compromise — using agile’s precepts but adding enough structure to manage it. “Agile is very tactical. When you look at two-week iterations, you’re not looking far out,” he says. “If you’re building software based on a market need, you need to do some upfront planning.” In fact, the only way he’s been able to succeed, he believes, is by taking the best methods of development and making them together. “I’d like to see the agile community address the architecture question more. It’s something that’s missing.”
Battle is on the right track, Forrester’s Schwaber believes. “There is no out-of-the-box process to use. Everyone builds a custom methodology, and they all think they’re the only ones that do it.”
That’s the way to make global services delivery work — combine the methods of development, communication, and collaboration that work for you and codify them. It requires a more rigorous oversight, but most development projects — and certainly most offshoring partnerships — can benefit from that. Global services delivery is not going away, and neither is the need for developers to be responsible to business. Ultimately, agile development may aid both these efforts.
