In this interview, recorded at the Agile 2009 Conference, Cyndi Mitchell, Managing Director of ThoughtWorks Studios, discusses the company's new Adaptive ALM (Application Lifecycle Management) offering, which was unveiled last month. According to Cyndi, the transition to TDD (Test-Driven Development) can be challenging for many organizations; however, with a strong, technical foundation, sound engineering practices and support from the business unit, companies can successfully transition to Agile. The 'holistic approach' to Agile adoption, as she calls it, not only requires good planning but also needs to be driven by a strategy that encompasses TDD, pairing, refactoring, continuous integration, and continuous deployment.
The complete transcript of the interview has been provided below.
DZone: Cyndi, can you tell us a little bit about some of the things that you are doing at ThoughtWorks Studios?
Cyndi Mitchell: Yes. So I'm the Managing Director of ThoughtWorks Studios, which is our product division. We set up about three years ago to start creating the next generation of products for Agile software development.
DZone: In your opinion, how has the Agile Tools market evolved over the last five years?
Cyndi: Well, actually that's an interesting question because one of the reasons why we actually set up ThoughtWorks Studios is because we felt that the tools market was really lagging behind what's actually needed to do good Agile software development. And so, we noticed that the tools that were on the market that were addressing the maybe 'old school,' more prescriptive waterfall type market, weren't keeping pace with the new practices that are out there for doing Agile software development. So things like TDD and refactoring and iterative planning and those sorts of things. That was really our motivation for setting up ThoughtWorks Studios in the first place.
DZone: ThoughtWorks Studio recently consolidated its three products (Mingle, Twist, Cruise) into an ALM offering. Could you provide us with an overview of these three products and what it means for them to now be 'ALM products'?
Cyndi: I guess the first thing I would describe is what the Agile ALM market means to us. There are probably two key things. One is that it needs to be adaptive in the way that Agile methods are adaptive. The idea of taking feedback and changing and continuously improving the way that you work. The tools need to be able to adapt to the way that the teams work rather than forcing the teams to adapt to the way that the tools work.
The older, waterfall-centric tools haven't quite caught up with that concept of being able to adapt to the way that teams work. So, that's the first thing. The second thing I would say is the need to address the entire set of engineering practices that go along with Agile. You can't do good Agile software development without doing good engineering practices. And a lot of the tools out there are focusing much more on the planning practices, which are important and necessary, but not as much on the test-driven development, pairing and refactoring, continuous integration, continuous deployment, and those sorts of things. We very much believe in a holistic solution that is adaptive and also addresses the entire set of Agile practices from the planning through to the engineering practices.
So with that, we have three tools on the market that integrate together to become an adaptive ALM. The first of that, which is our flagship product, is Mingle. We released Mingle in 2007. It is a project management tracking and collaboration tool. So, a place for teams to come together and plan and track and integrate over their work.
Late last year, we released Cruise, which is a continuous release management tool. So, really applying the concepts of continuous integration to that last mile. How do I automate the process of changing software and pushing those changes through all the battery of validations that they have to go through and push live into the production environment? Then there's continuous release management - shortening that last mile from in many cases months in many organizations down to just a few minutes. So that's Cruise.
Then Twist is our most recent release, and kind of the final piece in the crown for our adaptive ALM. Twist is an automated functional testing tool. It's geared towards bringing the whole team together around functional testing. Making functional testing the responsibility of developers, business analysts, and testers, helping them to create test suites and keep them in sync. Keep tests and specifications in code in sync with each other over time. Taking away the brittleness of the functional testing that we see in so many big enterprise projects and allowing teams to evolve and make tests long-lived, maintainable assets to help you maintain the system over time.
DZone: Have there been any challenges getting so many people across a cross-functional environment to use an integrated suite of tools? How difficult is it to integrate functional testing into a project management tool, and vice versa?
Cyndi: That's interesting. The approach that we have taken with Twist is to actually integrate not just functional testing. Twist addresses functional testing as well as functional specifications as well as code. You're using Twist to author specifications almost as executables, taking a sort of DSLish approach to write authoring specifications and tests. So that's already an integrated environment. We started with the concept that these two things should be integrated. The barrier between translation from what a business person needs and what a tester needs to validate should be very, very small. If they're using the same integrated development environment, if you will, then you actually don't have the barriers of translation that you might have had in the past.
DZone: How do organizations typically assess their need for a product like Twist or Mingle? At what stage of my company's Agile transformation do I typically begin to evaluate more sophisticated tooling?
Cyndi: Particularly, in the case of Mingle where you are talking about project management and tracking and collaboration, a lot of Agile teams start off with the good old fashion sharpies and sticky notes and note cards and those sorts of things. And that's actually a fine way of working even for the duration of a project. Where we see something like Mingle really coming into play is when you want to scale that process, particularly when you want to scale to a non-colocated development environment, which is common in most large enterprises that we work in. When you want to be able to do quick roll ups and tracking. If you're working with the traditional cards and spreadsheet model, there is some lag between an event occurring and that real information about what that event was being available to somebody. So, when you need the speed, the real, just in time visibility into what's going on in a project, that's where Mingle would really come into play and help you to be able to get that in real time.
The other aspect would be when there are a lot of granular collaborations happening among different people in a team, whether they are collocated or distributed. And you want to be able to record those collaborations. They are useful and interesting to you from a status reporting perspective or an information reporting perspective. Mingle gives you the ability to record effectively everything that's happening within the team. And then once you have all of that data, there is no end to the amount of real time visibility that you can get into what's going on in the project. So, that's probably the short answer that I would give for Mingle.
In the case of Twist, we see this over and over in most of the large enterprises that we work in. The brittleness of functional tests is a real problem. It is very common to see people go through an exercise of creating large suites of functional tests over a period of time. And at some point in time, people are just throwing out the tests. They just can't bare the overhead of maintaining them. They break all the time. It just becomes too much work to keep the tests in sync with the system that's changing over time.
Twist for a variety of reasons addresses that problem by decoupling the implementation, the mechanics from the intent of what's actually meant by a particular business scenario or business test. If you are in a position where you are suffering brittleness of functional tests, and that's pretty much every enterprise software environment that I've ever been in, Twist is the solution to that problem.
DZone: In addition to functional testing, does Twist also facilitate testing for areas related to security, performance, scalability?
Cyndi: That's within the road map. We've got some plans for things we're going to do specifically around performance testing. In the 1.0 release that we're on now, we don't specifically have performance testing, but we're moving down that path in our road map. And we do obviously have with Cruise, which facilitates this deployment pipeline concept, the concept of making a change and moving that change through functional testing, performance testing, whatever types of operational acceptance testing, like social security for example, and then pushing that change along.
Cruise is a place for operations people as well as software teams to come together and define what are those requirements. And actually define stages in a pipeline in which those requirements will be validated. And then, of course, altering the tests to plug into those environments so that you can ensure that the non functional as well as the functional are addressed as a part of the push of software live into production.
DZone: Do you have any interesting insights from customers in terms of some of the common pitfalls you're experiencing when bringing a test-driven approach to their organization?
Cyndi: We see a lot of organizations now embarking on big Agile transformation initiatives. As a part of that, they come to the Agile world looking at it from the planning practice world. And they say OK, a little bit of planning, adaptive planning, structuring my work and interactions. I get that, I understand that. That's something I think we could try. I can send my people on a certification course. They can learn this Scrum stuff pretty quickly, and they can start applying these planning practices.
The engineering practices, these are something that's a little bit more complicated. I don't even understand what they are yet. Maybe I'll just wait on those, and maybe I won't even do them at all.
I think that's probably the big antipattern that we see it a lot of large organizations start to push into the Agile spaces. They're leaving behind the good engineering practices. Obviously that's a shame. Because those practices are there to give us the feedback, the ability to adapt and change as we go. Because we injected all these little feedback loops into the process as we've gone through.
So I think in general - I wouldn't limit it just to TDD - in general, the engineering practices are being left behind in a lot of large enterprises as they go down the Agile journey. And that's something that we're very passionate about reversing.
DZone: So, it sounds like Agile adoption should really be driven by engineering and not so much the business folks, is what I'm hearing.
Cyndi: Well, it's got to be a holistic approach really. I don't think one is any more important than the other. You still need business people who are on board with this new way of working. The way you think about governance and budgeting and all that needs to change as well. It's just that I think those things are easier to understand in some ways. Programming is actually really hard. When you first come to the concept of TDD, it's a very weird thing to get your head around. I think in some ways, because they're so difficult, they just kind of get left behind; "We'll wait and see, let's see how we go, let's take this other stuff that we get now, and get moving with that, and we'll come to the other stuff later."
And what, unfortunately, many organizations that we see are not coming to the other stuff, just sticking to the plan stuff. But, they're both equally important. We're very much advocating a holistic approach.
DZone: Now, in addition to the performance stuff you talked about, are there any additional features we can expect to see in the coming release of your product suite?
Cyndi: Yes there are. [laughter].
DZone: We're going to find out about those shortly. What size of development team, Cyndi, would ThoughtWorks Studio primarily target?
Cyndi: All sizes really. We have anywhere from large global organizations doing large global implementations and large programs through to smaller sort of colocated Agile teams. I think both of them get differing advantages and value out of the tools from that perspective. I think the important thing, for us anyway, is to make sure that it would be suitable for all those types of environments. Because again, what we're trying to do, what we have done, is create tools that adapt to the way that teams work. The teams should be able to continuously discover and improve their process as they go.
Particularly, in the case of Mingle, we see organizations who are rolling out Agile across their entire enterprise. And they don't want to do it all at once. They want to start with a few teams here and there. Let these teams continue their way of working. Maybe those teams over there will never need to go to Agile. Maybe it will make sense for them to always be doing something more predictive.
Because our tools adapt and mold to the way that the teams work, we see our customers using it in Agile environments like Scrum and doing XP and other types of hybrid, Agile hybrids, as well as using it for predictive and waterfall approaches, because the tool does adapt to the way that the team works.
Obviously the advantage to the organization is that they can roll up across all those different types of projects to a single enterprise dashboard, and do reporting regardless of whether or not they're running new Agile initiatives, old Agile initiatives, or predictive initiatives that may never be agile at some point.
Through our adaptive philosophy we've built tools that will work for all different types and sizes of teams and organizations; both agile and non agile.
DZone: What would you say makes ThoughtWorks Studio unique in the market today?
Cyndi: Well, I think we're starting from a very privileged position. ThoughtWorks have 16 years of custom software development and experience. And we were the early pioneers of Agile and large enterprise space, and a lot of leadership in that space, and a lot of experience doing open source development for tools in the Agile space. We're coming at this from a large history of actually doing enterprise software development with Agile methods. And that's really helped us to inform what we think is a very exciting set of tools and exciting solutions for our customers.
So, not only are we coming at from real world experience, but we are also coming at it from the ground up. Many of the other providers are buying and acquiring tools and linking them together. We're actually building... All of our products are native. We built them from the ground up with the same adaptive philosophy, the same holistic philosophy of engineering practices are important.
So, you've got the same themes and philosophies running through all the tools, as opposed to kind of a mix and match approach. That's something that we've seen is a real gap in what's out there in the market.
DZone: Cyndi on behalf of the DZone community, I'd like to thank you for joining us today.
Cyndi: Thank you.