Many software businesses are frustrated by a decline in the productivity of their software development teams. It seems each release costs more, takes longer, and the business value in the release is less.
It’s an emotional subject that is key to the success of the business, and one with many stakeholders. Management teams know they need to move faster to capitalise on commercial opportunities, and investors are concerned about the impact that missed milestones have on market position and company value.
So what are the key factors that affect development productivity? Over many years Intuitus has performed technology due diligence on hundreds of software centric companies at key inflexion points in their history. Within these we have reviewed a wide range of software development teams and seen how they have contributed, both positively and negatively, to growing the business. This has helped us develop a methodology for measuring productivity.
It would be nice to have one easy answer as to how to improve software development productivity. Some companies pin all their hopes on one particular methodology - more on that later - but, as ever, life is more complicated and during our work with companies we have identified 7 problem areas that have the biggest impact on software productivity.
Different factors affect productivity at different points in a company’s growth. The characteristics that are critical to the efficiency of a startup team are not necessarily the same as those that are causing management conflict in a more mature company. So before reviewing the success (or failure) factors, let’s look at key points in the software development life cycle and the evolving pressures companies face:
Most successful software products take many years to mature to a point where their product revenues exceed their annual development costs. Many products never reach this point. The ones that do are mostly a result of extreme dedication and perseverance from a small team.
As the product revenues grow, the business invests in larger development teams and higher cost individuals.
While revenues continue to grow, there is often increasing frustration with software development productivity. Typically, this results in some common actions. Top of this list is adopting an Agile software development process. Next, or sometimes at the same time, there is often a change in development management. While these actions may improve productivity, without understanding the factors that affect productivity, the gains can be disappointing.
This is the point where revenue growth is levelling out. Only then does it become a business imperative to “fix the product”. Unfortunately, in many cases, the opportunity to address the biggest issues has passed. The result is lost longer term profitability for the product as high software development costs cannot be justified given the product’s return.
The problems we discuss here will have greater or less significance for companies depending on which of these stages they are at. But it’s important to note that the decisions made early on can make big differences a few years down the road. Are you in this for the long haul? In that case, software productivity good practice needs to start early in a software product’s life cycle.
There are trade off decisions to be made. In an ideal world everything could be planned out for optimal productivity, but in reality there are commercial pressures which will force compromises. Nothing is black and white and you have to weigh up the pros and cons yourself and make some tough choices along the way. Software development productivity does not need to decline as a product matures. Tackle these seven productivity killers before they become a problem and the payback isn’t just improved development speed, but also products with longer market life and greater profitability.
1. Using the wrong technologies
It may seem obvious, but the long term impact on software development efficiency from the wrong choice of technologies can bring a premature end to many products. This is more of an issue for younger companies but has a knock-on effect further down the line when they find themselves trapped with the wrong tech.
Fashions in technology come and go as they do everywhere else and some companies can start out with a programming language or development environment because it is the flavour of the month, or because it’s one the founders used at a previous job and know well, or because it seems easy to use, or is cheaper, or any one of a number of other reasons. Then the companies find the technology doesn’t last the course. It’s not being maintained well or doesn’t have the required functionality. Lack of compatibility with the latest browsers and operating systems is a common problem. Or, as the company grows, it becomes evident that there are not enough developers who know the programming language.
Young technology companies are breaking new ground themselves so are more open to using new languages and development environments, but they really need technologies that will live for years. The one exception is if you are developing purely for mobile. This is a different environment. Things have been changing quickly. It’s important to review latest technologies to support your iOS (Apple) and Android (Google) mobile operating systems. Otherwise, using well established programming languages, such as C# or Java and building on industry standard software frameworks such as Microsoft .NET has many advantages.
2. Thinking Agile is the answer to everything
Choosing the right development methodology is important, but it’s not a panacea that will solve all your development problems.
The two most common development methodologies are Agile and Waterfall. While Agile is considered modern and superior, and Waterfall is often labelled as dated, they both have their strengths and weaknesses.
Agile is best for cloud applications that need frequent new releases, typically weekly or less. If making more releases is your main challenge, then Agile will help address that. Waterfall has benefits where the software is complex and needs to be installed on end user devices or servers. In this case the release cycles tend to be longer, in the order of months, and productivity depends on maximising the functionality and quality of each release.
If they are the right choice, then either methodology can provide for an efficient development process. But neither of them is going to fix all your productivity issues. So, if you think that’s it, that the problem is solved just because you’ve picked Agile, then think again!
3. Getting your testing wrong
For Agile especially, if any one issue on this list has the biggest impact, it’s this one, and the seeds are often sown early on that cause problems later. Testing needs to be ultra-efficient when releases are going out every week.
Product quality combined with frequent releases depends on automating the test cases so that they can be run quickly. Payback from automated testing is huge, but the problem is that the initial expense of automated testing makes it hard to justify early on. If you don’t build it in from the start, it’s hard to do it later. Creating the automated test infrastructure early means it can be maintained and built on as the product grows.
The ideal scenario is that developers own the responsibility for building and maintaining the automated test systems, rather than hiving this off to a separate test team. Developers need to own both the quality of the product and the responsibility for making it easy to test. There is still a need for test engineers, but if developers have created the automated test environments, the test team can focus on maximising quality.
4. Lack of long term product strategy
Many businesses do not have a long term strategy for the product. Or they did - once - and have lost sight of it. Instead, short term customer needs drive the software development process. While these changes help drive early revenue growth, many make the product harder to maintain and support, resulting in ever decreasing software development productivity.
In extreme cases, companies may have a small number of very large customers, and have a different version of the product for each one. Unsurprisingly this has a huge impact on overall productivity. Every product change needs to be applied to and tested on all the multiple versions.
Obviously if you have some good customers you want to respond to their needs, but the question is how you do that while still maintaining a single version of your product. You have to make some decisions about your overall product strategy and consider your clients’ requests in the context of that strategy - are they compatible with your long term growth and business model? It could be the difference between ending up as a niche business or growing to be a large scale global success.
5. Weak product management
As the product gets to the early growth stage it’s important to adopt management processes that will maintain efficiency. Coupled with a long term product strategy, a solid product management process is key not just to driving short and medium term market needs, but also ensuring sufficient development time is invested in maintaining a modular and single code base. You need strong people in place to look after the product, not just focused on the short term revenue, but also keeping the strategy on course.
Your product manager needs to be the guardian and driver of the product plans, both to meet the market requirements, as well as to ensure there is investment in a sustainable product. Taking longer to develop a release in order to make sure it is maintainable may seem to slow down rather than improve productivity but - like many of the points in this article - it’s about improving the picture in the longer term. Typically at least 10% of development time should be invested in maintaining a modular architecture with a single code base that will facilitate future enhancements.
6. Teams split across multiple locations
This is another one that’s very important in an Agile environment. People like to believe virtual teams are just as productive, but it’s just not the case, despite the improvement in communications technology. For instance, when Agile is the chosen methodology, then daily face-to-face meetings called Scrums are core to its efficiency - and there’s a reason those daily meetings are part of the process. The immediacy and focus they bring is a significant boost to productivity. Without having the developers and test engineers for a product at a single location, much of the efficiency of Agile is lost. Where there are multiple products, then these can be developed at separate locations, but all the team members on a single product should ideally be in the same office.
7. Efficiency killers
Software productivity fundamentally depends on the efficiency of the people. A work environment that minimises interruptions is incredibly important. Don’t underestimate the negative effects of a ‘buzzing’ open plan office environment. Open plan offices are NOT a good thing for software development!
Based on its own experience, Microsoft typically has a maximum of two developers per office. Whether or not two is indeed the magic number, as a developer myself for many years I can speak for the fact that developers need to be able to have a quiet work environment, get their heads down and focus on writing whichever piece of software they’re working on. Many studies have shown that maximum efficiency is achieved when developers are “in flow” for at least 2 hours at a time. Each time the developer is interrupted by a question, phone call or meeting it takes 15 minutes for them to get back into flow.
Smaller companies will suffer from these problems less. If there are ten or fewer people in the office then there’s fewer interruptions. But when a company hits a rapid growth stage it’s tempting to invest heavily in new people. This can sometimes be counterproductive. The most productive teams are small with a core of experienced people who have worked on the product for at least three years.
Larger teams, with constant interruptions, can dramatically reduce efficiency.