Bill Curtis (pictured), who led the development of the Capability Maturity Model, believes the pace of change in the IT industry is hampering software quality.
He says the cost of software defects is a boardroom issue, but software development teams are fighting a constant battle, as by the time they have mastered one technology, something new comes along.
“Software has a turnover of technology that is faster than we see in other lines of business. We are constantly on a learning curve for new technology," he says.
“Every five years we have a major change in technology, so every time we get good at building quality code, we change the game. There are new technologies and new languages. You make more mistakes because you are on are on a learning curve,” says Curtis, senior vice-president and chief scientist at software analysis and measurement company Cast.
“We need a discipline for adopting new technology,” he adds, suggesting this discipline needs to cover continuous training and shortening the learning curve to enable software engineers to master new technologies quicker.
He suggests that businesses create an integrated group which can build an IT capability to help the business compete effectively. Rather than create a centre of excellence around a product or specific piece of technology, such as big data, he encourages CIOs to consider an approach that can adapt to any technology.
Curtis recommends that CIOs create centres of excellence for new technology, where people can research how to integrate emerging technologies into the business and prepare staff to use the technology.
Consider how cloud computing has affected software development. "The risks in cloud are huge," he says. "I am worried people are not paying attention to this. You are using software that you are not in control of. Cloud computing puts you at a greater distance to things that present potential risk.”
Read more about agile methodology
Risks of agile software development
From an IT perspective, Curtis says businesses want more applications, they want them developed more quickly, and they want them to work perfectly.
Agile software development is seen as a way to enable IT to respond quicker to these requests from the business. But Curtis warns that code quality can be compromised if agile is not followed thoroughly: “I can get the stuff out fast, but it may not be very good, so you end up with the concept of technology debt.”
This creates risk for the business, which may get stuck with a software architecture that does not work.
Curtis says software teams may say they use a methodology such as Scrum, but take out certain aspects such as not doing daily builds, which reduces the effectiveness of the methodology. “People are not effectively using the discipline,” he says.
“Companies such as Google, Microsoft and Amazon have software development needs that are staggering. Google runs 50 million test cases a day, and all the code lives in one large code tree. It is automating everything," says Curtis.
In fact, Agile has driven automation. "We need to automate the heck out of the software build process and automate testing, because there is not time for human-level work," he says.
The industrialisation of software development is essential, according to Curtis, otherwise people will have to prioritise maintenance issues, which could potentially lead to a costly workload for the IT department or a security breach.
"The people who work in companies like Google, Amazon and Microsoft have a high software turnaround requirement. They are creating tools, developing techniques and learning how to maintain the volume of code they build. A decade from now, best practices and tools will filter down to everyone else," he says.
Curtis says this was how the Capability Maturity Model (CMM) evolved.
"CMM started where we had the greatest technology challenges in the US Department of Defense. Then it was used in the telcos, and then moved across industries," he says.
IT departments are using programming frameworks to cut software development time, due to the demands of the business for new applications.
"You have to deal with frameworks, because the business demands larger systems," says Curtis.
By providing lots of functionality that does not need to be developed in-house, frameworks offer software developers a shortcut.
"The frameworks let me keep up with the business, but they don't necessarily improve the quality of the code," he says.
Software developers still have to learn the framework and how it interacts with the system, so there is a learning curve.
The frameworks are not without challenges, he warns. Relying on a third party poses a risk in terms of software quality. "Software vendors don't want you to look at their code," he says.
But given the popularity of programming frameworks, Curtis expects that over time there will be more requirements for software development teams to understand supplier code.
He says coding is now harder than ever due to the complexity of the systems: "In the old days, if you understood Cobol, you understood the whole system. These days, you have Java, SQL and ERP [enterprise resource planning], which all come together to create an IT system. No one can figure out the whole system."
What is really needed, he says, is to keep documentation current.
Read more on open source
Learning from open source
With open source, developers can see the code and documentation is extensive. "Some argue the code is better than commercial software because open source developers do a great job in controlling quality code," says Curtis.
But the downside, he says, is that open source cannot keep up with commercial products in some areas: "Open source works best when it provides a base, such as Eclipse or Apache."
How can businesses apply open source best practices in commercial software projects? Curtis believes that websites such as Top Coder may point to a solution, where it is possible to see a history of a person's work, which helps when selecting someone as a software contractor.
He says some companies are adapting this by using the best aspects of social media, to help project managers and team leaders identify expertise. This knowledge network spans continents and allows the company to improve the quality of software development by identifying the person with the right skills.
"In one company I know, the amount of code you develop that is reused elsewhere is a big part of your status. This drives increased professionalism and it improves the software development community in the company. People become far-flung technology resources, like diamonds in a rock," he says.
Curtis cites one instance where the expertise of a woman from the company's China-based operation was identified and people were reusing her code all over the world.