Why the quality of software is so bad?
Posted by davitb on September 23rd, 2009
Have you ever thought about why it’s easy for people to create an elegant piano, a huge building with astonishing architecture or an excellent working elevator but in the same time it’s so difficult to create a well working text editor for PC? Why only very few people, when they are opening the garage, have a think that the garage door will fall down and destroy their car… but in the same time I’m sure that all the people, when they type in a text editor on PC, continuously think that the program may crash anytime… really, anytime!
During last several decades people have created a completely new discipline, a new world, something that even the most genius fiction writers didn’t come up with – computers and software. Today it’s in our life everyday, everywhere, almost each second… I think that’s a wonderful progress…
Software and hardware are used everywhere, at home, when we drive a car, when we flight on airplane, when we talk by phone, when we watch TV, when we send a mail or wait in a queue in a restaurant…
However in parallel to this big progress there is one thing that I think is not improving very well over time. That is the quality of software. Although people realize and give a very high priority to the technologies, processes, methodologies, which will raise the opportunity to improve the quality of developing software, however there is no big progress in this area.
What you think is the reason behind this? How can we solve this problem and finally how is software security connected to its quality?
I will try to briefly discuss these questions in this article.
What you think is the reason behind this?
Today, medium size software is a very big, complex system which consists of a huge number of small elements and which is so huge that cannot be completely fit in the imagination of dozen of people. Hundred thousand lines of source code written in a language which, though is semantically correct, but sometimes is able to confuse any expert programmer… Infinite number of “if” and “branch” statements – a huge graph with million of edges and nodes… And you are asking why it is hard to test this monster?
If we also take into account the fact that this monster graph must be integrated to another, thousand times bigger, super monster graph (OS, internet, external devices with their software, etc) then it becomes clear that the testing of this huge system presents an extraordinary difficult engineering problem which is definitely not easier than the creation of this software.
There is an opinion that the profession of “test engineer” requires less knowledge and experience than the profession of software engineer. I find this opinion wrong. The testing of software is one of the most challenging problems that need to be addressed in the near future and every, even simple, progress in this area will be appreciated by the industry very much.
How can we address this problem?
Today’s giant companies are spending big efforts in order to improve the quality of their software. Microsoft, HP, Dell, IBM, etc spend huge money on increasing the quality of software development processes. The companies, which are able to produce qualitative, simple software, are having big success in the industry (google)…
It seems that the main focus, today, is in creating effective software development processes and addressing the problem of testing in the process itself. The processes are developed in a way that development and testing teams are separated from each other as much as possible in order to not affect on each other. There are even methodologies where test cases are developed before development of the software itself (extreme programming).
However all this doesn’t change the situation. Software fails and crashes everywhere.
Let us discuss several activities and methods which might improve the overall testing process if properly applied.
More time , more money, more people
Time to market has the highest priority for many products and it’s obvious that project/product managers have to schedule projects in the way to complete the product as soon as possible. This brings to situation where in most cases the lion part of product development time is dedicated to software development and only a small portion – to testing. The consequence is what we have now – a market filled with lot of products but with poor quality.
The industry shows that companies who spend more resources (money, time, people, etc) to test their products and create qualitative software – have big success in the market. The industry and people need software with high quality.
The quality of software will never become as important as the “time to market” requirement; however software companies need to realize that over time people will just refuse using bad written software and will prefer to pay more for a qualitative product.
Optimizing the testing process
It’s obvious that for today’s software the number of surrounding environments is so large (different platforms, different operating systems, etc) and the environments are so complex that it requires lot of time from Verification and Validation teams to complete a single cycle of test cases validation (and of course all test cases are not covered). This, in its turn, has a significant affect on project’s schedule which is not allowed from business perspective. If it were possible to speed up the test execution process and particularly speed up the prelease->test->fix->prelease->test cycle – it would have a great impact on overall project schedule, budget and quality.
Imagine if there were tools which allow the test team to perform specific steps once, record them and playback whenever they want. The test team would record test cases on different platforms once and would run them overnight, automatically.
I’m sure such tools already exist in the industry but they are not yet mature enough and need to cover more environments and support more business cases.
A globally admitted testing language
Many companies have internally developed special purpose languages – designed for testing the internal products. This makes sense. However as a rule such languages are very poor designed and are very specific to the target product. They are not object oriented and the best practices that are applicable to development languages are not applicable for those languages (very difficult to reuse code, difficult to maintain, not generic enough, etc).
It’s time to think about developing libraries for testing and facilitate the life of test engineers. Of course this is not a trivial task – however I believe the industry needs it and there is a big niche in the market for such tools and libraries.
Automation Tools
If it were possible to detect bugs earlier in the process of software development using automation tools – it would significantly speed up the overall testing process.
There is a notable tendency today to use static and dynamic analysis tools during software development process, which allow finding coding and runtime errors that might become bugs in the future – if not fixed. In ideal case such tools would run over source code or emulate the software programs and find all the issues which can bring to crashes and vulnerabilities. Of course they cannot find logical errors but still their impact will be very big – they allow decreasing the number of potential issues significantly.
Application Verifier, Valgrind, preFast, Coverity, Fortify… These and similar tools will be a very important part of software development process in the near future and I think it’s time for everyone to integrate and use them.
How is security connected to quality?
In today’s world an essential thing becomes the testing of software from security standpoint. How one can achieve this goal and what are the best practices – I will cover in another article. However I would like to briefly discuss this topic here as I find it to be very important and directly connected to software testing.
Many software engineers and technical leads have no idea what means software security and either don’t pay attention to it at all or give very low priority to security aspects of coding. Of course this is a bad tradition. This tradition has been started a long ago when there were no computer viruses and no one cared about computer security.
Respectful consulting companies report that around 70% of all attacks are being conducted on application level rather than architectural layer. This means that even if a security application has very well designed architecture from security point of view – the attackers will focus on bugs and vulnerabilities that are result of bad coding (buffer overflow, SQL injection, XSS, etc) and will successfully exploit them. Even if your application doesn’t have any security requirement – it may (will!) become a target of attackers and once exploit – will be used to attack the environment where it is running (internet browser, operating system, web server, embedded device, etc).
Security of software (secure coding) explicitly affects the quality of software. And while no appropriate attention is paid to the testing of software from security perspective this will remain one of the weakest points of software QA process.
Afterword
And as an afterword I would like to mention that software engineers, and in the first place, software companies need to change their opinion and approach about the profession of QA engineer. This profession will play one of the most important roles in the software development process in the near future and it’s time to treat it more seriously.
The time when a small group of software engineers created breathtaking programs in a month and immediately made them available for millions of people without testing the product well – is gone. And we all need to admit it. Otherwise we will need to continue pressing Ctrl+S each 5 seconds while working with programs like MS Word.
