Testing the Security of Software
Posted by davitb on September 25th, 2009
Citation from Bruce Schneier:
Think about the most recent security vulnerability you’ve read about. Maybe it’s a killer packet, which allows an attacker to crash some server by sending it a particular packet.
Maybe it’s one of the gazillions of buffer overflows, which allow an attacker to take control of a computer by sending it a particular malformed message. Maybe it’s an encryption vulnerability, which allows an attacker to read an encrypted message, or fool an authentication system. These are all software issues.
Testing the software from security standpoint is a key requirement especially for software products with security focus. Even if the product has been architected with the best security protocols and security designing best practices in mind, it doesn’t make much sense as the attacker will prefer focusing on the application layer and eventually will find a vulnerability there and exploit it…. Remember, on application layer…
When I was very new in security engineering I remember me asking people whether there are really real attackers who are trying to break the product’s security by attacking its cryptographic features, algorithms, protocols, etc (the architectural layer). Analyzing the architecture, trying to find problems in cryptography related parts of it and exploiting them? Say, you are using Diffie-Helman protocol to exchange a session key however the implementation you are using has to have a small modulus because of the computational restrictions. The attacker analyses the program, finds this problem out and conducts a brute-force attack on the algorithm trying to obtain the session key. Wow… do you believe in such attack? Why the attacker would do that if there are much simpler ways to attack the software? It’s much simpler to just find a vulnerability in application layer and exploit it.
Of course the architectural layer of product’s security is very important and of course my conclusion in the previous statement was too naive… however what I’m trying to say here is that I have an impression that all the cryptographic primitives that we are including in our solutions are more interesting to marketing and sales people rather than the real attackers (at least for products which are not used in government or military related projects)… and this needs to make people think that something is definitely wrong in the security engineering process…
The goal of this article is to discuss what means testing the security of product from application standpoint; which points need to be taken into account while developing test cases and what kind of attacks exist on software application level.
Recently I was planning the strategy of security testing of the product I’m working on. After some research and thinking I have divided the possible application layer attacks in to the following groups:
- Data Structure Attacks
- Input Data Modification Attacks
- Data Leakage Attacks
- Attacks based on Abuse of Functionality
- Attacks on the Implementation of Architecture Features
Let’s go over them quickly and discuss how we should test the system against such attacks.
This article doesn’t reflect all the possible attacks and weak points that are possible at the application layer however I think it’s a good start and a good experience sharing. I will probably cover topics like proper using of ACL, potentially exploitable code patterns, penetration testing, etc in a separate article.
Data Structure Attacks
I’m sure you heard about buffer overflow, string format, and integer overflow attacks. These are well known attacks which use the vulnerabilities that the bad coding may produce and inject code in the running program. I’m not going to explain how these attacks are conducted or what the reason behind them is in this article. You can find all questions to such answers in the following links:
http://en.wikipedia.org/wiki/Format_string_attack
http://en.wikipedia.org/wiki/Integer_overflow
http://en.wikipedia.org/wiki/Buffer_overflow
Instead we will focus more on how to test the source code against such vulnerabilities and how to find them in the code.
Code review, code review and code review. Code review is an excellent way to find architectural mistakes, logical and coding bugs in the program. It’s also very useful while looking for an overflow bug in the source code. The teams where code review process is well established as a rule are having lesser number of bugs and much more secure code. So code review is always good to conduct.
However there is one bad thing with code review. It’s manual and thus very slow…
It’s not always necessary to conduct the code review process manually. There are actually tools which can automate this process and save lot of time for developers. These tools are called static code analysis tools. They run over static source code and find potential bugs. The type of problems that they are able to find is big and one of these types is the problem of overflow. I have utilized two static code analysis tools during my experience – preFast (from Microsoft) and coverity (from Coverity). Both they are very useful and do great job. I will probably dedicate another article to static code analysis tools and will cover lot of details there. But for now – remember, integrating and using static code analysis tools is a MUST step in building a mature software development process and building secure code. They may significantly reduce the number of potential bugs in software if properly configured and continuously used.
There is one more thing that I find useful for prevention of overflow vulnerabilities in the code. Different compilers provide security checks, protections during compilation or generate a code which automatically prevents overflow attacks.
Let me list several recommendations on regarding this:
- Turn on the highest possible warning level of compiler while compiling the code (/W4 for Visual C++ compiler, -Wall for GCC)
- Use the /GS flag of Visual C++ compiler or StackGuard for GCC. With this flag the compiler injects code into compiling application to help detect buffer overflows at run time.
- /SAFESEH – after compiling with this flag the complier will add extra exception-handler information which will be verified by the operating system to make sure that the real exception handler hasn’t been overwritten. (This is not related to overflow problem however it’s still a useful thing to know ;))
- Don’t use banned functions.
The latest compilers usually deprecate a set of old functions (mostly string based functions) and the programmers should never use those functions. They all have potential overflow vulnerability. Note that such functions can be easily found by static code analysis tools.
The following article is a good reference describing these recommendations in more details.
http://en.wikipedia.org/wiki/Buffer_overflow_protection
Input Data Modification Attacks
It’s important to understand that usually the only way the attacker can influence your program is through an external interface (we assume that he doesn’t have access to program’s internal memory). The main external interface of an application is the modules which process input data. During its execution any program receives input data from a file, registry, network, GUI, system, etc. If the program doesn’t verify this information carefully before using it – the attacker may be able to find and exploit a vulnerability because of that.
The exploit, that such vulnerability will eventually bring to, is based on overflow attack that is discussed in the previous section. However I have decided to separate these two vulnerability schemes because of the way we will fight against them. Finding a potential overflow and verifying that all input data is verified against its format are different operations and need to be handled differently.
The best way to avoid such problems in your program is to go over all input data that the program can receive during its execution lifecycle and make sure that each and every of these data has a predefined format and is checked against this format before being further used.
It’s also very useful to use fuzzers to test the program against bad input. The following Wikipedia article talks about fuzz testing and its applicableness:
http://en.wikipedia.org/wiki/Fuzz_testing
Data Leakage Attacks
Often attackers, while playing with the victim program, are analyzing the output that the program produces and try to find a sensitive information leakage in this output. I call this type of attack Data Leakage Attack.
During its execution a program may produce different types of outputs:
- Trace
- Temporary (or persistent!) data stored in registry, hard disc
- Data exchanging in a protocol
- etc
All these outputs have a potential to contain sensitive information (confidential, important, etc) and if the program leaks such information – the attack will be very happy to eavesdrop them.
Unfortunately I don’t know any effective way to prevent programmers of leaking sensitive information. Of course the programmer doesn’t leak this data intentionally (I hope they don’t!). It can be a debug message, which was useful during debugging or just an unintentional mistake. No matter how this data leaked there must be a way to detect it…
The only way I’m aware of is to develop QA process (test cases) where QA engineers will go over all test cases and make sure that none of the outputs contains a leaked data. This is of course a poor approach and I would be glad to hear a better technique from you…
Attacks based on Abuse of Functionality
Many security oriented products have user interface – graphical or command line. This interface provides an opportunity to users to perform different operations and an opportunity to attackers to find a vulnerability in the program. In general the user interface can be considered another form of “external input” to the program and the attacks that are applicable here will be also covered by “Input Data Modification Attacks”. However again the approach to prevent such attacks is different here and I would like to briefly focus on that.
Suppose the program has 3 different buttons (1, 2, 3) on its GUI and the right combination of the usage of these buttons is – 2, 1, 3. The attacker will try to use other combinations (123, 321, etc) and see what happens with the program. If it’s crashes or misbehaves – bingo!
Automatic GUI testing tools must be used or just well covering test cases must be developed to check all the possible combinations of GUI control usage. In practice this is not always possible and the consequence is that such attacks are pretty much popular now.
Attacks on the Implementation of Architecture Features
The last type of attacks I’m going to discuss here is implementation level attacks on security architecture. Suppose on architectural layer you have decided to use AES128 algorithm to encrypt data. However your implementation (source code) has a bug and upon receiving some type of input data – doesn’t encrypt the data or does partially…. this is bad. Another example… Suppose in your architecture it is said that some confidential data needs to be stored on the hard disc and be deleted after 5 seconds. However your implementation has a mistake and the data is never deleted…
These examples show a simple thing – the implementation doesn’t not always correspond to the architecture (I would say in most cases it doesn’t) and careful test cases need to be created to cover all the logical statements that are present in the architecture.
In general testing that the actual implementation corresponds to its original security design is a tough process. It requires test case developers to carefully understand the security architecture and test each step that is defined there.
Let’s see a “simple” scenario… suppose the architecture defines that step A can be performed only when the system is in specific state B. Now, there should be a test case which not only tests that this requirement is met (step A can be performed in state B) but it also tests that step A cannot be performed in other state than state B. This of course complicates the life of test case developers…
Hope after reading this article you will come up with better understanding of why it’s important to write secure code and why it’s important to test the security software from application standpoint.
Useful Information
http://www.exploitingsoftware.com/
http://www.webappsec.org/projects/threat/
http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis

