exploring software and hardware security

How to choose the right security protocol?

Posted by davitb on November 2nd, 2009

We all know that sending data over network and being sure that its confidentiality is not damaged is one of the crucial problems in security engineering. In fact, except some special cases the entire information security area is based on the problem of being able to send information from point A to point B and making sure that’s its security properties are maintained properly.

Creating secure protocols is not an easy task and this article is not intended to help you to create new protocols from the scratch. There already exist many well designed protocols which will give you all the necessary features to meet your requirements. There are many books and articles about how different secure protocols work and how they are designed. This article is not trying to describe details of these protocols rather it tries to recommend the “ideal” protocol for you, which of course doesn’t exist. The question that this article will try to answer is how you should choose the right secure protocol for your particular application.

Choosing the right protocol is also not a trivial task however I believe there are patterns which will help you to solve this task for your particular application. I’m able to recognize these patterns and I’m sure you will also be able to do it once you get the proper knowledge and experience.

We will go over the following topics:

Data confidentiality and integrity in protocols
Two types of security protocols
Attacks you should worry about while thinking about protocols
Protocol Choosing Patterns
A demonstrative example
Recommended references

So let’s start.

Data confidentiality and integrity in protocols

Before going deep into different security protocols and their problems let’s understand why they exist and what they are protecting from.

In my last article (How to design secure systems? Key concept in Information Security) I was talking about security properties of information. I expressed an idea in that article that the whole problem that information security area is trying to solve is “how to make sure that security properties of an information block are properly maintained during its lifetime and that there is no way to damage them?”. Security protocols are tools which allow us designing systems where the information can be transferred from point A to point B over insecure/un-trusted networks without loosing its security properties. I have highlighted six properties in that article – confidentiality, integrity, authenticity, accessibility, availability, non-repudiation. Although in particular applications other properties also can be damaged while transferring data over un-trusted networks – in most cases only confidentiality, integrity and authenticity are of the major concern of security architects. In this article we will mostly focus on these three security properties.

Let’s see what mean confidentiality, integrity and authenticity of data when it comes to transferring information. We transfer sensitive (confidential, private, important) data everyday in our life – when browsing over web, when chating over Skype or other messengers, when talking over phone, transferring files to other locations in network or internet, sending and receiving emails, watching HD TV at home, when locating our location with GPS, etc.

When we transfer this information we want to be sure that nobody else can get it except the addressee. We also want to be sure that the addressee will receive the information exactly in the form we sent it to her. And finally, when the addressee receives our information she wants to be able to make sure that information really came from us and nobody else who is claiming to be us. So our first requirement is about confidentiality, the second is about integrity and the third – authenticity.

Two types of security protocols

Although there may be many criteria which can be used to define different categories for security protocols – I have decided to divide them into two types:

Authentication based
Zero knowledge

Authentication is a key part in a security protocol. Remember, if you want to maintain confidentiality of data – you need to have an authentication part in your protocol. Otherwise how you would know who you are sending your data to?

However it’s not always possible to have authentication primitives in the systems. This is especially true for embedded systems with restricted capabilities. Sometimes you have to restrict yourself with protocols which don’t require authentication and thus are prone to man in the middle attacks.

Let’s see what kinds of attacks are possible on security protocols.

Attacks you should worry about while thinking about protocols

There are three important attacks you should worry about while designing a security protocol for your system:

Eavesdropping
- The attacker has access to the communication channel over which your data is going to be transferred
- The attacker can read everything on this channel

Replay
- This attack is applicable only to authentication based security protocols
- When two legitimate users create a secure session – attacker records the authentication part of the protocol.
- After this the attacker replays recorded messages to one of the users and tries to impersonate as the previous legitimate user.
- If the protocol doesn’t have means to protect against replay attack – the attack will be successful.

Man in the Middle
- Any protocol which doesn’t have authentication part is prone to this attack.
- The main idea is that the attacker is acting as a malware which stays between two legitimate users – A and B.
- When user A sends a message to user B, the attacker is able to receive this message, change it and send the modified message to user B and vice versa.

If you designed/chose a protocol which is protected against these three attacks – you can be sure that your protocol is good enough. There can be cases when your system will have specific requirements and those three attacks won’t cover all possibilities. I’m afraid these cases must be addressed uniquely. However I can assure you that in most cases protecting against eavesdropping, replay and MitM attacks is enough.

Protocol Choosing Patterns

There are lots of security protocols available in different cryptography books and publications – Challenge-Response, OTP, EKE family (EKE, SPEKE, J-PAKE, Augmented-EKE, etc), Kerberos, SSL, Diffie-Hellman, and more. How should you choose the right protocol for you?

Choosing the right protocol highly depends on your requirements. I will try to describe an algorithm here aimed to help you in making the right decision but you should understand that it’s not always going to solve the exact problem you have. There will always be cases specific to your requirements and for these cases the described algorithm will serve you just as a direction and not as a solution.

Suppose you have two components in your system – C1 and C2. These components are connected via an un-trusted network. Your general requirement is to be able to send data from C1 to C2 (and vice versa) and yet maintain the security properties of the sending data.

If (C1 and C2 don’t share a secret)
1. If (you are not using PKI) then
  1. Implement Diffie-Hellman protocol between them
  2. You will be protected against Replay and Eavesdropping attacks
  3. However, note that you are not protected against Man in the Middle attacks.
2. Else
3. If (you want one way authentication – C1 authenticates C2 and C2 doesn’t authenticate C1) then
  1. Implement one way SSL between them. C1 authenticates C2 with certificates (this is how web sites are usually authenticated by the browsers).
  2. You will be protected against Replay, Eavesdropping and Man in the Middle attacks
  3. Note that you need to address the problem of CLR (certificate revocation list) on C1. For embedded devices it’s not always possible.
4. Else
5. If (you are able to use PKI) then
  1. Implement SSL between them
  2. You will be protected against Replay, Eavesdropping and Man in the Middle attacks
  3. Note that you need to address the problem of CLR (certificate revocation list). For embedded devices it’s not always possible

If (C1 and C2 share a secret) then
1. If (there is no trusted Root server) then
  1. If (you want one way authentication – C1 authenticates C2 and C2 doesn’t authenticate C1) then
    1. Use Challenge-Response or OTP based protocols
    2. Authentication will be protected against Replay and Eavesdropping attacks
    3. However, note that authentication process is not protected against Man in the Middle attacks.
  2. Else
  3. If (You want mutual authentication between C1 and C2) then
    1. Use SPEKE (or other EKE based strong algorithm)
    2. You will be protected against Replay, Eavesdropping and Man in the Middle attacks
  4. Else
  5. If (You want mutual authentication between C1 and C2 but don’t want to have exponentiation block in your protocol) then
    1. v. Use SSL which is not based on PKI. Instead of encrypting the pre-master secret with a private key, encrypt it with the shared symmetric key and transfer to C2.
    2. This is quite effective.
    3. You will be protected against Replay, Eavesdropping and Man in the Middle attacks
2. Else
3. If (there is a trusted Root server) then
  1. i. Use Kerberos
  2. ii. You will be protected against Replay, Eavesdropping and Man in the Middle attacks

A demonstrative example

Let’s assume we have a USB based biometric sensor. After it is attached to a USB port of a PC, user’s biometrics can be scanned and transferred to the PC where it will be processed.

Now, user’s biometric information is used for authentication purpose and it usually opens privacy problems if not protected well… That’s why it is required to protect the confidentiality, integrity and authenticity of biometric data when it is being sent from the sensor to PC.

Biometric sensor usually doesn’t have enough computation power and internal persistent memory and that’s why we are not always free to choose the “ideal” protocol for our system. We will go over several possible cases and discuss how to choose the right protocol.

No shared secret

Biometric sensors are now very popular and there are companies which produce such sensors by dozens million of units per year. It’s not always easy to share a secret between these sensors and PC. One of the problems is “where exactly the shared secret will be stored on PC?”. It’s not very secure to store it on hard disc. So this is a real problem. Another problem is how the secret will be provisioned in the sensor? This may impact the manufacturing process and make it more expensive. That’s why it’s quite practical to assume that there is no secret shared between the sensor and PC.

So what should we do here to protect the confidentiality of biometric data?

Let’s see what happens if we decide to send biometric data without protecting it:

Eavesdropping

Hopa, the malware can just read the data by installing a USB Kernel driver and eavesdropping the traffic. That’s not good.

Ok, let’s refer to the algorithm described above and see what it suggests. We are not sharing a secret between Biometric Sensor and PC and we are not using PKI – so we fell in 1.a. After implementing Diffie-Hellman only the man in the middle attack will be possible on our system. Note that because we don’t have authentication – we cannot have a better protocol here. So Diffie-Hellman is the best choice.

Shared secret

Let’s assume we designed a system where each sensor possesses a symmetric key and there is a way to securely install this key on PC when we first time attach the sensor to the PC.

The first idea that will come to mind is to encrypt biometric data with the shared secret when the user provides it and send it to PC.

Encrypting biometric data

Data confidentiality is protected here – the eavesdropper cannot decrypt data being send. But, hey, what if the malware records the traffic when a legitimate user provides biometrics and replays this traffic next time? The malware will impersonate a legitimate user in this case and gain access to the system without even having the shared secret. That’s bad and that is called a replay attack.

Replay attack

Ok, let’s go further. Usually replay attacks can be protected by using a random challenge in the protocol. If PC sends a random challenge each time before receiving the biometric data and verifies that random challenge exists in the received package – we might be protected against replays.

Random Challenge

This is already not bad. First we protect the confidentiality of biometric data by encrypting it and second, we protect authenticity and integrity by verifying the hashed MAC and making sure that it contains the random challenge. In this case we will also be protected against man in the middle attacks as we are doing one way authentication.

Sometimes it’s also required to authenticate the PC in order for the attacker to not be possible to impersonate PC. In this case above mentioned algorithm will direct us to the cases 2.iii or 2.v.