In part 1 we started the discussion about keeping secrets secret, which is the theme of the Open Web Application Security Project’s (OWASP) #6 most critical risk for web applications, sensitive data exposure. In that first part we discussed proper user credentials storage. In this part we will continue that discussion with a focus on encryption techniques for data at rest, a big hurdle that must be overcome for anyone storing encrypted data.
Encrypting Data at Rest
Are You Sure, Absolutely, Positively? Positively, Absolutely Sure?
Question #1: do you truly need to store “that” data in an encrypted state? You believe you need to keep it and now you’re looking at storing it encrypted. However, the most secure option is to not store the data to begin with. They can’t steal what you don’t have.
You are responsible for any sensitive data you keep. Data compromises are highly publicized, with all kinds of legal, financial and reputational ramifications. While you might perform best practices for storing sensitive data, motivated attackers seem to find other weak access points. And your security is only as strong as the weakest link.
Therefore,if you simply must keep it, take the time to look for alternative solutions to storage. A lot of application features and data collection points originate from the belief that someday we will need it. Is this data you’ve labeled as “important”, really needed?
Back to Your Regularly Scheduled Program
If you decide on storage, you will probably turn to two-way encryption to protect that information if it falls into the wrong hands or to keep away prying eyes of the internal sort.
Two of the most common types of two-way encryption are Asymmetric and Symmetric Encryption. While both can obfuscate data into an unreadable format (1-way) and return it to the original readable (2-way), there are significant differences.
Asymmetric Encryption
Asymmetric encryption is part of a larger group called Asymmetric cryptography that encompasses a number of distinct applications such as digital signatures and encryption. A digital signature can provide authenticity and integrity and encryption provides the confidentiality.
Asymmetric encryption uses two keys; a non-secret key that encrypts the data and a secret key that decrypts the data. Asymmetric encryption is part of the TLS/SSL handshake which we’ll be looking at in connection with data in transit. For now, our focus is on the second type: symmetric encryption
Symmetric Encryption
In symmetric encryption, the same key encrypts and decrypts data. Given its dual function, the key is secret. Symmetric encryption makes sense when the same entity handles both parts of the process (end-to-end).
In our original scenario, where we have data we want to store in an encrypted state, symmetric encryption sounds like a good fit. Unfortunately, not only is encryption hard to begin with, but there are problems with symmetric encryption. Let’s begin by familiarizing ourselves with a simple example.
Symmetric Encryption in .NET Example:
byte[] iv; // initialization vector
byte[] key; // secret
byte[] encryptedByteArr;
string decryptedData;
using (var aesManaged = new AesManaged ())
{
iv = aesManaged. IV;
key = aesManaged. Key;
var encryptor = aesManaged.CreateEncryptor();
using (var ms = new MemoryStream ())
using (var crytpoStream = new CryptoStream (ms, encryptor, CryptoStreamMode .Write))
{
using (var writer = new StreamWriter (crytpoStream))
{
writer .Write(MessageDigest);
}
encryptedByteArr = ms. ToArray();
}
}
using (var aesManaged = new AesManaged ())
{
aesManaged .IV = iv;
aesManaged .Key = key;
var decryptor = aesManaged.CreateDecryptor(key, iv);
using (var memStream = new MemoryStream (encryptedByteArr))
using (var crytpoStream = new CryptoStream (memStream, decryptor, CryptoStreamMode .Read))
using (var reader = new StreamReader (crytpoStream))
{
decryptedData = reader. ReadToEnd();
}
}
Nothing exciting, nothing over complicated. As you can see we’re using .NET’s AES (Advanced Encryption Standard) library to perform the encryption and decryption process. I am no cryptographer, but if experts such as Stan Drapkin says to always use AES, I’m listening. Now let’s meet the players in our symmetric encryption problem.
Key
The key is a cryptographic strong secret. Along with the initialization vector (which we’ll see next), the same key is required for decrypting the data.
IV (Initialization Vector)
The initialization vector introduces entropy into the encryption process. Simply put,by using a unique initialization vector on every execution of encryption, you avoid any “sameness” or predictable patterns that might permit exact or partial matching of encrypted messages. Unfortunately, as I alluded to earlier, there are known issues with symmetric encryption in .NET.
Here Be Dragons
Do you recall back in 2010 when ASP.NET security was destroyed by the oracle padding? You don’t? No problem. You can read about it here, here and also this video to get an idea of the complexity and profound problem.
The short version is that the problem was rooted in the padding mode used by the cipher mode. Block ciphers operate on fixed length blocks of bytes (i.e. 16 bytes). If a block of data doesn’t fit nicely into a block, it will be filled out with specialized padding. If a submitted cipher text is not properly padded, the receiver processing the encrypted string will emit a related error. For a clear overview of how an attacker can repeatedly make small augmentations to a cipher message and based on the response in the case of a padding error or success, end up deciphering each byte, see Moxie Marlinspike article.
Because of this vulnerability, there are experts who would go as far as applying the following rule: If you’re using symmetric encryption, it has to be authenticated encryption.
Authenticated Encryption
The authenticated encryption method we are going to look at is “encrypt-then-MAC” (ETM). The following process is taken from Stan Drapkin’s Security Driven .NET book (which if you haven’t read, is well worth it).
The process requires two keys, one to encrypt the original plaintext. The other generates a message authentication code (MAC) of the entire message. The following is a quick overview of the encrypt and decrypt process using the ETM authenticated encryption.
To Encrypt:
- AES-encrypt plaintext P with a secret key Ke and a freshly-generated random IV to obtain ciphertext C1.
- Append the ciphertext C1 to the used IV to obtain C2 = IV + C1.
- Calculate MAC = HMAC(Km , C2) where Km is a different secret key independent of Ke.
- Append MAC to C2 and return C3 = C2 + MAC.
To Decrypt:
- If input C length is less than (expected HMAC length + expected AES IV length), abort.
- Read MACexpected = last-HMAC-size bytes of C.
- Calculate MACactual = HMAC(Km , C-without-last-HMAC-size bytes).
- If BAC(MACexpected , MACactual) is false, abort.
- Set IV = (take-IV-size bytes from start of C). Set C2 = (bytes of C between IV and MAC).
- AES-decrypt C2 with IV and Ke to obtain plaintext P. Return P
– Security Driven .NET -Stan Drapkin
To encrypt, using a unique IV we generate the ciphertext (C1) with our first key. Then, we generate an MAC of the concatenated IV and the ciphertext (C2). Finally, we append the MAC to the (C2) for a final value of (C3).
This allows us to first check expected lengths at the beginning of the decryption process before doing any further steps. We can then generate the actual MAC and compare that to the submitted MAC of the (C3) ciphertext. We can again validate the authenticity of the message and abort if this value isn’t correct. We then proceed to decrypt the ciphertext (C1) using the appended IV with (C2).
So what would the above verbose explanation look like in a code example? Glad you asked. Below is a .NET code example of the process for clarification:
private static readonly Func<Aes > AesFactory = () => new AesCryptoServiceProvider ();
private static readonly Func<HMAC > HmacFactory = () => new HMACSHA512 ();
private static readonly Aes Aes = AesFactory();
private static readonly HMAC Hmac = HmacFactory();
private static readonly int AesIvLength = Aes.BlockSize /8;
private static readonly int MacLength = Math. Min(128, Hmac .HashSize)/ 8;
private static readonly int MinCipherTextLength = AesIvLength + MacLength;
private static readonly int AesKeyLength = Aes.KeySize /8;
private static readonly int MacKeyLength = Math. Max(256, Hmac .HashSize - Aes.KeySize)/ 8;
private static readonly byte[] Enckey = GenerateRandomKey(AesKeyLength); //Creates Cryptographically strong Random Key based on size
private static readonly byte[] MacKey = GenerateRandomKey(MacKeyLength); //Creates Cryptographically strong Random Key based on size
private static string _messageToEncrypt = "She turned me into a newt" ;
private static void AeSymmetricEncryptionExample()
{
var messageToEncryptByteArr = Encoding .UTF8. GetBytes(_messageToEncrypt);
var encryptedByteArray = Encrypt(messageToEncryptByteArr);
var decryptedByteArray = Decrypt(encryptedByteArray);
var decryptedText = Encoding. UTF8.GetString(decryptedByteArray);
}
private static byte[] Encrypt( byte[] text)
{
using (var aes = AesFactory())
{
aes .Key = Enckey;
var iv = aes.IV;
using (var ms = new MemoryStream ())
{
ms .Write(iv, 0 , iv.Length);
using (var encryptor = aes .CreateEncryptor())
{
using (var crypto = new CryptoStream (ms, encryptor, CryptoStreamMode .Write))
{
crypto .Write(text, 0 , text.Length);
crypto .FlushFinalBlock();
using (var hmac = HmacFactory())
{
hmac .Key = MacKey;
var mac = hmac.ComputeHash(ms .GetBuffer(), 0 , (int) ms. Length);
ms .Write(mac, 0 , MacLength);
return ms. ToArray();
}
}
}
}
}
}
private static byte[] Decrypt( byte[] cipherText)
{
var cipherLength = cipherText.Length - MinCipherTextLength;
if (cipherLength <= 0) return null ;
var ivCipherLength = AesIvLength + cipherLength;
using (var aes = AesFactory())
{
aes .Key = Enckey;
using (var hmac = HmacFactory())
{
hmac .Key = MacKey;
var actualMac = hmac.ComputeHash(cipherText, 0, ivCipherLength);
// Compares before moving on
if (!XorCompare(actualMac, 0, MacLength, cipherText, ivCipherLength, MacLength)) return null;
var iv = new byte[AesIvLength];
Buffer.BlockCopy(cipherText, 0, iv, 0 , AesIvLength);
aes .IV = iv;
using (var ms = new MemoryStream ())
{
using (var decryptor = aes .CreateDecryptor())
{
using (var crypto = new CryptoStream (ms, decryptor, CryptoStreamMode .Write))
{
crypto .Write(cipherText, AesIvLength, cipherLength);
}
}
return ms. ToArray();
}
}
}
}
private static byte[] GenerateRandomKey(int keyLen)
{
var csprng = new RNGCryptoServiceProvider();
var generatedSalt = new byte[keyLen];
csprng.GetBytes(generatedSalt);
return generatedSalt;
}
// XorCompare courtesy of Stan Drapkin - Security Driven .NET
public static bool XorCompare(byte[] a, int first, int firstCount, byte[] b, int second, int secondCount)
{
var x = firstCount ^ secondCount;
for (var i = 0; i < firstCount; ++i)
{
x |= a[first + i] ^ b[second + i % secondCount];
}
return x == 0;
}
But we aren’t cryptographers, so before rolling your own, I would highly recommend using a library such as the Inferno or Libsodium if at all possible, for performing authenticated encryption. But the above example can clearly demonstrate what an ETM authenticated encryption process entails. But there is one glaring issue in all of these examples – the key. How do we safely store the key?
Key Management
One of the biggest problems with all encryption is key management, regardless of the type of encryption or the purpose, Safeguarding the key is a weak link, especially when we are talking about symmetric encryption where the same key is used for both processes. There are no great answers. However, instead of going on about how difficult the problem is, I am going to propose a possible solution. Granted, this solution is not going to be a perfect fit for everyone’s circumstances, but for those that it does, I think it delivers the protection, the ease of accessibility and the support of a well-tested system.
Azure Key Vault
Azure Key Vault provides a safe and affordable way to store both keys as well as secrets while being backed by the security of a Hardware Security Module (HSM) if needed. Azure Key Vault can do the following and more::
- Provide the decrypted version of the locally stored encrypted key (or master key) for authenticated encryption on request
- Serve encryption keys as secrets in Azure Key Vault
- Perform the necessary encryption/decryption (limited)
To use Azure Key Vault, you need to register your application with Azure Active Directory (AAD) in order to gain authorization to the Key Vault. The dependencies are outside the scope of this article, but if you want to check out getting started with Azure Key Vault, you can check out this article, also this article and finally another step-by-step article from Microsoft.
In Summary (but not the end)
When talking about data at rest, these are the takeaways to remember:
- Most importantly, evaluate whether there is a true need to store the data in an encrypted state. Remember, what you don’t have, can’t be stolen.
- When storing encrypted data using symmetric encryption, use authentic encryption.
- If at all possible, take advantage of a tried and true crypto library such as Inferno or Libsodium.
- Finally, the toughest problem to solve when it comes to encrypting data at rest is key management. If your circumstances allow, consider cloud services like Azure’s Key Vault.
In the next post, we’ll be looking at the last and final part to keeping secrets, secret – securing data in transit.