Cryptography on Windows Part 1 - Introduction

Security is currently the No. 1 priority for the software industry; and if that's not the case, it should be, given the current state of affairs with daily reports of major computer break-ins, credit card fraud, identity theft etc. It is important for applications, and application writers, to be aware of these issues and make use of all available technologies to protect against attacks.

For Tcl applications on Windows platforms, TWAPI releases prior to 4.2 already addressed two of the pillars of security on Windows — authentication and authorization — and I have covered these topics in Windows Security. However, outside of SSL/TLS, earlier versions of TWAPI only had limited support for the third pillar — cryptographic functions related to the integrity and privacy of data. TWAPI 4.2 addresses this area. In practice, the use of cryptographic API's can be very confusing and I hope this series of blog posts, this being the first, will serve as an informal guide and tutorial for this new functionality.

A cautionary note

First, a cautionary note about these posts. Cryptography is a complicated topic on which a thousand page tome could be written and in fact several such exist. Our goals are more modest, aimed at readers somewhat familiar with these technologies and focused on describing how to make use of the related services provided in Windows from Tcl. Along the way, we will try to provide enough of an introduction for readers who are completely new to this space to at least follow along. Even if you are not specifically interested in Tcl programming, its interactive nature and high level constructs may help you to explore this complicated topic and gain a deeper understanding of how all the pieces fit together.

However, what these posts will not do is give you the skills to design and develop a cryptographically secure an application any more than reading the Sunday newspaper cooking column will make you a chef. They will not answer questions like

• Should I use a random initialization vector or a sequential nonce?

• What algorithms and key lengths are adequate for my application?

• Is it better to encrypt and sign or sign and encrypt?

• How do I choose between CBC, CFB and CTR encryption modes?

For the deeper understanding required, I suggest the following references:

• Everyday Cryptography: Fundamental Principles and Applications, 2nd Edition, by Keith Martin, Oxford University Press, is the most accessible introduction to cryptography I have read. It covers the concepts and ideas at the right level of detail — sufficiently in-depth but not bogged down with mathematical details that are not required for using cryptography.

• Secure Programming Cookbook for C and C++, by Viega and Messier, O'Reilly, 2003. I find this book a good companion to the above. While Everyday Cryptography is all about concepts, this book, as its name suggests, focuses on the practical aspects of secure programming (and not just cryptography) with fully functional working code.

• Engineering Security, by Peter Gutmann, a draft of which is available via his home page is broader in scope and higher level than either of the above. The page also links to his other articles which are worth a read for their humour and entertainment value even if you have no interest in security. If you have the time, skip this introductory post and read his 1000-slide godzilla crypto tutorial instead.

So if you are just interested in getting some familiarity with the subject, I hope these posts help you do that. If however, you are actually implementing a real application in production, you need to read the above references or similar sources. This series of posts will then (I hope) let you implement your design in Tcl on Windows.

Concepts

The subject of cryptography involves multiple interrelated concepts which leads to a chicken and egg problem of where to start when describing them in detail to readers new to the field. Therefore we begin this series with a short summary of concepts and terms intended to make it easier to follow subsequent posts.

Messages

Cryptography deals with the issue of protection of data in various scenarios, such as communication between two or more parties, storage on media and so on. Protection includes one or more of maintaining secrecy, preventing tampering and verifying the identity of the entity generating the data. Cryptographic algorithms (for the most part, but not always) operate on "units" of data that we will refer to as messages although no communication may be involved. Similarly, the terms sender and receiver are used interchangeably with the terms producer and consumer of the data.

Confidentiality

The word cryptography itself stems from the Greek words that translate to secret writing and one of its basic functions is exactly that — maintaining confidentiality of data by transforming it into a seemingly garbled form from which no useful information about the content can be extracted. The original data is referred to as plaintext and the process of transforming into the garbled form, ciphertext, is known as encryption. The intent of the transform is that after conversion no information about the plaintext can be gleaned by observing the ciphertext unless the reverse process, decryption, which transforms the ciphertext back into plaintext, is executed.

Cryptographic keys

The processes of encryption and decryption involve the use of data that is at least partially secret. The various cryptographic transforms can be viewed as functions that take as input the plaintext and this secret, called a key, and produce an output, the ciphertext. Actually, this is not quite accurate as the key might not be secret, but we'll come to that in a bit.

Thus encryption can be viewed as the function

ciphertext = fenc(plaintext, keyenc)

Similarly, decryption is the inverse function

plaintext = fdec(ciphertext, keydec)

Depending on the algorithm, f_enc and f_dec may even be the same function. Likewise, the key needed for decryption may or may not be the same as the encryption key.

Protection of secret keys from unauthorized access is crucial for secure cryptographic operations. This includes protecting them in storage, in transit, and even in memory. A cryptographic system needs to provide services for this purpose.

Data integrity, message digests and message authentication codes

While confidentiality deals with keeping data secret, data integrity ensures it has not been modified since the time it was generated. Modifications may occur either inadvertently, for example via transmission errors, or through the actions of a malicious attacker. In either case, data integrity protections indicate when such modifications have taken place.

In its most basic form, integrity checks use a message digest, also called a hash, which is a value calculated from the data using a one-way function. It is not computationally feasible to reconstruct the original data from the hash value and hence the term one-way. Moreover, from a cryptographic perspective, a hash function must have the properties that it is infeasible to find a message that maps to a given message digest or to find two messages that have the same message digest.

In a communication scenario, the originator of a message computes and sends the message digest along with the message. The receiver recomputes the digest from the received message and compares it with the one sent by the originator. If they are equal the message has not been corrupted.

Note that there are no secrets (keys) involved in computing a message digest. It is purely a function of the data message. An obvious consequence of this is that a message digest provides no protection against a malicious party that substitutes both the data and the digest with its own versions. For example, some file protection schemes use cryptographic integrity checks on file content to protect against viruses replacing important system files. If these were based on message digests, an attacker could overwrite the file and replace the corresponding digest with the one computed from the bogus file.

The use of message authentication codes (MAC), aka message integrity codes (MIC), targets this problem. The solution again involves the use of a secret, a key. Instead of computing the hash solely based on the message data, it is computed over some (carefully chosen) combination of the secret key and the message data. To verify the message integrity, the receiver again computes the hash over the combination of the secret key and message data and checks it against the received hash. In our above example, an attacker's attempt to replace the file and its hash without detection would fail because the hash cannot be faked without access to the secret key.

It should be noted that certain encryption algorithms also provide message integrity in addition to confidentiality without the use of a separate MAC.

Symmetric and asymmetric cryptography

Algorithms in cryptography can be classified as symmetric and asymmetric algorithms.

Symmetric algorithms involve the use of a single secret key. Both encryption and decryption make use of this single key shared between the sender of a message and the receiver.

In asymmetric algorithms, the key used for encryption is different from the one used for decryption. These two keys, termed a key pair, are called the public key and the private key for reasons that will shortly become clear. Each party involved in secure communication using asymmetric algorithms holds at least one such key pair. The key pair has the relation that encryption using the public key can be decrypted using the private key. Thus the encryption operation is

ciphertext = f(plaintext, keypublic)

Similarly, the decryption operation is

plaintext = f(ciphertext, keyprivate)

The public key from this key pair is made, well, public. It can be freely distributed to the entire world. Anyone wishing to send secret communication to the holder of the key pair can encrypt the message using this key. The private key on the other hand is secret known only to the owner of the key pair. Any messages received encrypted with the public key can be decrypted with this private key. Note that communication going the other way would use the key pair belonging to the other party.

Asymmetric cryptography is also known (possibly more commonly so) as public key cryptography.

Compared to symmetric keying, asymmetric algorithms have two major advantages:

• An application conversing with multiple parties need not keep track of, and protect, the different shared keys used with each. All communication to the application can be encrypted with its well-known public key. This makes key management easier and more scalable.

• Perhaps more important, because the private keys are not shared, asymmetric algorithms can be used to establish identity using digital signatures, to be described in a future post.

On the other hand asymmetric algorithms are much slower in operation than symmetric algorithms which makes them unsuitable for encrypting bulk data.

Session keys

Because of the above tradeoffs between symmetric and asymmetric algorithms, most cryptographic systems use a combination of the two. A random symmetric key is generated which is then used to encrypt the data. This key is then encrypted with the public key of the receiver and sent along with the data block. The receiver decrypts the symmetric key using his private key and uses that to decrypt the data itself. Since no one else has access to the receiver's private key, the symmetric key and consequently the data both remain private. Because the asymmetric algorithms only operate on relatively small amounts of data (the symmetric key), performance is not an issue.

These ephemeral symmetric keys are called session keys because they are used for the duration of a session such as a SSL transfer, or encrypting a specific set of files, and then discarded.

Signatures

The use of shared secrets in MIC's as discussed previously has one drawback. Because they are shared between the sender and the receiver, they cannot be used as digital signatures to prove that a message was sent by a particular party. Either of the two parties could have generated the MIC; there is no way to tell who.

On the other hand, when an public key algorithm is used the private key is not known to anyone except the owner of the key pair. A MIC generated using the private key could not have been generated by anyone except the party possessing that key. At the same time, the public key, known to all, can be used by anyone who needs to verify the signature. Thus public key algorithms can be used as the basis for signatures in digital messages.

Block versus stream ciphers

Symmetric algorithms can be classified as being stream ciphers or block ciphers. Streaming ciphers, such as the RC4 algorithm, process the plaintext a byte at a time producing a corresponding byte of ciphertext. Block ciphers, such as DES or AES, operate on fixed size blocks of plaintext in turn producing a block of ciphertext. If the plaintext is not a multiple of the block size, it has to be padded appropriately.

Encryption modes

In the case of block ciphers, the cipher mode refers to how the algorithm relates the different blocks in the input data. In the simplest case, each block of data in the message is encrypted independently of the others. This mode is called the ECB (Electronic Code Book) mode and should never be used because its vulnerable to several forms of attacks. More secure modes such as CTR (Counter), CBC (Cipher Block Chaining), or GCM (Galois/Counter Mode) use various techniques to make the encryption more resilient and are to be preferred. You do not need to know the details about how they work; simply select the appropriate options when calling cryptographics API's.

NOTE: Again, most applications should use high level interfaces such as SSL/TLS where the ciphers and modes are determined by the standardised protocols in use so the programmer is not required to make the choice.

Salts, initialization vectors and nonces

It seems to me that the terms salt, initialization vector and nonce are sometimes used interchangeably in various contexts. Below is how I view them conceptually.

A salt is non-secret, randomly generated value that is used to ensure that a given input message and key combination produce different cryptographic output every time. They are commonly used to make attacks based on precomputation more difficult. For example, a dictionary based attack uses a precomputed dictionary that maps hash values corresponding to common passwords to the plaintext passwords. If stored password hashes are computed without a salt, this greatly speeds up brute force attacks as the hash value can be used to directly look up this map. If however the hash values are computed from the concatenation of a randomly generated salt, say 8 bytes or so, and the password, this attack becomes infeasible as the attacker would have to hold a precomputed dictionary thats 2⁶⁴ times larger. Note that the salt need not be a secret and can be stored as plaintext.

An initialization vector (IV) is also a randomly generated, non-secret value but serves a slightly different purpose. Many cryptographic algorithms run through a sequence of stages each of which processes the output of a previous stage. An initialization vector provides the value to be used as the input in the first stage, to "seed" it so to speak. Different IV values will result in differing output ciphertext for a given message and key. Like the salt, an IV need not be a secret but must be randomly generated. In particular, an attacker must not be able to predict the IV that will be used for a message by observing those in previous messages and the same IV must not be reused for a particular key value. Initialization vectors are generally used in algorithms based on block ciphers.

A nonce stands for a "number used just once" and is again a non-secret value that is input into a cryptographic process. A nonce value should not be reused with respect to a particular key but unlike a salt or IV, nonces need not be unpredictable and in fact often are sequentially incremented in cryptographic protocols that protect against message replay attacks. However, purely incrementing values have weakness against some forms of attack and thus nonces are often constructed by concatenating a random portion with a sequentially incrementing one.

The use of these values will become clearer when we illustrate symmetric encryption with code examples in a future post.

Next up

That concludes our 10 minute introduction to terms and concepts in cryptography. This should suffice as background for the rest of this series and in the next post we will be a little more hands-on and introduce the Windows components and API's that provide these services.