Cryptography on Windows Part 2 - CSPs, contexts and containers

This is the second in a series of posts on the use of cryptography on Windows. The previous blog post introduced the basic concepts related to cryptography. Here we delve into how those concepts are implemented in Windows at a system or architectural level and of course, how one accesses them from Tcl. This will lay the ground for discussing the actual cryptographic operations in future posts.

First a bit of history. Cryptographic services were initially provided on Windows through an application programming interface called CryptoAPI. This API, also referenced as CAPI, is still supported on current Windows versions. Windows Vista brought along the Cryptography API Next Generation, or CNG, designed to be a long term replacement for CAPI. The functionality implemented in TWAPI 4.2 is based on the original CAPI interfaces as Tcl and TWAPI still support Windows XP. For the most part, this is irrelevant when using TWAPI calls as the underlying details are hidden. However, you need to be aware that algorithms and functionality only available in CNG is not accessible through TWAPI.

The twapi_crypto package

An interactive exploration of CAPI will aid in understanding so let us first load the twapi_crypto module from the twapi package. To lessen the typing burden, also add twapi to the namespace path.

% package require twapi_crypto
4.2.13
% namespace path twapi

Cryptographic Service Providers (CSPs)

Windows defines common interfaces for a wide variety of cryptographic operations. The implementations behind these interfaces are provided by software components called Cryptographic Service Providers (CSPs). Some of these CSPs are implemented by Microsoft and shipped as part of Windows itself. Others are third party products that offer features and enhancements beyond the ones that come with the system.

There are several motivations for this multiple-CSP model:

• A CSP may implement algorithms that are not in the base system

• A CSP may offer performance benefits, for example through specialized hardware support

• A CSP may provide additional security such as biometrics or smartcard support for certificate storage

The algorithms and operations supported by the different CSPs may overlap. It is up to the application to choose which CSP is best suited for its purpose and can of course make use of more than one. The services provided by a CSP include not only the base cryptographic operations but also protection of keys and credentials. From an application's perspective, a CSP is completely opaque. It can ask the CSP to invoke operations on its behalf but does not have direct access to the CSP's internal structures such as encryption keys. This is important to prevent any inadvertent leakage or disclosure of private cryptographic data.

CSP types

When an application needs to make use of a CSP and knows which one it wants, it can specify it by name. More often however, the application knows the cryptographic operations and algorithms it wants to use and does not necessarily know or care which CSP supports those on that system.

Windows therefore defines CSP types that map to a specific set of cryptographics capabilities such as the

• the cryptographic algorithms implemented by the CSP

• key attributes such as supported key lengths

• the supported operations such as signing, encryption etc.

A CSP can associate itself with one or more CSP types provided it complies with all the required functionality for that type. When an application needs the services of a CSP, it can specify the CSP type instead of, or in addition to, the name. Windows will then find one that matches the specified type. The commonly used CSP types, along with the supported functions and algorithms for each type, are shown in the table below. See the TWAPI documentation or the Windows SDK reference for a full list.

Enumerating CSPs and CSP types

The CSP's present on a system and associated types may be enumerated using the twapi::csps command

foreach csp [csps] {
    lassign $csp type name
    puts "$type: $name"
}
prov_rsa_full: Microsoft Base Cryptographic Provider v1.0
prov_dss_dh: Microsoft Base DSS and Diffie-Hellman Cryptographic Provider
prov_dss: Microsoft Base DSS Cryptographic Provider
prov_rsa_full: Microsoft Base Smart Card Crypto Provider
prov_dh_schannel: Microsoft DH SChannel Cryptographic Provider
prov_rsa_full: Microsoft Enhanced Cryptographic Provider v1.0
prov_dss_dh: Microsoft Enhanced DSS and Diffie-Hellman Cryptographic Provider
prov_rsa_aes: Microsoft Enhanced RSA and AES Cryptographic Provider
prov_rsa_schannel: Microsoft RSA SChannel Cryptographic Provider
prov_rsa_full: Microsoft Strong Cryptographic Provider

Notice from the list that there can be multiple CSPs of a specific type.

A related command is csp_types which lists the CSP types available on the system along with a description of each.

% foreach csp [csp_types] {
    lassign $csp type desc
    puts "$type: $desc"
}
prov_rsa_full: RSA Full (Signature and Key Exchange)
prov_dss: DSS Signature
prov_rsa_schannel: RSA SChannel
prov_dss_dh: DSS Signature with Diffie-Hellman Key Exchange
prov_dh_schannel: Diffie-Hellman SChannel
prov_rsa_aes: RSA Full and AES

Cryptographic contexts

Cryptographic operations require an application to choose at a minimum

• the algorithms to be used, such as DES, RSA etc.

• the cryptographic keys

• a suitable CSP with the desired capabilities

In addition, cryptographic operations may hold state that needs to be maintained across successive operations.

Rather than requiring the application to pass all this information on every individual operation, CAPI encapsulates it into a cryptographic context represented by an opaque handle. This handle is then passed to APIs that do the actual cryptographic functions.

From Tcl, the context is created with the twapi::crypt_acquire command.

Note: TWAPI commands related to cryptographic contexts generally have the crypt prefix.

Allocating a default context

In the simplest case, you can call crypt_acquire without any options to get a default cryptographic context.

% set hcrypt [crypt_acquire]
1993632484576 HCRYPTPROV

This returns a handle to a context whose CSP type is prov_rsa_full using the default CSP for the user. This is sufficient for many common use cases where access to private keys is not required. More on this later.

Retrieving context information

Once you have a handle to a cryptographic context, you can retrieve various pieces of information about it. For example, you can retrieve the CSP type and the CSP that we landed up with by default.

% crypt_csp_type $hcrypt
prov_rsa_full
% crypt_csp $hcrypt
Microsoft Strong Cryptographic Provider

Perhaps more useful is information returned by the crypt_algorithms command about the algorithms and key sizes that are supported by the context.

% foreach algo [crypt_algorithms $hcrypt] {puts $algo}
algid 26114 defkeylen 128 minkeylen 40 maxkeylen 128 protocols {} name RC2 description {RSA Data Security's RC2}
algid 26625 defkeylen 128 minkeylen 40 maxkeylen 128 protocols {} name RC4 description {RSA Data Security's RC4}
algid 26113 defkeylen 56 minkeylen 56 maxkeylen 56 protocols {} name DES description {Data Encryption Standard (DES)}
...

These commands are not often needed in an application but are useful for exploring a CSP's capabilities.

We will see various other commands to retrieve information from a context as we go along.

Freeing a context

When a cryptographic context is no longer needed, it should be freed by calling the twapi::crypt_free command.

% crypt_free $hcrypt

Specifying CSPs and CSP types

In cases where the default CSP does not provide the functionality you need, you can specify a different type of CSP, or even pick a specific CSP, implementation with the -csptype and -csp options to crypt_acquire. Both options can be used by themselves or in conjunction.

% set hcrypt [crypt_acquire -csptype prov_rsa_aes]
1993680247440 HCRYPTPROV
% crypt_csp $hcrypt
Microsoft Enhanced RSA and AES Cryptographic Provider
% crypt_csp_type $hcrypt
prov_rsa_aes
% crypt_free $hcrypt

The above command will pick the first CSP that supports AES encryption. Alternatively, you could explicitly request (for example) the Microsoft CSP for AES built into Windows.

% set hcrypt [crypt_acquire -csp "Microsoft Enhanced RSA and AES Cryptographic Provider" -csptype prov_rsa_aes]
1993680247440 HCRYPTPROV
% crypt_csp_type $hcrypt
prov_rsa_aes
% crypt_free $hcrypt

Note that the command will raise an error if the CSP is specified by name but does not match the CSP type. In the above example, we explicitly passed the -csptype option as well since the CSP we specified would not have matched the default -csptype option value of prov_rsa_full.

Key containers

When all is said and done, keys are the center of the cryptographic universe. They are sufficiently important that the system even tries to protect them from the application owning the keys by keeping them within the control of the owning CSP as much as possible.

These keys are stored in key containers which are wholly managed by the CSP and whose internals are not directly accessible to applications. Any operations involving the keys are carried out by the CSP on the application's behalf.

The key containers are normally persisted to physical storage, which may be the file system, the registry or even a smartcard depending on the CSP. The built-in Microsoft CSPs store key containers based on the owning user accounts under the %APPDATA%\Microsoft\Crypto directory. The specific subdirectory depends on both the CSP in use as well as the user account owning the key container. These file based stores are protected with the operating system's access control settings so by default processes under different user accounts cannot access the private keys of a user. In addition, there is a machine wide store which can be used for keys that are shared across multiple accounts. We will explore this further when we give examples of operations on keys containers later.

A CSP stores multiple key containers, each identified by a name. Each container may contain multiple keys, each targeted for a different purpose, for example a public/private pair for signing, a symmetric key for encryption and so on.

The key containers under the control of a CSP can be listed by passing a cryptographic context associated with the CSP to the twapi::crypt_key_container_names command. (The command may take a few seconds to run.)

% join [crypt_key_container_names $hcrypt] \n
{0A15D46B-C9BB-4377-B36E-572419E0CBF9}
{CD6E722E-EA82-4F4F-A5D1-BC00D0724498}
twapitestintermediate
{56261871-4204-4455-8DA4-171BDB7928F0}
...

NOTE: The key containers listed with the above command are those under the ownership of the CSP associated with the context, and not only those associated with the context itself.

As you see, most key container names are GUIDs, though they are not restricted in being so. They do need to be unique though which is why most applications use GUIDs for the purpose.

Only asymmetric keys are persisted to permanent storage. Symmetric keys are not as they are intended to be regenerated for each cryptographic session. Morever, even public keys are persisted only if there is an associated private key.

We are going to postpone discussion of how keys are stored in a key container to the next post. For now, just know that key containers hold the keys used in cryptographic operations.

Contexts and key containers

When a cryptographic context is created, it is always with reference to a key container. We said earlier that a context ties together a CSP, the cryptographic algorithms and the keys to be used in cryptographic operations. These keys are the ones in the container associated with the context. When we obtained a context earlier using crypt_acquire, we did not explicitly specify a key container. In such a case, a context with the default container is acquired. In the case of Microsoft CSP's, this results in a new non-persistent context created in memory on every invocation. Keys can be added to this context but will not be saved to persistent storage. In the case of other CSP's, each invocation may or may not result in a new container if the CSP chooses to use a single container for all default context allocation.

Key containers storage is shared among applications. Therefore applications than make use of persistent keys should ensure they create containers with unique names for their own use so as to not create conflicts. This is commonly done by using a GUID to name the container. For the same reason, the default key containers should not be used when private key operations are required.

Allocating contexts for key containers

When an application needs to use persistent private keys, it needs to associate a non-default key container with the cryptographic context. This is done by specifying the -keycontainer option to crypt_acquire when a new context is allocated. However the command will fail if the key container does not exist.

% set hcrypt [crypt_acquire -keycontainer mynewcontainer]
Keyset does not exist

To create context with a new key container, specify the -create true option. When specified, the context will create a new key container of the given name if one does not exist.

% set hcrypt [crypt_acquire -keycontainer mynewcontainer -create 1]
1993680210064 HCRYPTPROV

NOTE: Even the default key container is not guaranteed to exist and must be explicitly created if needed.

You can verify the key container associated with a context with crypt_key_container_name.

% crypt_key_container_name $hcrypt
mynewcontainer
% crypt_free $hcrypt

Some CSPs may require the user to authenticate themselves to be granted access to the container, for example by typing the PIN for a hardware crypto token. An application can prevent this prompt by specifying the -silent option to crypt_acquire. However, this will cause an error to be raised if a user prompt was required.

Deleting key containers

To avoid littering the storage system with orphan key containers, application should delete key containers that are no longer needed, for example when the application is uninstalled. The command crypt_key_container_delete does the needful.

% crypt_key_container_delete mynewcontainer

In the case where the container is not owned by the default CSP, the appropriate CSP and CSP types need to be specified with the -csp and -csptype options to the command. Applications should generally avoid deleting the default containers in a CSP as they might be used by other applications.

The -verifycontext option

For performance reasons, it is important to distinguish between two categories of cryptographic operations:

• those that need a persisted private key

• those that don't

The first includes signing and decryption of messages, when the private keys used are persisted in a key container.

The second includes

• all operations requiring shared secrets since these are never persisted,

• computing hashes, since these do not use keys at all

• encryption and signature verification as these use public, not private, keys

• actions like retrieving CSP properties that do not perform cryptographic operations

Contexts that will be used for the first category must be created with the -verifycontext option to crypt_acquire set to false while operations in the second category should have it set as true. This option normally does not need to be specified because TWAPI sets it appropriately based on whether the default key container is used. If so, it is assumed the context will be used solely for operations in the second category and -verifycontext defaults to true. On the other hand , if a key container is specified with -keycontainer, TWAPI assumes operations in the first category are also required and will default -verifycontext to false.

The executive summary then is that generally,

• If you are only using operations in the second category, create a security context using crypt_acquire without any options (-verifycontext will default to true)

• If you will also use operations in the first category, use the -keycontainer to specify the key container to use (-verifycontext will default to false)

The above really describes the rules when using the Microsoft supplied CSPs. In other cases, for example hardware supported CSPs, you may need to explicitly specify the -verifycontext option. See the Microsoft SDK documentation of CryptAcquire for details.

User and machine containers

Persisted key containers may be stored in a location that is specific to the user account or in a common system location. This is controlled by specifying the -keysettype user (default) or -keysettype machine option respectively when the container is created with the crypt_acquire command.

For the Microsoft provided CSP's, the user-specific location is the user's profile on disk. These are then accessible only by that user account, or if the account is an administrative account, from the LocalSystem account on XP and from any account with administrative privileges on newer Windows versions. We can verify this by examining the associated security descriptor:

% set hcrypt [crypt_acquire -keycontainer twapitestintermediate]
1993680218320 HCRYPTPROV
% get_security_descriptor_text [crypt_get_security_descriptor $hcrypt]
Flags:  dacl_present self_relative
Owner:  ashok
Group:  None
DACL:
  Rev: 2
  ACE #1
    Type: Allow
    User: S-1-5-18 (SYSTEM)
    Rights: generic_read, generic_all
    Inherited: Yes
    Include self: Yes
    Recurse containers: No
    Recurse objects: No
    Recurse single level only: No
  ACE #2
    Type: Allow
    User: S-1-5-32-544 (Administrators)
    Rights: generic_read, generic_all
    Inherited: Yes
    Include self: Yes
    Recurse containers: No
    Recurse objects: No
    Recurse single level only: No
  ACE #3
    Type: Allow
    User: S-1-5-21-3445488791-2581517190-2081048042-1001 (ashok)
    Rights: generic_read, generic_all
    Inherited: Yes
    Include self: Yes
    Recurse containers: No
    Recurse objects: No
    Recurse single level only: No
SACL:
  Rev: null
% crypt_free $hcrypt

Key containers that are system-wide are accessible from the creator's account and from LocalSystem on XP and any system with administrative privileges on later Windows systems.

If the keys are to be accessed from other accounts, for example NetworkService, the security descriptor for the container can be modified using crypt_set_security_descriptor appropriately. See Windows Security for more on security descriptors.

NOTE: Any calls to crypt_acquire that reference a key container created with the -keysettype machine option must also specify that option whenever that container is acquired.

Coming up next

So far we have only been putzing around the topic of cryptography. We have not actually talked about the actual operations of encryption, signing etc. and you might be wondering when I will get around to those topics. Alas, dear reader, that will not happen in the next post either, because we have yet to talk about keys, how they are generated, stored in key containers, and accessed.