Does SSL decryption deserve a place in the enterprise data center?

Many companies use SSL decryption to monitor network traffic and add another layer of security, but what does it actually do for data centers?

SSL decryption by enterprise networks has been controversial, to say the least.

Privacy advocates maintain that the inspection of Secure Sockets Layer (SSL) encryption packets, such as those in Web mail, violates end-user trust on several levels. Conversely, those who have a more security-minded perspective on IT say SSL decryption is a necessary evil for combatting malware that uses SSL. So, what exactly is SSL decryption?

When an end user within a network chooses to initiate an encrypted session with an SSL-enabled Web server, the end-user device sends an HTTP GET message, and the server responds with a page signed by a certificate authority indicating that the client's Web browser can trust the certificate.

The Web server's public key certificate is submitted to the end-user device, and the end user uses this public certificate to encrypt the session key. After the session key is established, the end user and the Web server can communicate securely. For SSL decryption, a device is placed inline between the end user and the SSL-enabled Web server. The decryption device, in essence, mimics the behavior of the intended Web server, and tricks the end user into communicating with it rather than with the Web server. The SSL decryption device then reads all traffic between the end user and the intended Web server.

What becomes immediately apparent is that end users in such networks should not have any expectation of privacy and should assume that all SSL traffic is being observed as though it were in the clear. While this practice might seem underhanded at first glance, there are valid reasons why SSL decryption has been implemented in numerous enterprise networks. The first reason pertains to protecting proprietary or sensitive information from being stolen from an enterprise network. However, a report by analyst and research firm NSS Labs Inc. states that the primary reason for implementing SSL decryption has to do with SSL-encrypted malware.

The concept of encrypted malware should be on every data center administrator's radar as something to watch. Several attack vectors that use encrypted malware communication exist in the wild, but whichever attack vector administrators choose, it should be noted that not performing SSL decryption leaves a significant portion of traffic traversing the data center unexamined and therefore oblivious to SSL-encrypted malware.

According to NSS Labs' estimates, SSL-encrypted traffic accounts for 25% to 35% of all outbound traffic within enterprise networks. If this is accurate, data center administrators could be blind to one-third of all traffic without SSL decryption.

Furthermore, according to NIST's Special Publication 800-5, the standard default encryption cipher will change to 2,048 bits by the end of this year. This new standard is expected to increase network overhead significantly, and place an added strain on in-place SSL decryption solutions.

Taking this information into account, should a data center use SSL decryption as a part of its security solution? The answer actually depends on the organization.

The NSS Labs report focuses primarily on end users accessing SSL-enabled Web servers from inside an enterprise network. This is a slightly different scenario from what goes on inside the typical enterprise network where the end user resides.

From a data center perspective, the data center is often the entity hosting the Web server that the end user is trying to access. Add to this the fact that NSS Labs estimates that only around 1% of all malware attempting to access the enterprise is SSL-encrypted, and one may justifiably conclude that SSL decryption in the data center is not worth the cost financially or in performance.

SSL decryption doesn't protect the data center itself, but installing SSL decryption devices within the data center can monitor traffic from end users within the enterprise who first traverse the data center before leaping into the wild. From this perspective, data center administrators may rest assured that SSL decryption devices will at some point find a place in the data center, so it would behoove them to be fluent in SSL decryption methods and best practices.

About the author:

Brad Casey is an expert on network security with experience in penetration testing, public key infrastructure, VoIP and network packet analysis. He also covers system administration, Active Directory and Windows Server 2008, with interest in Linux virtualization and Wireshark captures. He spent five years in security assessment testing for the U.S. Air Force.

[email protected]

Dig Deeper on Data center budget and culture