Monitoring Encrypted Client Hello (ECH) With Cisco Secure Firewall

Additional Contributor: Christopher Grabowski

The Encrypted Visibility Engine (EVE) in Cisco Secure Firewall has become one of the detection staples that we rely on in the Security Operations Center (SOC) that Cisco deploys to conferences and events throughout the year. EVE provides invaluable detections in conferences that have predominantly encrypted traffic. With no TLS (Transport Layer Security) decryption possible at these conferences (we actively encourage attendees to use encryption), EVE’s contribution to visibility is even more crucial. However, we can also take a lesson from the Buddha and recognize that while we have a great thing with EVE now and in the future, the world is constantly in a state of change.

A big change that we’re already seeing in conferences like GovWare is the usage of the Encrypted Client Hello (ECH). In this blog post we’ll take a brief look at EVE visibility, what ECH is, what ECH obscures, what visibility we still have for ECH traffic, and how much ECH activity we observed at GovWare. ECH is an important change in network visibility that admins and analysts will want to get ahead of. ECH is an important change in network visibility that admins and analysts will want to get ahead of.Let’s start with the Encrypted Visibility Engine. EVE can fingerprint HTTP and HTTPS sessions that are initiated by malicious processes, like this one from the GovWare conference.

This event is a port 80 HTTP connection that we had full visibility into, which can occur either for native HTTP connections or HTTPS connections that are fully decrypted. Let’s take a look at what we can see in the HTTP connection using the full session packet capture solution for our SOC, Endace.

With the above data, EVE has full access to the HTTP POST request and the URI, which is the most visibility we can achieve and gives us the most accurate detection. While EVE shines brightest in fingerprinting encrypted sessions, the more visibility we have into the session, the better.

Let’s also look at an EVE fingerprint of a TLS encrypted HTTPS session from GovWare.

For TLS encrypted traffic, one of the datapoints we can use is the Server Name Indicator (SNI) field.

SNI is a datapoint that EVE can use as part of its fingerprinting process for this TLS session. Even though this session was HTTPS, the SNI field is still transmitted in clear text, before the session is encrypted.

Now let’s pivot to ECH. Starting with VDB 416, Cisco Secure Firewall has application detection to identify connections to known ECH servers starting with cloudflare-ech.com, as shown in the events below.

These connections differ from our prior HTTPS example in that the actual SNI for the connection is obscured and replaced with a generic Cloudflare ECH.

With ECH, the actual SNI is only transmitted after the HTTPS session has been encrypted. Returning to the prior EVE results:

There are two underlying detections in the screenshot above. ECH has obscured the SNI field, replacing it with cloudflare-ech.com and resulting in an accurate Web Application detection for ECH Servers. However, even with ECH obscuring the SNI field, EVE was still able to fingerprint the session and deliver a 100% confidence rate for the underlying process that established the ECH connection, the Firefox browser. While the loss of the SNI field due to ECH decreases our visibility, EVE still provides some context that we can use in assessing the connection. Below is the full EVE fingerprint for the connection, showing both the process identification and the ECH obscured SNI.

With ECH obscuring the SNI field, can we simply fall back to DNS to learn the destination? Unfortunately, we cannot, as ECH can be paired with encrypted DNS requests. Below is the DNS request that preceded the ECH connection.

With the DNS request also going over HTTPS, we are still unable to see the destination, a marked loss in visibility. The full EVE fingerprint for the DNS over HTTPS session is below.

How prevalent was ECH traffic at GovWare? The good news for our visibility (or the bad news for personal privacy, depending on your perspective) is that ECH didn’t crack our list of top Web Applications seen.

ECH was actually extremely rare, registering only 33 matches over the course of the conference.

Still, the fact that this traffic is already in the wild should spur security teams to start tracking its prevalence within their networks, assess loss in traffic visibility, and consider whether the traffic is acceptable within the context of encryption or whether administrative actions should be taken to limit it.

The web application detection shown in this blog post is a good place to start for visibility into how prevalent ECH traffic is within a given network. Lastly, the additional context that EVE provides for ECH sessions should also be considered from a threat and admin perspective. In a world with growing levels of encryption, any additional visibility helps. At the Cisco SOC, we’ll be monitoring for ECH connections coming from unexpected source processes in future conferences.

Keep an eye out for an upcoming blog post by Cisco Firewall TME Christopher Grabowski for a more in-depth look at ECH and how administrators can respond.

Check out the other blogs by my colleagues in the GovWare SOC.

About GovWare

GovWare Conference and Exhibition is the region’s premier cyber information and connectivity platform, offering multi-channel touchpoints to drive community intel sharing, training, and strategic collaborations.

A trusted nexus for over three decades, GovWare unites policymakers, tech innovators, and end-users across Asia and beyond, driving pertinent dialogues on the latest trends and critical information flow. It empowers growth and innovation through collective insights and partnerships.

Its success lies in the trust and support from the cybersecurity and broader cyber community that it has had the privilege to serve over the years, as well as organisational partners who share the same values and mission to enrich the cyber ecosystem.

We’d love to hear what you think! Ask a question and stay connected with Cisco Security on social media.

Cisco Security Social Media

LinkedInFacebookInstagramX