2017 OWASP A8 Update: Insecure Deserialization

Introduction

2017 saw a new addition to the Open Web Application Security Project’s (OWASP) Top Ten list of web application vulnerabilities — insecure deserialization. This vulnerability enables malicious use of untrusted data to exploit existing application code, inflict a denial of service (DoS) attack or execute arbitrary code upon it being deserialized.

Although the Java deserialization bug has been around for years, it was considered by many too difficult to exploit and only got attention after the 2015 publication of research by Foxglove Security. Foxglove demonstrated how the vulnerability can easily be exploited in popular software like WebSphere, WebLogic, JBoss, Jenkins and OpenNMS. What these applications have in common is the Apache Commons Collections (ACC) Java library, and while many of the vulnerabilities in the ACC have been patched, too many organizations are still using outdated versions and are at risk of attack.

Having said that, security experts point out Java default deserialization methods deserialize every supplied serialized object, without doing any checks; it’s not a case of particular Java libraries being vulnerable.

The threat is growing. Imperva research found the number of deserialization vulnerabilities from 2016-2017 increased by up to 300 percent from previous years, certainly enough of a threat to get it on the OWASP list.

Interestingly, with the surge in Bitcoin prices making headline news over the past year, the security industry has seen a corresponding increase in deserialization attacks that are launched with the intent of installing cryptomining malware on vulnerable web servers, throttling their CPU usage.

What Is Insecure Deserialization?

OWASP describes the core of the insecure deserialization vulnerability as malformed data or unexpected data that can be used to abuse application logic, deny service or execute arbitrary code.

To understand the vulnerability, you need to understand the concepts of serialization and deserialization. Serialization refers to the process of converting an object instance or data structure into a specific format — e.g., XML or JSON — which can be saved to a file or to memory, or sent over a network. Its main purpose is to save the state of an object and recreate it when needed. Deserialization is the reverse of this process; it takes the serialized data and rebuilds it. For example, two applications written in different languages may use these processes to standardize shared information.

Code Mentor has a great metaphor: When we stop or pause a running computer game, it usually starts from the state where it was left off when we play it again. This is possible through the process of serialization, which is saving the current object state as a byte stream into a file or database. Then, to restore the object’s state when we access the game again, deserialization takes place.

In terms of risk, OWASP ranks the opportunity for insecure deserialization as relatively low, prevalence and detectability both as medium, and impact as high.

Who Is Vulnerable to Insecure Deserialization?

Anyone who performs the deserialization process in their code. Versions of some popular products that are vulnerable include: Cisco, Apache Tomcat, Oracle Hyperion, Atlassian Jira and HP. Applications written in C++, Ruby, Java, Python, PHP, ASP.NET and other languages can all be susceptible to insecure deserialization vulnerabilities.

How Does it Insecure Deserialization Work?

Let’s start by looking at untrusted data, e.g., unknown user input to a form. Untrusted data cannot be trusted to be well formed. The problem is many programming languages allow native, customizable serialization processes. Unfortunately, these native deserialization mechanisms can be repurposed for malicious effect when operating on untrusted data. And that can happen because the deserialization purpose effectively may allow for some code to be executed before the untrusted data can be verified.

Default Java classes responsible for deserialization first deserialize objects and then try to cast the object to the expected Java class. This happens even if the objects are not instances of the expected type; if not, after deserialization an exception arises when trying to cast the object to the expected type because Java's strict type system is meant to ensure you only get valid objects. However, at this stage, some (and possibly malicious) application code may have already been executed.

In Java, for instance, the code to read a data object from a serialized stream is very straightforward. Unfortunately, according to Jeff Williams, co-founder and chief technology officer of Contrast Security, there’s no way to know what you’re deserializing before you’ve decoded it. “So an attacker can serialize a bunch of malicious objects and send them to your application. Once you call readObject() [to deserialize the data], it’s too late.” As we’ve seen above, the attackers already have their foot in the door.

Read the long version of this explanation and more of the technical details here and here.

How Can You Detect Insecure Deserialization?

The insecure deserialization error is difficult to detect and while some tools can discover deserialization flaws, human assistance is frequently needed to validate the problem. The following tools include useful documentation and step-by-step instructions to get started.

• The Java Deserialization Scanner is an extension for Burp Suite that gives it the ability to find Java deserialization vulnerabilities. The extension allows the user to discover and exploit Java deserialization vulnerabilities with different encodings. After a Java deserialization vulnerability has been found, an exploitation tab provides an interface to exploit the found vulnerabilities.
• Guidance on how to effectively find vulnerabilities in web applications and APIs is provided in the OWASP Testing Guide.
• ysoserial, the brainchild of Chris Frohoff  and Gabriel Lawrence, is a collection of utilities and property-oriented programming "gadget chains" discovered in common Java libraries that can, under the right conditions, exploit Java applications performing unsafe deserialization of objects. Watch their video on deserializing objects here.
• The FindBugs plugin can detect 125 different vulnerability types with over 787 unique API signatures.
• Serianalyzer is a static byte code analyzer for Java deserialization gadget research. The developers say the main purpose of this tool is as a research tool to audit code for dangerous behavior during deserialization. They warn it is not really useful to determine whether your application is vulnerable or not. If your application deserializes data crossing trust boundaries — you should assume it is. They admit one of the limitations of the tool is multiple false positives.
• The Serializekiller script enables you to scan many servers in a short time. To get started, you need to install Python 2, Curl and NMAP first. The script only scans some default ports. If you want to scan non-default ports, you can specify those ports in the target’s file.
• Jinfinity is designed to consume all the memory of a target that's deserializing objects and, as the developer Arshan Dabirsiaghi says, eventually blow it up.

How Do You Prevent Insecure Deserialization?

Contrast Security’s Jeff Williams says the first thing organizations must do is find all the places where they are using deserialization of untrusted data. He has a few tips for preventing malicious deserialization. The trick is to allow deserialization but specify what classes are allowed, i.e., those classes your application reasonably expects to receive. You can explore Williams’ code suggestions here.

OWASP recommends the following to prevent this vulnerability:

• Implement regular integrity checks or encryption of the serialized objects
• Enforce strict type constraints during deserialization
• Isolate code that deserializes, so it runs in very low privilege environments like temporary containers
• Log deserialization exceptions and failures
• Restrict or monitor incoming and outgoing network connectivity from containers or servers that deserialize
• Monitor deserialization, alerting if a user deserializes constantly

How Do You Protect Your System From Insecure Deserialization?

First, make sure your developers have security awareness training. Pierre Ernst's paper, Look-Ahead Java Deserialization, is a good start. A great take away from this paper is that a service will deserialize a malicious object only if:

• The malicious object's class exists in the server's classpath. The attacker cannot simply send a serialized object of any class, because the service will be unable to load the class.
• The malicious object's class is either serializable or externalizable.

Some useful tools include:

• Notsoserial: A Java agent which mitigates deserialisation attacks by making certain classes unserializable. Its developers dub it a "deserialization firewall," giving you control over which classes your application should be allowed to deserialize.
• SerialKiller: A look-ahead Java deserialization library to secure applications from untrusted input. It works by inspecting Java classes during naming resolution and allows a combination of blacklisting/whitelisting techniques to secure your application.
• SWAT: A JVM instrumentation agent that helps you build a whitelist for classic Java deserialization occurrences. Once (on a test server) the instrumentation runs, it creates and logs files with the observed deserialization classes. This can be used to build a reasonable whitelist by observing the deserializations on a test server.

In addition, keep all software updated, restrict access to the network where possible and always be suspicious of deserialized data from unknown sources.

Insecure Deserialization Case Studies

San Francisco Metropolitan Transit Agency (SFMTA)

The attacker who gained access to computers at SFMTA in 2016 and infected them with ransomware probably took advantage of a deserialization vulnerability in an Oracle WebLogic server. After the event, which fortunately didn’t do much damage, security organization Krebs On Security was contacted by a security researcher who had managed to hack the attacker’s email by guessing the answer to his secret question. Another security searcher, Alex Holden, then analyzed the hacker’s emails. It was deduced the attacker used various open source tools to help find and infect new victims. He then used the “weblogic unserialize exploit” and especially targeted Oracle Corp. server products, including Primavera project portfolio management software.

The author of this article, posted on the Krebs website, suggests people should choose gibberish or completely unrelated answer to secret questions, answers that can’t be guessed or found in clues on personal and social media pages.

PayPal Engineering

In 2016, PayPal was forced to patch one of its business websites after researcher Michael Stepankin reported he’d uncovered a vulnerability that allowed him to execute arbitrary shell commands and access PayPal’s production databases. Stepankin was pipped to the post in his find by security researcher Mark Litchfield, a member of the PayPal Bug Bounty community who had reported the vulnerability a couple of days earlier. The bug was apparently missed because it was in an application that was not on the company’s core Java frameworks.

In a blog post, Stepankin said he noticed an unusual post form parameter “oldFormData” when testing one of the PayPal applications. The parameter was a Java serialized object without any signature, which means you can send a serialized object of any existing class to the server, and the “readObject” (or “readResolve”) method of that class will be called. Using the ysoserial tool, he then was able to request the PayPal server to make DNS and HTTP requests to Stepankin’s own server. He also requested, and received, PayPal's /etc/passwd file. This vulnerability could have enabled an attacker to breach PayPal’s servers, execute arbitrary commands on its server and potentially install a backdoor.

PayPal Engineering later published its take on the insecure deserialization vulnerability,

Conclusion

Knowledge is power. In order to raise awareness of the importance of OWASP's recommendations, Infosec Institute has created several secure coding modules for developers on its SecurityIQ platform. It also offers in-depth secure coding courses and Boot Camp trainings taught by security professionals with years of industry experience.

Sources

https://cwe.mitre.org/data/definitions/502.html

https://www.owasp.org/index.php/Top_10-2017_A8-Insecure_Deserialization

https://www.owasp.org/index.php/Deserialization_of_untrusted_data

https://www.owasp.org/index.php/Deserialization_Cheat_Sheet

https://www.contrastsecurity.com/security-influencers/java-serialization-vulnerability-threatens-millions-of-applications

https://www.ibm.com/developerworks/library/se-lookahead/

https://www.acunetix.com/blog/articles/what-is-insecure-deserialization/

https://www.acunetix.com/blog/articles/what-is-insecure-deserialization/

https://www.darkreading.com/informationweek-home/why-the-java-deserialization-bug-is-a-big-deal/d/d-id/1323237?

https://lgtm.com/blog/apache_struts_CVE-2017-9805_announcement

https://www.codementor.io/java/tutorial/serialization-and-deserialization-in-java

https://techblog.mediaservice.net/2017/05/reliable-discovery-and-exploitation-of-java-deserialization-vulnerabilities/

https://github.com/GrrrDog/Java-Deserialization-Cheat-Sheet#vulnerable-apps-without-public-sploitsneed-more-info

https://krebsonsecurity.com/2016/11/san-francisco-rail-system-hacker-hacked/

http://www.holdsecurity.com/

https://arstechnica.com/information-technology/2016/11/san-francisco-transit-ransomware-attacker-likely-used-year-old-java-exploit/

https://foxglovesecurity.com/2015/11/06/what-do-weblogic-websphere-jboss-jenkins-opennms-and-your-application-have-in-common-this-vulnerability/#thevulnerability

https://www.imperva.com/blog/2018/01/deserialization-attacks-surge-motivated-by-illegal-crypto-mining/

https://commons.apache.org/proper/commons-collections/security-reports.html

https://www.cryptocurrencychart.com/

https://www.ibm.com/developerworks/library/se-lookahead/