Tuesday, December 10, 2013

The "behind the scene" of DexWare, a DalvikVM based service for the iCTF 2013.

This write-up will describe the "behind the scene" of DexWare, a service I wrote for the iCTF 2013. To the best of my knowledge, this is the first service in the history of CTFs to be based on Dalvik-bytecode!! I hope this write-up will be a useful starting point for those who will attempt something similar!

You can find the source code and the compiled binaries on github (link). Also, feel free to ping me on twitter (@reyammer) for any questions.


It seems that many of the teams found this service particularly challenging to hack, even after discovering a possible attack vector. The fact that heavy usage of Java reflection was required to hack the service might be related to that :D. In any case, let me tell you, this was a pretty complex service to be set up as well, especially as it needed to run for 8+ hours, on 100+ VMs, at a risk of compromising the entire game if unstable.

These are the points I will touch in this write-up:

  • How to compile a DalvikVM for Linux/x86
  • How to actually run DEX bytecode on Linux/x86
  • Discussion about Dexware, the service
  • Quick overview on how to exploit the service
  • How to make the service reliable (bug in DexClassLoader?)

How to compile a DalvikVM for Linux/x86

The vulnerable box for the iCTF needed to be a classic Linux/x86 VM. Hence, to run a service like this, you first need a DalvikVM runnable on Linux/x86. Once you know how to do it, it's pretty straightforward.

After you download the entire AOSP repo, just fire the following commands:

$ cd <aospdir>
$ source build/envsetup.sh
$ lunch # and then select "aosp_x86-eng"
$ make -j8

You could actually compile only the parts related to the DalvikVM, but compiling everything on a moder machine doesn't take that much in any case. Once you have that, you will find two different directories in the "out" folder. The "host" directory, that contains everything related to the host (your current machine), and the "target" directory, that contains a bunch of files for the target. As we want to run the VM on the host itself, the file interesting to us are those ones in the "host" directory.

After the compilation is over, you will have your nice dalvikvm x86 executable!

$ file <aospdir>/out/host/linux-x86/bin/dalvikvm
dalvikvm: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped

Note that not all the files in the "host" directory are actually needed to run the VM. The files that I included on github (link) should be the smallest subset of files that is required.

How to run DEX bytecode on Linux/x86

Once you have the binary, it is not entirely straightforward to run pure Dalvik bytecode. Indeed, the DalvikVM executable expects to be run within the context of the Android OS, where many files are in specific directories and specific environment variables are set. Of course, the goal here is to run dalvik bytecode, not Android applications! A quick clarification to the readers not familiar with the Android ecosystem: Android applications are APK files, that, among many other components, contain the classes.dex file, the Dalvik bytecode. Here we are going to run only the actual code, while we'll forget about the remaining Android-related components. In fact, to run a full-fledge Android app, we would need a full-fledge Android OS with all its components running, something that is not going to happen given the many space/memory constraints we have in this setting (iCTF).

In any case, long story short, this is how I managed to run the VM. First, you need to create a "dalvik" bash script (full version here)

$ cat dalvik

CURRDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
export ANDROID_ROOT=$CURRDIR/linux-x86

export ANDROID_DATA=/tmp/dalvik-data
mkdir -p $ANDROID_DATA/dalvik-cache

$CURRDIR/linux-x86/bin/dalvikvm \
:$ANDROID_ROOT/framework/apache-xml-hostdex.jar \

Once you have this script, you can actually run your piece of Dalvik-bytecode by first wrapping the .dex file in a .jar, and by then calling the script with "./dalvik -cp <jar_fp> <MainClass>", where <jar_fp> is the path to the JAR file, and MainClass is the name of the class that defines the main() method.

How to do that is pretty simple: you first compile your Java application into .class files. Then, you create a DEX wrapper in a JAR with dx:

$ dx --dex --output=dexware.jar *.class

DexWare, the service

The DexWare service (source code) was meant to be an innovative firwmare to manage super-secret chemical formulas related to uranium :-). Each of the formula was identified by a formula ID, and each of them was protected by a different password. Such formulas were the CTF's flags the teams needed to steal!

The service was maintaining a DB of the formulas in a hashmap field, and, in theory, a user could access to a formula only by knowing the associated password.

Of course, a vulnerability was added to the service, such that an attacker was able to steal the formula (the flag!) even without knowing the password.

Which vulnerability? Well, I wanted it to be a nice service where the teams need to exploit a real-world vulnerability. The decision was really simple: as our most recent paper (that is going to be presented at NDSS 2014, stay tuned!) is about code injection vulnerabilities in Android apps, that was a no-brainer. Gotta be a code injection vulnerability :-)

The service has a self-upgrade functionality. When invoked, the service goes through the list of files in a specific temporary directory, check for files with the .jar extension, load them, and execute the upgrade() method of the DexWareUpgrader class, if implemented. And how were the teams supposed to write a jar in the temp dir? Well, by a pure coincidence, a hidden functionality was added to the service so that it was possible to dump arbitrary files to that specific temporary directory. Easy peasy, isn't it?

The Exploit

Actually, even after knowing the attack vector, it is not that easy. In fact, a lot of Java reflective calls were necessary to get a hold on the DB formula field, and, to do that, the teams needed to access the (undocumented?) "this$0" field to first access the outer class (that had the formula DB field). As some nice write-ups on how to exploit this service already popped out around the net, I will stop with the details here, and I invite the interested readers to check those links out: link1, link2 :)

How to make the service reliable. Bug in DexClassLoader?

After developing my own exploit, I tried to stress-test my service to check whether the memory requirements were reasonable. And something interesting happened: after I run my exploit (together with benign traffic) for ~5K times, the DalvikVM was nicely and reliably segfaulting in my face. Awesome, isn't it? :/

As it turned out after a quick investigation, everything was crashing due to "Too many opened files" errors. Wait, what?

After a deeper investigation, I started to blame the DexClassLoader. In fact, at each run of the exploit, the service was creating a DexClassLoader object, that in turn would open the jar file, to then execute the upgrade method. What's the issue? It seems that the DexClassLoader does *never* close the loaded jar file, and that's why my service was going out of file handles!

I tried my best to force the DexClassLoader to close the associate handle, but nothing worked out (even after deleting the JAR, setting to null all the Java objects involved, and explicitly calling the garbage collector). I also checked in the actual implementation of DexClassLoader, but I failed to find any reference on how to close the loaded JAR.

As a workaround, I implemented a routine in the service that every once in a while was going through the JAR files in that temporary directory and, through JNI calls, manually closing all the handles. Dirty as sh*t, but pretty effective :) (source code)

I do understand that an innocuous DexClassLoader is not supposed to be abused the way I did, but, still, I think it's a bug :-)


All in all, that was a really interesting service to write, and it seems the teams enjoyed it as well. For the future, I'm planning to have a service where actual Dalvik bytecode-related skills are required: in fact, the key technical challenges were related to perform tons of Java reflective calls, while no specific Dalvik technical skill was required. I'll try to pull something out for the next iCTF :-)

As always, any sort of feedback is welcome! Let's keep in touch and feel free to ping me on twitter at @reyammer. Finally, if you played the CTF and you solved this challenge: remote high-five man, awesome job :-)