Java, being widely used by the applications, has also been actively targeted by malware authors. One of the most common techniques to exploit Java applications, is to disable the security manager. This blog provides widely used logic used by malware authors to disable the security manager.
Per the Java tutorial:
‘A security manager is an object that defines a security policy for an application. This policy specifies actions that are unsafe or sensitive. Any actions not allowed by the security policy cause a SecurityException to be thrown. An application can also query its security manager to discover which actions are allowed.’
Once an application has a reference to the security manager object, it can request permission to do specific things. For example, System.exit, which terminates the Java virtual machine with an exit status, invokes SecurityManager.checkExit to ensure that the current thread has permission to shut down the application.
One very common technique used by malware authors to exploit Java is to disable the security manager. Once the security manager has been disabled, the next steps may involve loading a malicious executable or connecting to a malicious website.
One of the common calls which generally appear in a malicious jar to disable security manager is SetSecurityManager(null). Besides making a direct call to SetSecurityManager, there can be other approaches for example locating the address of java/lang/system and using it to disable the security manager. Basically, once the address has been located, memory is traversed until the address of getSecurityManager() is located. wrmMem() function is then called and null is written directly to the address of getSecurityManager(), thus disabling the security manager.
For more information regarding the pseudo code and exploitation of a vulnerability to disable a security manager, we encourage our readers to refer to recently published article in Virus Bulletin June 2013 issue.