Unable to create an instance on the Java Virtual Machine
After installing JDeveloper 11G successfully, we encountered the following error while starting JDeveloper:
Unable to create an instance on the Java Virtual Machine located on c:\jdeveloper11g\Middleware\jdk160_05\jre\bin\client\jvm.dll

Using Google and Oracle Support showed us that the virtual memory for the PC was not set to level that allowed the JVM to initialise.
The solution is to increase the virtual memory or swap area. To do this on Vista:
My Computer > Properties > Advanced System Settings > tab Advanced > Performance - Settings > tab Advanced > Virtual memory - Change
I have read this solution from Internet and I did it but it still happen again! That is why I said it is funny Oracle Jdeveloper!
What OS are you using? On some Windows versions a workaround is limiting the maximum heap size of your application to e.g. 512M instead of 1024M.
hth