Why is Java faster when using a JIT vs. compiling to machine code?

A JIT compiler can be faster because the machine code is being generated on the exact machine that it will also execute on. This means that the JIT has the best possible information available to it to emit optimized code.

If you pre-compile bytecode into machine code, the compiler cannot optimize for the target machine(s), only the build machine.

I will paste an interesting answer given by the James Gosling in the Book Masterminds of Programming.

Well, I’ve heard it said thateffectively you have two compilers inthe Java world. You have the compilerto Java bytecode, and then you haveyour JIT, which basically recompileseverything specifically again. All ofyour scary optimizations are in theJIT.

James: Exactly. These days we’rebeating the really good C and C++compilers pretty much always. When yougo to the dynamic compiler, you gettwo advantages when the compiler’srunning right at the last moment. Oneis you know exactly what chipsetyou’re running on. So many times whenpeople are compiling a piece of Ccode, they have to compile it to runon kind of the generic x86architecture. Almost none of thebinaries you get are particularly welltuned for any of them. You downloadthe latest copy of Mozilla,and it’llrun on pretty much any Intelarchitecture CPU. There’s pretty muchone Linux binary. It’s pretty generic,and it’s compiled with GCC, which isnot a very good C compiler.

When HotSpot runs, it knows exactlywhat chipset you’re running on. Itknows exactly how the cache works. Itknows exactly how the memory hierarchyworks. It knows exactly how all thepipeline interlocks work in the CPU.It knows what instruction setextensions this chip has got. Itoptimizes for precisely what machineyou’re on. Then the other half of itis that it actually sees theapplication as it’s running. It’s ableto have statistics that know whichthings are important. It’s able toinline things that a C compiler couldnever do. The kind of stuff that getsinlined in the Java world is prettyamazing. Then you tack onto that theway the storage management works withthe modern garbage collectors. With amodern garbage collector, storageallocation is extremely fast.

The real killer for any AOT compiler is:

Class.forName(...)

This means that you cannot write a AOT compiler which covers ALL Java programs as there is information available only at runtime about the characteristics of the program. You can, however, do it on a subset of Java which is what I believe that gcj does.

Another typical example is the ability of a JIT to inline methods like getX() directly in the calling methods if it is found that it is safe to do so, and undoing it if appropriate, even if not explicitly helped by the programmer by telling that a method is final. The JIT can see that in the running program a given method is not overriden and is therefore in this instance can be treated as final. This might be different in the next invocation.

Edit 2019: Oracle has introduced GraalVM which allows AOT compilation on a subset of Java (a quite large one, but still a subset) with the primary requirement that all code is available at compile time. This allows for millisecond startup time of web containers.