Why is Jython much slower than CPython, despite the JVM's advances?

Keep in mind that IronPython was started by one of the original Jython devs (Jim Huginin) in an attempt to prove that the .NET CLR was a poor platform for dynamic languages. He ended up proving himself wrong and the core of IronPython eventually became the .NET Dynamic Language Runtime (making other dynamic language implementations on .NET, such as IronRuby, significantly easier to build).

So there's two major points of difference there:

• the original .NET CLR devs benefited from additional industry VM experience relative to the early versions of the JVM, allowing them to avoid known problems without backwards compatibility concerns
• the same applied for Jim in knowing what traps to avoid based on his Jython experience

Add in a simple lack of development resources devoted to Jython relative to both CPython and IronPython, and Jython development priorities that focused on bringing it up to feature parity with recent versions of Python moreso than speed optimisations and it's quite understandable that Jython would lag when it came to speed.

That said, Jython is similar to both CPython and IronPython, in that the use of better algorithms often trumps poorer performance at microbenchmarks. The JVM/CLR also mean that dropping down to Java or C# for particular components is easier than dropping down into a C extension for CPython (although tools like Cython try to close that gap a bit).