However, there is the problem of responsiveness as well as speed. For a Just In Time (JIT) compiler you have to allow for the time it takes to compile the code before it executes. To avoid keeping the user hanging around, V8 performs two compilation stages. The first is triggered just before the code executes and it is fast, but doesn't attempt any optimization. Code that executes more than once is subjected to a second pass of compilation using an optimizing compiler. This pass takes more time and sometimes the user notices the moments of unresponsiveness and it can produce stuttering and dropped frames in complex applications like games.
The latest improvement in V8 is that the second phase of compilation runs concurrently and on a separate background thread. This simple change is said to make things run smoothly and boosts benchmark performance by 27%.
Activity diagram for single threaded compilation
Activity diagram for two threads
The improved V8 is currently available in the Chrome beta channel, Android included, and unless there is some really difficult subtle problem with the idea it will make its way to general release soon.
Personally I'm surprised that the two-pass compiler didn't make use of a separate thread from the start. If you want to reduce latency then threading is the way to do it, especially so in this era of multicore processors. My guess is that the difficulty has been in separating the compilation from the main thread and in sorting out the sychronization and communication. Such things are always much more difficult in practice than in theory.