TornadoVM is a plug-in to OpenJDK that allows developers to automatically run Java programs on heterogeneous hardware.
The project has just matured to version 1.0.

That means that TornadoVM allows you to run Java on GPU's, FPGAs and dedicated hardware in order to achieve massive gains in performance. Being able to run on such hardware opens the gates to running resource-hungry applications in Machine Learning, NLP or LLM. And computer graphics too.

The gold standard of raytracing which until now was a sole privilege of C++, is now capable of running with Java too, courtesy of the TornadoVM showcasing the Java Ray Tracer project that is run on Intel HD Graphics and NVIDIA GPUs.

Coding wise, TornadoVM allows developers to write code that can be easily offloaded to hardware accelerators without having to get deep into the underlaying computing architectures or heterogeneous parallel programming models.

As a high level overview, TornadoVM provides the following features:

• Hardware agnostic APIs: The APIs are hardware agnostic, which means that, from the developers’ view, the source code is exactly the same for CPUs, GPUs and FPGAs. It is the TornadoVM runtime and the TornadoVM JIT Compiler that specialises the code per architecture.
• Optimising Just In Time (JIT) compiler per device architecture: This means that the code is specialised per architecture (e. g. , the GPU generated code is specialised differently compared to FPGAs and CPUs).
• Live task migration: this means that TornadoVM can migrate, at runtime, tasks from one accelerator to another (e. g. , from CPU to the GPU).
• Batch processing: TornadoVM can partition datasets to process in batches on devices that has less physical memory than the main device (CPUs).
• Deployable from edge to cloud: TornadoVM can be deployed on low-power device (e. g. , NVIDIA Jetson Nano), Desktop PCs, servers and data centers.
• Containers: TornadoVM can be used within Docker containers for running on NVIDIA GPUs, Intel CPUs and Intel GPUs.

We had covered the project upon taking its baby steps steps in TornadoVM Makes It Possible To Run Java on GPUs and FPGAs under version 0.13. Now the project reaches the mileston version 1.0, going GA.

This version comes with support for the JDK21 and the GraalVM 23.1.0 JIT Compiler, as well as the Truffle languages (the languages which interpreters are implemented with the Truffle framework of GraalVM) of Python, Ruby and Javascript.
This last property means that you can execute a code segment expressed in Python from Java and vice versa, invoke Java functions from those programming languages. For instance , here's an example of GraalVM Polyglot using Python and Tornado. The Python program calls MyCompute. compute() to accelerate vector addition on a GPU/FPGA.

Version 1 also comes with custom off-heap data types taking advantage of JDK21's Panama Memory Segment API which enables native interop. Off-Heap memory refers to memory that is managed by the operating system and not the JVM, as such these types encapsulate a Memory Segment, that is, a contiguous region of memory outside the Java heap.

Below is a list of the new types, with an arrow pointing from the on-heap primitive array types to their off-heap equivalent.

  int[ ]        ->    IntArray

  float[ ]     ->    FloatArray

  double[ ] ->    DoubleArray

  long[ ]     ->    LongArray

  char[ ]     ->    CharArray

  short[ ]    ->    ShortArray

  byte [ ]    ->    ByteArray

The existing Matrix and Vector collection types that TornadoVM offers (e. g. , VectorFloat, Matrix2DDouble, etc. ) have been refactored to use internally these off-heap data types instead of primitive arrays.

Of course, when running an older TornadoVM version, to use the new off heap types you have some tweaking to do. There's detailed instructions provided by the official docs.

These might have been the most important features added to the new version but there are others more minor too:

• cospi and sinpi math operations supported for OpenCL, PTX and SPIR-V.
• Vector 16 data types supported for float, double and int.
• Support for Mesa's rusticl.
• Device default ordering improved based on maximum thread size.
• Move all the installation and configuration scripts from Bash to Python.
• The installation process has been improved for Linux and OSx with M1/M2 chips.

TornadoVM can be installed manually on a variety of platforms and is also available as Docker images. The project is open source and hosted on Github. The official docs are pretty detailed and easy to get started with.

So there you have it. Java stepping into C++ and CUDA territories and another breakthrough for Java in 2023, on all fronts.

More Information

TornadoVM v1.0

TornadoVM docs

Related Articles

TornadoVM Makes It Possible To Run Java on GPUs and FPGAs

The I Programmer Java 2023 Recap

GraalVM under the covers

Program Deep Learning on the GPU with Triton