Ravi Performance Benchmarks¶
Ravi’s reason for existence is to achieve greater performance than standard Lua 5.3. Hence performance benchmarks are of interest.
The programs used in the performance testing can be found at Ravi Tests folder.
|Program||Lua5.3.2||Ravi Int||Ravi(LLVM)||LuaJIT2.1 Int||LuaJIT2.1|
Following points are worth bearing in mind when looking at above benchmarks.
- For Ravi the timings above do not include the LLVM compilation time.
- The benchmarks were run on Windows 10 64-bit. LLVM version 3.9 was used. Ravi and Lua 5.3.2 were compiled using Visual C++ 2015.
- Some of the Ravi benchmarks are based on code that uses optional static types; additionally for the matmul benchmark a setting was used to disable array bounds checks for array read operations.
- Above benchmarks are primarily numerical. In real life scenarios there are other factors that affect performance. For instance, via FFI LuaJIT is able to make efficient calls to external C functions, but Ravi does not have a similar FFI interface. LuaJIT can also inline Lua function calls but Ravi does not have this ability and hence function calls go via the Lua infrastructure and are therefore expensive. Ravi’s code generation is best when types are annotated as otherwise the dynamic type checks degrade performance as above benchmarks show. Finally LLVM is a slow compiler relative to LuaJIT’s JIT compiler which is extremely fast.
- Performance of Lua 5.3.2 is better than 5.3.0 or 5.3.1, thanks to the table optimizations in this version.
In general to obtain the best performanc with Ravi, following steps are necessary.
- Annotate types as much as possible.
- Use fornum loops with integer counters.
- Avoid function calls inside loop bodies.
- Do not assume that JIT compilation is beneficial - benchmark code with and without JIT compilation.
- Try to compile a set of functions (in a table) preferably at program startup. This way you pay for the JIT compilation cost only once.
- Dump the generated Lua bytecode to see if specialised Ravi bytecodes are being generated or not. If not you may be missing type annotations.
- Avoid using globals.
- Note that only functions executing in the main Lua thread are run in JIT mode. Coroutines in particular are always interpreted.
- Also note that tail calls are expensive in JIT mode as they are treated as normal function calls; so it is better to avoid JIT compilation of code that relies upon tail calls.