Cache Julia module for faster startup and usage in Python Cache Julia module for faster startup and usage in Python python python

Cache Julia module for faster startup and usage in Python


This is a complex question. I will give the simplest approach. I assume you use Julia 0.7 (it should be released very soon):

A. generate a package in Julia; switch to package manager in Julia REPL and in Pkg manager session (you switch to it using ]) write: generate Ex1 (Ex1 will be our package)

B. you will have a package skeleton created; add whatever dependencies you need. I assume your simple example. Then the file Ex1.jl in src directory should be changed to:

__precompile__()module Ex1export simsim(a,b) = 1precompile(sim, (Int, Int))end

C. the key line is precompile(sim, (Int, Int)) as you force precompilation of sim function - you have to specify argument types for which you want the function to be precompiled

D. change Julia directory to the directory of Ex1 package and write activate . in Pkg manager mode to activate the package

E. while still in Pkg manager mode write precompile; this will trigger precompilation of your module. You can check in DEPOT_PATH[1] directory in compiled/v0.7/ subdirectory that Ex1 directory was created and it contains ji file with precompiled contents of your package

How to check that the package is actually precompiled? Exit Julia and enter it again in the directory of Ex1 package (to have a clean REPL). Now write:

A. activate . in Pkg manager mode (to have Ex1 visible)

B. go back to normal Julia mode and run the following:

julia> using Ex1julia> @time sim(1,1)  0.000003 seconds (5 allocations: 208 bytes)1julia> @time sim(1,1)  0.000003 seconds (4 allocations: 160 bytes)1julia> @time sim(1.0,1.0)  0.000182 seconds (394 allocations: 26.641 KiB)1julia> @time sim(1.0,1.0)  0.000002 seconds (4 allocations: 160 bytes)1

You can see that sim(1,1) is not compiled as we have precompiled sim for (Int,Int) arguments, but sim(1.0,1.0) had to be compiled as we did not precompile sim for (Float64,Float64).

In summary: the key thing to understand is that you precompile not a function but a specific method of this function. You will have to precompile all methods that you would use this way. All the rest in the explanation above are technical steps to get Julia do the precompilation.

Additional issues to keep in mind:

  • you can build the package into system image, this is complex; here you have the instructions: https://docs.julialang.org/en/latest/devdocs/sysimg/; You can also check out PackageCompiler.jl you have mentioned to help you in the process; it can be expected that static precompilation gets much better in the future;
  • in order for your package to be precompilable all dependencies have to be precompilable.

I hope the above works for you.