Numba supports jitting ctypes and CFFI function calls. Numba will automatically figure out the signatures of the functions and data. Below is a Gibbs sampling implementation that accesses ctypes (or CFFI) functions defined in another module ( rk_seed, rk_gamma and rk_normal), and that passes in a pointer to a struct also allocated with ctypes (state_p):
def gibbs(N, thin):
rng.rk_seed(0, rng.state_p)
x = 0
y = 0
samples = np.empty((N,2))
for i in range(N):
for j in range(thin):
x = rng.rk_gamma(rng.state_p, 3.0, 1.0/(y**2+4))
y = rng.rk_normal(rng.state_p, 1.0/(x+1), 1.0/math.sqrt(2+2*x))
samples[i, 0] = x
samples[i, 1] = y
return samples
Note
Passing in ctypes or CFFI libraries to autojit functions does not yet work. However, passing in individual functions does.
Numba allows users to deal with high-level as well as low-level C-like code. Users can access and define new or external structs, deal with pointers, and call C functions natively.
Note
Type declarations are covered in in Types and Variables.
Structs can be declared on the stack, passed in as arguments, returned from external or Numba functions, or originate from record arrays. Structs currently have copy-by-value semantics, but this is likely to change.
Struct fields can accessed as follows:
- struct.attr
- struct['attr']
- struct[field_index]
An example is shown below:
# -*- coding: utf-8 -*-
from __future__ import print_function, division, absolute_import
from numba import struct, jit, double
import numpy as np
record_type = struct([('x', double), ('y', double)])
record_dtype = record_type.get_dtype()
a = np.array([(1.0, 2.0), (3.0, 4.0)], dtype=record_dtype)
@jit(argtypes=[record_type[:]])
def hypot(data):
# return types of numpy functions are inferred
result = np.empty_like(data, dtype=np.float64)
# notice access to structure elements 'x' and 'y' via attribute access
# You can also index by field name or field index:
# data[i].x == data[i]['x'] == data[i][0]
for i in range(data.shape[0]):
result[i] = np.sqrt(data[i].x * data[i].x + data[i].y * data[i].y)
return result
print(hypot(a))
# Notice inferred return type
print(hypot.signature)
# Notice native sqrt calls and for.body direct access to memory...
#print(hypot.lfunc)
Pointers in Numba can be used in a similar way to C. They can be cast, indexed and operated on with pointer arithmetic. Currently it is however not possible to obtain the address of an lvalue.
An example is shown below:
# -*- coding: utf-8 -*-
from __future__ import print_function, division, absolute_import
import numba
from numba import *
from numba.tests.test_support import autojit_py3doc
import numpy as np
int32p = int32.pointer()
voidp = void.pointer()
@autojit_py3doc
def test_pointer_arithmetic():
"""
>>> test_pointer_arithmetic()
48L
"""
p = int32p(Py_uintptr_t(0))
p = p + 10
p += 2
return Py_uintptr_t(p) # 0 + 4 * 12
@autojit_py3doc(locals={"pointer_value": Py_uintptr_t})
def test_pointer_indexing(pointer_value, type_p):
"""
>>> a = np.array([1, 2, 3, 4], dtype=np.float32)
>>> test_pointer_indexing(a.ctypes.data, float32.pointer())
(1.0, 2.0, 3.0, 4.0)
>>> a = np.array([1, 2, 3, 4], dtype=np.int64)
>>> test_pointer_indexing(a.ctypes.data, int64.pointer())
(1L, 2L, 3L, 4L)
"""
p = type_p(pointer_value)
return p[0], p[1], p[2], p[3]
@autojit
def test_compare_null():
"""
>>> test_compare_null()
True
"""
return voidp(Py_uintptr_t(0)) == numba.NULL
numba.testing.testmod()
Numba allows users to declare and use LLVM intrinsics and instructions directly. This allows one to implement very low-level features directly in Python, while maintaining compatibility with Python code not compiled with numba itself.
This declares an LLVM intrinsic function with the given name. E.g.
# declare i64 @llvm.readcyclecounter()
intrin = numba.declare_intrinsic(int64(), "llvm.readcyclecounter")
print intrin()
Note
Intrinsics are not yet implemented. Only the instructions below are.
This declares an LLVM instruction named name as a function. E.g. we can use the srem instruction [1] to calculate the remainder of a signed integer, in an equivalent manner to how modulo works in C (see [2] and [3] for how this differs from Python).
>>> rem = nb.declare_instruction(int32(int32, int32), 'srem')
>>>
>>> @jit(int32(int32, int32))
... def py_modulo(a, n):
... r = rem(a, n)
... if r != 0 and (r ^ n) < 0:
... r += n
... return r
>>> # Instructions and intrinsics works directly in Python
>>> print rem(5, 2), rem(5, -2), rem(-5, 2), rem(-5, -2)
1 1 -1 -1
>>> # ... and are jitted in Numba functions
>>> print py_modulo(5, 2), py_modulo(5, -2), py_modulo(-5, 2), py_modulo(-5, -2)
1 -1 1 -1
>>> print py_modulo.lfunc
define i32 @__numba_specialized_6___main___2E_py_modulo(i32 %a, i32 %n) nounwind readnone {
entry:
%0 = srem i32 %a, %n
%1 = icmp ne i32 %0, 0
%2 = xor i32 %0, %n
%3 = icmp slt i32 %2, 0
%or.cond = and i1 %1, %3
%4 = select i1 %or.cond, i32 %n, i32 0
%. = add i32 %4, %0
ret i32 %.
}
As you can see the instructions can be used in Numba, where the instruction is inserted directly in the instruction stream, or pure-Python, where arguments are converted to native values, the operation executed, and the result returned as a Python object. We can verify that our modulo function works in pure Python:
>>> print py_modulo.py_func(5, 2), py_modulo.py_func( 5, -2), \
... py_modulo.py_func(-5, 2), py_modulo.py_func(-5, -2)
1 -1 1 -1
Note
Numba does not validate the signatures or validity of instructions and intrinsics. This is the responsibility of the user. Fortunately, the validity can be quickly verified :)
Users can take the address of a numba compiled function using numba.addressof:
Take the address of jit_func as a ctypes function. propagate indicates whether uncaught exceptions propogate or are written to stderr.
@jit(int32(int32, int32))
def mul(a, b):
return a * b
cmul = numba.addressof(mul)
print cmul(5, 2)
# Get the address as an Python int
addr_int = ctypes.cast(cmul, ctypes.c_void_p).value
print hex(addr_int)
Callers can currently check for exceptions (where appropriate) using PyErr_Occurred() (which requires the GIL).
nopython functions which do not directly or indirectly call functions requiring the GIL or use the with python construct can be called without the GIL. Numba does not check whether this is valid, nor does it currently acquire the GIL.
Currently supported bad values for return types:
| Return Type | Bad Value |
|---|---|
| object_ | NULL |
| floating | NaN |
These can be checked as follows:
float ret = my_numba_func(...);
if (ret != ret && PyErr_Occurred()) {
// Handle error
}
The error indicator for integer values is currently undecided, since constants in LLVM bitcode are printed in decimal form, but hex codes can be a better choice for other scenarios.