2.6. Supported Python features

Apart from the Language part below, which applies to both object mode and nopython mode, this page only lists the features supported in nopython mode.

Warning

Numba behavior differs from Python semantics in some situations. We strongly advise reviewing Deviations from Python Semantics to become familiar with these differences.

2.6.1. Language

2.6.1.1. Constructs

Numba strives to support as much of the Python language as possible, but some language features are not available inside Numba-compiled functions. The following Python language features are not currently supported:

  • Class definition
  • Exception handling (try .. except, try .. finally)
  • Context management (the with statement)
  • Some comprehensions (list comprehension is supported, but not dict, set or generator comprehensions)
  • Generator delegation (yield from)

The raise statement is supported in several forms:

Similarly, the assert statement is supported with or without an error message.

2.6.1.2. Functions

2.6.1.2.1. Function calls

Numba supports function calls using positional and named arguments, as well as arguments with default values and *args (note the argument for *args can only be a tuple, not a list). Explicit **kwargs are not supported.

Function calls to locally defined inner functions are supported as long as they can be fully inlined.

2.6.1.2.2. Functions as arguments

Functions can be passed as argument into another function. But, they cannot be returned. For example:

from numba import jit

@jit
def add1(x):
    return x + 1

@jit
def bar(fn, x):
    return fn(x)

@jit
def foo(x):
    return bar(add1, x)

# Passing add1 within numba compiled code.
print(foo(1))
# Passing add1 into bar from interpreted code
print(bar(add1, 1))

Note

Numba does not handle function objects as real objects. Once a function is assigned to a variable, the variable cannot be re-assigned to a different function.

2.6.1.2.3. Inner function and closure

Numba now supports inner functions as long as they are non-recursive and only called locally, but not passed as argument or returned as result. The use of closure variables (variables defined in outer scopes) within an inner function is also supported.

2.6.1.2.4. Recursive calls

Most recursive call patterns are supported. The only restriction is that the recursive callee must have a control-flow path that returns without recursing. Numba is able to type-infer recursive functions without specifying the function type signature (which is required in numba 0.28 and earlier). Recursive calls can even call into a different overload of the function.

2.6.1.3. Generators

Numba supports generator functions and is able to compile them in object mode and nopython mode. The returned generator can be used both from Numba-compiled code and from regular Python code.

Coroutine features of generators are not supported (i.e. the generator.send(), generator.throw(), generator.close() methods).

2.6.2. Built-in types

2.6.2.1. int, bool

Arithmetic operations as well as truth values are supported.

The following attributes and methods are supported:

  • .conjugate()
  • .real
  • .imag

2.6.2.2. float, complex

Arithmetic operations as well as truth values are supported.

The following attributes and methods are supported:

  • .conjugate()
  • .real
  • .imag

2.6.2.3. str

Numba supports (Unicode) strings in Python 3. Strings can be passed into nopython mode as arguments, as well as constructed and returned from nopython mode. As in Python, slices (even of length 1) return a new, reference counted string. Optimized code paths for efficiently accessing single characters may be introduced in the future.

The in-memory representation is the same as was introduced in Python 3.4, with each string having a tag to indicate whether the string is using a 1, 2, or 4 byte character width in memory. When strings of different encodings are combined (as in concatenation), the resulting string automatically uses the larger character width of the two input strings. String slices also use the same character width as the original string, even if the slice could be represented with a narrower character width. (These details are invisible to the user, of course.)

The following functions, attributes and methods are currently supported:

  • len()
  • + (concatenation of strings)
  • in, .contains()
  • ==, <, <=, >, >= (comparison)
  • .startswith()
  • .endswith()
  • .find()
  • .split()
  • .join()

Additional operations as well as support for Python 2 strings / Python 3 bytes will be added in a future version of Numba. Python 2 Unicode objects will likely never be supported.

Warning

The performance of some operations is known to be slower than the CPython implementation. These include substring search (in, .contains() and find()) and string creation (like .split()). Improving the string performance is an ongoing task, but the speed of CPython is unlikely to be surpassed for basic string operation in isolation. Numba is most successfuly used for larger algorithms that happen to involve strings, where basic string operations are not the bottleneck.

2.6.2.4. tuple

The following operations are supported:

  • tuple construction
  • tuple unpacking
  • comparison between tuples
  • iteration and indexing over homogeneous tuples
  • addition (concatenation) between tuples
  • slicing tuples with a constant slice
  • the index method on tuples

2.6.2.5. list

Creating and returning lists from JIT-compiled functions is supported, as well as all methods and operations. Lists must be strictly homogeneous: Numba will reject any list containing objects of different types, even if the types are compatible (for example, [1, 2.5] is rejected as it contains a int and a float).

For example, to create a list of arrays:

In [1]: from numba import njit

In [2]: import numpy as np

In [3]: @njit
  ...: def foo(x):
  ...:     lst = []
  ...:     for i in range(x):
  ...:         lst.append(np.arange(i))
  ...:     return lst
  ...:

In [4]: foo(4)
Out[4]: [array([], dtype=int64), array([0]), array([0, 1]), array([0, 1, 2])]

2.6.2.5.1. List Reflection

In nopython mode, Numba does not operate on Python objects. list are compiled into an internal representation. Any list arguments must be converted into this representation on the way in to nopython mode and their contained elements must be restored in the original Python objects via a process called reflection. Reflection is required to maintain the same semantics as found in regular Python code. However, the reflection process can be expensive for large lists and it is not supported for lists that contain reflected data types. Users cannot use list-of-list as an argument because of this limitation.

Note

When passing a list into a JIT-compiled function, any modifications made to the list will not be visible to the Python interpreter until the function returns. (A limitation of the reflection process.)

Warning

List sorting currently uses a quicksort algorithm, which has different performance characterics than the algorithm used by Python.

2.6.2.5.2. List comprehension

Numba supports list comprehension. For example:

In [1]: from numba import njit

In [2]: @njit
  ...: def foo(x):
  ...:     return [[i for i in range(n)] for n in range(x)]
  ...:

In [3]: foo(3)
Out[3]: [[], [0], [0, 1]]

Note

Prior to version 0.39.0, Numba did not support the creation of nested lists.

Numba also supports “array comprehension” that is a list comprehension followed immediately by a call to numpy.array(). The following is an example that produces a 2D Numpy array:

from numba import jit
import numpy as np

@jit(nopython=True)
def f(n):
  return np.array([ [ x * y for x in range(n) ] for y in range(n) ])

In this case, Numba is able to optimize the program to allocate and initialize the result array directly without allocating intermediate list objects. Therefore, the nesting of list comprehension here is not a problem since a multi-dimensional array is being created here instead of a nested list.

Additionally, Numba supports parallel array comphension when combined with the parallel option on CPUs.

2.6.2.6. set

All methods and operations on sets are supported in JIT-compiled functions.

Sets must be strictly homogeneous: Numba will reject any set containing objects of different types, even if the types are compatible (for example, {1, 2.5} is rejected as it contains a int and a float).

Note

When passing a set into a JIT-compiled function, any modifications made to the set will not be visible to the Python interpreter until the function returns.

2.6.2.7. dict

Warning

numba.typed.Dict is an experimental feature. The API may change in the future releases.

Numba does not directly support the Python dict because it is an untyped container that can have any Python types as members. To generate efficient machine code, Numba needs the keys and the values of the dictionary to have fixed types, declared in advance. To achieve this, Numba has a typed dictionary, numba.typed.Dict, for which the user must explicitly declare the key-type and the value-type using the Dict.empty() constructor method. This typed dictionary has the same API as the Python dict, it implements the collections.MutableMapping interface and is usable in both interpreted Python code and JIT-compiled Numba functions. Because the typed dictionary stores keys and values in Numba’s native, unboxed data layout, passing a Numba dictionary into nopython mode has very low overhead. However, this means that using a typed dictionary from the Python interpreter is slower than a regular dictionary because Numba has to box and unbox key and value objects when getting or setting items.

An important difference of the typed dictionary in comparison to Python’s dict is that implicit casting occurs when a key or value is stored. As a result the setitem operation may fail should the type-casting fail.

Note

A numba.typed.Dict cannot yet be constructed with Dict(), the Dict.empty(key_type, value_type) class method must be used to construct a typed dictionary instead.

It should be noted that the Numba typed dictionary is implemented using the same algorithm as the CPython 3.7 dictionary. As a consequence, the typed dictionary is ordered and has the same collision resolution as the CPython implementation.

Further to the above in relation to type specification, there are limitations placed on the types that can be used as keys and/or values in the typed dictionary, most notably the Numba Set and List types are currently unsupported. Acceptable key/value types include but are not limited to: unicode strings, arrays, scalars, tuples. It is expected that these limitations will be relaxed as Numba continues to improve.

Here’s an example of creating a numba.typed.Dict instance from interpreted code and using the dictionary in jit code:

from ex_typed_dict_from_cpython of examples/dict_usage.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import numpy as np
from numba import njit
from numba import types
from numba.typed import Dict

# The Dict.empty() constructs a typed dictionary.
# The key and value typed must be explicitly declared.
d = Dict.empty(
    key_type=types.unicode_type,
    value_type=types.float64[:],
)

# The typed-dict can be used from the interpreter.
d['posx'] = np.asarray([1, 0.5, 2], dtype='f8')
d['posy'] = np.asarray([1.5, 3.5, 2], dtype='f8')
d['velx'] = np.asarray([0.5, 0, 0.7], dtype='f8')
d['vely'] = np.asarray([0.2, -0.2, 0.1], dtype='f8')

# Here's a function that expects a typed-dict as the argument
@njit
def move(d):
    # inplace operations on the arrays
    d['posx'] += d['velx']
    d['posy'] += d['vely']

print('posx: ', d['posx'])  # Out: posx:  [1.  0.5 2. ]
print('posy: ', d['posy'])  # Out: posy:  [1.5 3.5 2. ]

# Call move(d) to inplace update the arrays in the typed-dict.
move(d)

print('posx: ', d['posx'])  # Out: posx:  [1.5 0.5 2.7]
print('posy: ', d['posy'])  # Out: posy:  [1.7 3.3 2.1]

Here’s an example of creating a numba.typed.Dict instance from jit code and using the dictionary in interpreted code:

from ex_typed_dict_njit of examples/dict_usage.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import numpy as np
from numba import njit
from numba import types
from numba.typed import Dict

# Make array type.  Type-expression is not supported in jit functions.
float_array = types.float64[:]

@njit
def foo():
    # Make dictionary
    d = Dict.empty(
        key_type=types.unicode_type,
        value_type=float_array,
    )
    # Fill the dictionary
    d["posx"] = np.arange(3).astype(np.float64)
    d["posy"] = np.arange(3, 6).astype(np.float64)
    return d

d = foo()
# Print the dictionary
print(d)  # Out: {posx: [0. 1. 2.], posy: [3. 4. 5.]}

2.6.2.8. None

The None value is supported for identity testing (when using an optional type).

2.6.2.9. bytes, bytearray, memoryview

The bytearray type and, on Python 3, the bytes type support indexing, iteration and retrieving the len().

The memoryview type supports indexing, slicing, iteration, retrieving the len(), and also the following attributes:

2.6.3. Built-in functions

The following built-in functions are supported:

  • abs()
  • bool
  • complex
  • divmod()
  • enumerate()
  • float
  • hash() (see Hashing below)
  • int: only the one-argument form
  • iter(): only the one-argument form
  • len()
  • min()
  • max()
  • next(): only the one-argument form
  • print(): only numbers and strings; no file or sep argument
  • range: semantics are similar to those of Python 3 even in Python 2: a range object is returned instead of an array of values. The only permitted use of range is as a callable function (cannot pass range as an argument to a jitted function or return a range from a jitted function).
  • round()
  • sorted(): the key argument is not supported
  • type(): only the one-argument form, and only on some types (e.g. numbers and named tuples)
  • zip()

2.6.3.1. Hashing

The hash() built-in is supported and produces hash values for all supported hashable types with the following Python version specific behavior:

Under Python 3, hash values computed by Numba will exactly match those computed in CPython under the condition that the sys.hash_info.algorithm is siphash24 (default).

Under Python 2, hash values computed by Numba will follow the behavior described for Python 3 with the sys.hash_info.algorithm emulated as siphash24. No attempt is made to replicate Python 2 hashing behavior.

The PYTHONHASHSEED environment variable influences the hashing behavior in precisely the manner described in the CPython documentation.

2.6.4. Standard library modules

2.6.4.1. array

Limited support for the array.array type is provided through the buffer protocol. Indexing, iteration and taking the len() is supported. All type codes are supported except for "u".

2.6.4.3. collections

Named tuple classes, as returned by collections.namedtuple(), are supported in the same way regular tuples are supported. Attribute access and named parameters in the constructor are also supported.

Creating a named tuple class inside Numba code is not supported; the class must be created at the global level.

2.6.4.4. ctypes

Numba is able to call ctypes-declared functions with the following argument and return types:

2.6.4.5. enum

Both enum.Enum and enum.IntEnum subclasses are supported.

2.6.4.8. functools

The functools.reduce() function is supported but the initializer argument is required.

2.6.4.9. random

Numba supports top-level functions from the random module, but does not allow you to create individual Random instances. A Mersenne-Twister generator is used, with a dedicated internal state. It is initialized at startup with entropy drawn from the operating system.

Note

Calling random.seed() from non-Numba code (or from object mode code) will seed the Python random generator, not the Numba random generator.

Note

Since version 0.28.0, the generator is thread-safe and fork-safe. Each thread and each process will produce independent streams of random numbers.

See also

Numba also supports most additional distributions from the Numpy random module.

2.6.4.10. heapq

The following functions from the heapq module are supported:

Note: the heap must be seeded with at least one value to allow its type to be inferred; heap items are assumed to be homogeneous in type.

2.6.5. Third-party modules

2.6.5.1. cffi

Similarly to ctypes, Numba is able to call into cffi-declared external functions, using the following C types and any derived pointer types:

  • char
  • short
  • int
  • long
  • long long
  • unsigned char
  • unsigned short
  • unsigned int
  • unsigned long
  • unsigned long long
  • int8_t
  • uint8_t
  • int16_t
  • uint16_t
  • int32_t
  • uint32_t
  • int64_t
  • uint64_t
  • float
  • double
  • ssize_t
  • size_t
  • void

The from_buffer() method of cffi.FFI and CompiledFFI objects is supported for passing Numpy arrays and other buffer-like objects. Only contiguous arguments are accepted. The argument to from_buffer() is converted to a raw pointer of the appropriate C type (for example a double * for a float64 array).

Additional type mappings for the conversion from a buffer to the appropriate C type may be registered with Numba. This may include struct types, though it is only permitted to call functions that accept pointers to structs - passing a struct by value is unsupported. For registering a mapping, use:

numba.cffi_support.register_type(cffi_type, numba_type)

Out-of-line cffi modules must be registered with Numba prior to the use of any of their functions from within Numba-compiled functions:

numba.cffi_support.register_module(mod)

Register the cffi out-of-line module mod with Numba.

Inline cffi modules require no registration.