Extension attribute table type. Supports ordered (struct) fields, or unordered (hash-based) fields.
Bases: object
Type for extension type attributes.
Create a consistent attribute ordering with the base types.
ordering ∈ { unordered, extending, ... }
Create a final attribute table from a list of attribute (name, type).
Returns whether this extension type needs a tp_dealloc, tp_traverse and tp_clear filled out.
Compiling @autojit extension classes works as follows:
Create an extension Numba/minivect type holding a symtab
Capture attribute types in the symtab in the same was as @jit
Build attribute hash-based vtable, hashing on (attr_name, attr_type).
(attr_name, attr_type) is the only allowed key for that attribute (i.e. this is fixed at compile time (for now). This means consumers will always know the attribute type (and don’t need to specialize on different attribute types).
However, using a hash-based attribute table allows easy implementation of multiple inheritance (virtual inheritance), without complicated C++ dynamic offsets to base objects (see also virtual.py).
- For all methods M with static input types:
- Compile M
- Register M in a list of compiled methods
Build initial hash-based virtual method table from compiled methods
- Create pre-hash values for the signatures
- We use these values to look up methods at runtime
Parametrize the virtual method table to build a final hash function:
- slot_index = (((prehash >> table.r) & self.table.m_f) ^
self.displacements[prehash & self.table.m_g])
See also virtual.py and the following SEPs:
And the following paper to understand the perfect hashing scheme:
Hash and Displace: Efficient Evaluation of Minimal Perfect Hash Functions (1999) by Rasmus Pagn:
Create descriptors that wrap the native attributes
Create an extension type:
- {
hash-based virtual method table (PyCustomSlots_Table **) PyGC_HEAD PyObject_HEAD ... native attributes
}
We precede the object with the table to make this work in a more generic scheme, e.g. where a caller is dealing with an unknown object, and we quickly want to see whether it support such a perfect-hashing virtual method table:
We need to store a PyCustomSlots_Table ** in the object to allow the producer of the table to replace the table with a new table for all live objects (e.g. by adding a specialization for an autojit method).
Bases: numba.exttypes.compileclass.AttributeBuilder
Create a descriptor that accesses the attribute on the ctypes struct. TODO: Use a perfect-hashed attribute table.
Bases: numba.exttypes.compileclass.ExtensionCompiler
Compile @autojit extension classes.
Get base classes for the resulting extension type.
We can try several inheritance schemes. We can choose between:
One unspecialized - general - class
- This must bind specialized methods on each object instance to allow method calls from Python, making object allocation more expensive
- We must take the max() of tp_basicsize to allow enough space for all specializations. However, the specialization universe is not known up front, so we must allocate attributes separately on the heap (or perhaps we must override tp_alloc to use a dynamic size depending on the specialization - but this may conflict with non-numba base classes).
One specialized class per instance
- This allows us to experiment with static layouts for attributes or methods more easily
- It seems more intuitive to back specialized objects with a specialized type
We will go with 2). We could go for a specialization tree as follows:
A/ | A0 | A1
| |B |/ |B0 B1
Which gets us:
issubclass(A_specialized, A) isinstance(A_specialized(), A)
as well as
issubclass(B_specialized, A_specialized)
However, to support this scheme, the unspecialized class A must:
- Be subclassable
- Return specialized object instances when instantiated
- Support unbound method calls
1) requires that A be a class, and then 2) implies that A’s metaclass overrides __call__ or that A implements __new__.
However, since A_specialized subclasses A, A_specialized.__new__ would need to skip A.__new__, which requires numba to insert a __new__ or modify a user’s __new__ method in A_specialized.
The metaclass option seems more feasible:
A_meta.__call__ -> specialized object instance
Users can then override a metaclass in a Python (or numba?) subclass as follows:
- class MyMeta(type(MyNumbaClass)):
- ...
The metaclass can also support indexing:
A_specialized = A[{‘attrib_a’: double}]
Bases: numba.exttypes.compileclass.Filterer
Bases: numba.exttypes.compileclass.MethodWrapperBuilder
Update the extension class with the function wrappers.
Process autojit methods (undecorated methods). Use the fast NumbaSpecializingWrapper cache when for when we’re being called from python. When we need to add a new specialization, autojit_method_compiler is invoked to compile the method.
extclass: the extension type ext_type.py_class: the unspecialized class that was decorated
Bases: object
Function in the unspecialized class that is used for delegation to a method in a specialized class, i.e.
A.method(A(10.0)) -> A(10.0).method()
This method can never be bound, since __new__ always returns specialized instances (so the unspecialized class cannot be instantiated!).
Invoked when a class is decorated with @autojit.
| Parameters: |
|
|---|---|
| Returns: | py_class that returns specialized object instances |
Called to compile a new specialized method. The result should be added to the perfect hash-based vtable.
Create a partial environment to compile specialized versions of the extension class in.
Inovked when calling the wrapped class to compile a specialized new extension type.
Make delegation unbound methods that delegate from the unspecialized class to the specialized class. E.g.
m = A.method m(A(10.0)) # Delegate to A[double].method
Autojit meta class.
When the autojit cache compiles a new class specialization, it invokes it with the constructor arguments. Since A_specialized inherits from A, AutojitMeta.__call__ again tries to specialize the class. We need to override this behaviour and instead instantiate A_specialized through type.__call__ (invoking A_specialized.{__new__,__init__}).
Create an unspecialized class.
Bases: object
Build attribute descriptors for Python-level access.
Cram descriptors into the class dict
Create a descriptor that accesses the attribute from Python space.
Finalize the attribute table (and fix the order if necessary)
Bases: object
Inherit attributes and methods from parent classes:
For attributes and methods ...
- Build a table type
- Copy supertype slots into subclass table type
Inherit attributes and methods from superclasses
Inherit attributes from a parent class. May be called multiple times for multiple bases.
Inherit methods from a parent class. May be called multiple times for multiple bases.
Bases: object
Build extension type from llvm methods and pointers and a populated virtual method table.
Compile extension methods:
Compile all methods, reuse function environments from type inference stage.
∀ methods M sets M.lfunc, M.lfunc_pointer and M.wrapper_func
Finalize (fix) the attribute and method tables.
Get base classes for the resulting extension type.
For jit types, these are simply the bases of the Python class we decorated. For autojit-decorated classes we get a more complicated inheritance scheme (see AutojitExtensionCompiler.get_bases).
Return the metaclass for the specialized extension type.
Initialize:
Inherit attributes and methods
- Also build the vtab and attribute table types
Process class attribute types:
- class Foo(object):
myattr = double
Process method signatures @void(double) etc
Process all method signatures:
Validate that we can build the extension type.
Bases: object
Update the extension class with the function wrappers.
Create symbol table for all attributes of the extension type. These are Variables which are used by the type inferencer and used to type check attribute assignments.
New attribute assignments create new ExtensionAttributeVariable variables in the symtab. These variables update the attribute table during type inference:
class Foo(object):
value1 = double
- def __init__(self, value2):
- self.value2 = int_(value2)
Before type inference of __init__ we have:
symtab = { ‘value1’: Variable(double) }
and after type inference of __init__ we have:
- symtab = {
- ‘value1’: Variable(double), # type is fixed ‘value2’: ExtensionAttributeVariable(int_), # type is inferred
}
Process class attribute types:
@jit class Foo(object):
attr = double
Compile an extension class given the NumbaEnvironment and the Python class that contains the functions that are to be compiled.
Compiling @jit extension classes works as follows:
Create an extension Numba/minivect type holding a symtab
Capture attribute types in the symtab ...
... from the class attributes:
@jit class Foo(object):
attr = double
... from __init__
@jit class Foo(object):
- def __init__(self, attr):
self.attr = double(attr)
Type infer all methods
Compile all extension methods
- Process signatures such as @void(double)
- Infer native attributes through type inference on __init__
- Path the extension type with a native attributes struct
- Infer types for all other methods
- Update the ext_type with a vtab type
- Compile all methods
Create descriptors that wrap the native attributes
Create an extension type:
- {
PyObject_HEAD ... virtual function table (func **) native attributes
}
The virtual function table (vtab) is a ctypes structure set as attribute of the extension types. Objects have a direct pointer for efficiency.
Bases: numba.exttypes.compileclass.AttributeBuilder
Create a descriptor that accesses the attribute on the ctypes struct. This is set by the extension type constructor __new__.
Bases: numba.exttypes.compileclass.ExtensionCompiler
Compile @jit extension classes.
Compile an extension class given the NumbaEnvironment and the Python class that contains the functions that are to be compiled.
Virtual method table types and ordering.
Bases: object
Virtual method table type.
Add a method to the vtab type and verify it with any parent method signatures.
Create a consistent method ordering with the base types.
ordering ∈ { unordered, extending, ... }
Create an empty finalized vtable type
Get the signature for the given method name. Returns ExtMethodType
Return methods in the order they were set in
This module defines ordering schemes for virtual methods and attributes.
If we use hash-based virtual (method/attribute) tables, we don’t care about the ordering. If we’re using a C++ like virtual method/attribute table (like normal Python extension types do for attributes), we need to have a layout compatible with base classes (i.e. we may only add more attributes, but not reorder any existing ones).
Bases: object
Order the table entities according to the given parent tables, i.e. we can only extend existing tables.
Sort parent tables by size
Return table entities in a random order
Handle signatures of methods in @jit and @autojit classes.
Bases: numba.exttypes.signatures.MethodMaker
Bases: numba.exttypes.signatures.MethodMaker
Bases: object
py_func: the python ‘def’ function
Bases: object
Creates Methods from python functions and validates user-declared signatures.
Retrieve the default method signature for the given method if no user-declared signature exists.
Create a method type for the given Method and declared signature
Called when no signature is found for the method
Bases: object
Processes signatures of extension types.
Return [Method] for each decorated method in the class
Update a method signature with the extension type for ‘self’.
Get the Python function the classmethod or staticmethod is wrapping.
In Python2.6 classmethod and staticmethod don’t have the ‘__func__’ attribute.
Verify a method signature.
Returns a Method object and the resolved signature. Returns None if the object isn’t a method.
Simple utilities related to extension types
Returns whether the given class is an unspecialized autojit class
Validate method signatures and inheritance compatiblity.
Bases: numba.exttypes.validators.MethodValidator
Validate a signature against the number of arguments the function expects.
Validate a signature (which is None if not declared by the user) for a method.
Bases: numba.exttypes.validators.ExtTypeValidator
Validate attribute table with static order (non-hash-based).
Bases: numba.exttypes.validators.ExtTypeValidator
Validate attribute types in the table with attribute types in the parent table.
E.g. if attribute ‘foo’ has type ‘double’ in the base class, then it should also have type ‘double’ in the derived class.
Bases: object
Interface for validators that check for compatible inheritance trees.
Validate an extension type with its parents.
Bases: numba.exttypes.validators.MethodValidator
Validate the init method of extension classes.
Bases: numba.exttypes.validators.MethodValidator
Validate the init method for jit functions. Issue a warning when the signature is omitted.
Bases: numba.exttypes.validators.ExtTypeValidator
Validate method table with static order (non-hash-based).
Bases: numba.exttypes.validators.ExtTypeValidator
Validate method signatures in the vtable with method signatures in the parent table.
Bases: object
Interface for method validators
Validate a Method. Raise an exception for user typing errors.
Determine the compatability of this table with its parents given an ordering.AbstractTable and a type compare function ((type1, type2) -> bool).
Extension attribute Variables used for attribute type inference. See also compileclass.build_extension_symtab().
Bases: numba.symtab.Variable
Variable created during type inference for assignments to extension attributes for which we don’t know the type yet.
When the assignment happens, update ext_type.attributedict.
Virtual methods using virtual method tables.
Note that for @jit classes, we do not support multiple inheritance with incompatible base objects. We could use a dynamic offset to base classes, and adjust object pointers for method calls, like in C++:
However, this is quite complicated, and still doesn’t allow dynamic extension for autojit classes. Instead we will use Dag Sverre Seljebotn’s hash-based virtual method tables:
Bases: numba.exttypes.virtual.VTabBuilder
Bases: numba.exttypes.virtual.VTabBuilder
Bases: object
Build virtual method table for quick calling from Numba.
Build a virtual method table. The result will be kept alive on the extension type.
Finalize the method table (and fix the order if necessary)
Wrap the vtable such that users can get a pointer to the underlying data (extension_types.{Static,Dynamic}VtableWrapper).
Build hash-based vtable.
Create ctypes virtual method table.
vtab_type: the vtab struct type (typesystem.struct) method_pointers: a list of method pointers ([int])
Mangle method names for the vtab (ctypes doesn’t handle this)