7.2. A Map of the Numba Repository

The Numba repository is quite large, and due to age has functionality spread around many locations. To help orient developers, this document will try to summarize where different categories of functionality can be found.

Note

It is likely that the organization of the code base will change in the future to improve organization. Follow issue #3807 for more details.

7.2.1. Support Files

7.2.1.1. Build and Packaging

7.2.1.2. Continuous Integration

7.2.1.3. Documentation / Examples

  • LICENSE - License for Numba
  • LICENSES.third-party - License for third party code vendored into Numba
  • README.rst - README for repo, also uploaded to PyPI
  • CONTRIBUTING.md - Documentation on how to contribute to project (out of date, should be updated to point to Sphinx docs)
  • AUTHORS - List of Github users who have contributed PRs (out of date)
  • CHANGE_LOG - History of Numba releases, also directly embedded into Sphinx documentation
  • docs/ - Documentation source
  • docs/_templates/ - Directory for templates (to override defaults with Sphinx theme)
  • docs/Makefile - Used to build Sphinx docs with make
  • docs/source - ReST source for Numba documentation
  • docs/_static/ - Static CSS and image assets for Numba docs
  • docs/gh-pages.py - Utility script to update Numba docs (stored as gh-pages)
  • docs/make.bat - Not used (remove?)
  • examples/ - Example scripts demonstrating numba (re/move to numba-examples repo?)
  • examples/notebooks/ - Example notebooks (re/move to numba-examples repo?)
  • benchmarks/ - Benchmark scripts (re/move to numba-examples repo?)
  • tutorials/ - Tutorial notebooks (definitely out of date, should remove and direct to external tutorials)
  • numba/scripts/generate_lower_listing.py - Dump all registered implementations decorated with @lower* for reference documentation. Currently misses implementations from the higher level extension API.

7.2.2. Numba Source Code

Numba ships with both the source code and tests in one package.

  • numba/ - all of the source code and tests

7.2.2.1. Public API

These define aspects of the public Numba interface.

7.2.2.2. Dispatching

7.2.2.3. Compiler Pipeline

7.2.2.4. Type Management

7.2.2.5. Compiled Extensions

Numba uses a small amount of compiled C/C++ code for core functionality, like dispatching and type matching where performance matters, and it is more convenient to encapsulate direct interaction with CPython APIs.

7.2.2.6. Misc Support

7.2.2.7. Core Python Data Types

7.2.2.8. Math

  • numba/_random.c - Reimplementation of NumPy / CPython random number generator
  • numba/_lapack.c - Wrappers for calling BLAS and LAPACK functions (requires SciPy)

7.2.2.9. ParallelAccelerator

Code transformation passes that extract parallelizable code from a function and convert it into multithreaded gufunc calls.

7.2.2.10. Debugging Support

7.2.2.11. Type Signatures (CPU)

Some (usually older) Numba supported functionality separates the declaration of allowed type signatures from the definition of implementations. This package contains registries of type signatures that must be matched during type inference.

7.2.2.12. Target Implementations (CPU)

Implementations of Python / NumPy functions and some data models. These modules are responsible for generating LLVM IR during lowering. Note that some of these modules do not have counterparts in the typing package because newer Numba extension APIs (like overload) allow typing and implementation to be specified together.

7.2.2.13. Ufunc Compiler and Runtime

7.2.2.14. Unit Tests (CPU)

CPU unit tests (GPU target unit tests listed in later sections

7.2.2.15. Command Line Utilities

7.2.2.16. CUDA GPU Target

Note that the CUDA target does reuse some parts of the CPU target.

7.2.2.17. ROCm GPU Target

Note that the ROCm target does reuse some parts of the CPU target, and duplicates some code from CUDA target. A future refactoring could pull out the common subset of CUDA and ROCm. An older version of this target was based on the HSA API, so “hsa” appears in many places.