OUTDATED DOCUMENTATION

You are viewing archived documentation from the old Numba documentation site. The current documentation is located at https://numba.readthedocs.io.

Notes on Debugging¶

This section describes techniques that can be useful in debugging the compilation and execution of generated code.

Memcheck¶

Memcheck is a memory error detector implemented using Valgrind. It is useful for detecting memory errors in compiled code, particularly out-of-bounds accesses and use-after-free errors. Buggy or miscompiled native code can generate these kinds of errors. The Memcheck documentation explains its usage; here, we discuss only the specifics of using it with Numba.

The Python interpreter and some of the libraries used by Numba can generate false positives with Memcheck - see this section of the manual for more information on why false positives occur. The false positives can make it difficult to determine when an actual error has occurred, so it is helpful to suppress known false positives. This can be done by supplying a suppressions file, which instructs Memcheck to ignore errors that match the suppressions defined in it.

The CPython source distribution includes a suppressions file, in the file Misc/valgrind-python.supp. Using this file prevents a lot of spurious errors generated by Python’s memory allocation implementation. Additionally, the Numba repository includes a suppressions file in contrib/valgrind-numba.supp.

Note

It is important to use the suppressions files from the versions of the Python interpreter and Numba that you are using - these files evolve over time, so non-current versions can fail to suppress some errors, or erroneously suppress actual errors.

To run the Python interpreter under Memcheck with both suppressions files, it is invoked with the following command:

valgrind --tool=memcheck \
         --suppressions=${CPYTHON_SRC_DIR}/Misc/valgrind-python.supp \
         --suppressions=${NUMBA_SRC_DIR}/contrib/valgrind-numba.supp \
         python ${PYTHON_ARGS}

where ${CPYTHON_SRC_DIR} is set to the location of the CPython source distribution, ${NUMBA_SRC_DIR} is the location of the Numba source dir, and ${PYTHON_ARGS} are the arguments to the Python interpreter.

If there are errors, then messages describing them will be printed to standard error. An example of an error is:

==77113==    at 0x24169A: PyLong_FromLong (longobject.c:251)
==77113==    by 0x241881: striter_next (bytesobject.c:3084)
==77113==    by 0x2D3C95: _PyEval_EvalFrameDefault (ceval.c:2809)
==77113==    by 0x21B499: _PyEval_EvalCodeWithName (ceval.c:3930)
==77113==    by 0x26B436: _PyFunction_FastCallKeywords (call.c:433)
==77113==    by 0x2D3605: call_function (ceval.c:4616)
==77113==    by 0x2D3605: _PyEval_EvalFrameDefault (ceval.c:3124)
==77113==    by 0x21B977: _PyEval_EvalCodeWithName (ceval.c:3930)
==77113==    by 0x21C2A4: _PyFunction_FastCallDict (call.c:376)
==77113==    by 0x2D5129: do_call_core (ceval.c:4645)
==77113==    by 0x2D5129: _PyEval_EvalFrameDefault (ceval.c:3191)
==77113==    by 0x21B499: _PyEval_EvalCodeWithName (ceval.c:3930)
==77113==    by 0x26B436: _PyFunction_FastCallKeywords (call.c:433)
==77113==    by 0x2D46DA: call_function (ceval.c:4616)
==77113==    by 0x2D46DA: _PyEval_EvalFrameDefault (ceval.c:3139)
==77113==
==77113== Use of uninitialised value of size 8

The traceback provided only outlines the C call stack, which can make it difficult to determine what the Python interpreter was doing at the time of the error. One can learn more about the state of the stack by looking at the backtrace in the GNU Debugger (GDB). Launch valgrind with an additional argument, --vgdb-error=0 and attach to the process using GDB as instructed by the output. Once an error is encountered, GDB will stop at the error and the stack can be inspected.

GDB does provide support for backtracing through the Python stack, but this requires symbols which may not be easily available in your Python distribution. In this case, it is still possible to determine some information about what was happening in Python, but this depends on examining the backtrace closely. For example, in a backtrace corresponding to the above error, we see items such as:

We can see some of the arguments, in particular the names of the compiled functions, e.g:

_ZN5numba4cuda5tests6cudapy13test_constmem19cuconstRecAlign$247E5ArrayIdLi1E1C7mutable7alignedE5ArrayIdLi1E1C7mutable7alignedE5ArrayIdLi1E1C7mutable7alignedE5ArrayIdLi1E1C7mutable7alignedE5ArrayIdLi1E1C7mutable7alignedE

We can run this through c++filt to see a more human-readable representation:

numba::cuda::tests::cudapy::test_constmem::cuconstRecAlign$247(
  Array<double, 1, C, mutable, aligned>,
  Array<double, 1, C, mutable, aligned>,
  Array<double, 1, C, mutable, aligned>,
  Array<double, 1, C, mutable, aligned>,
  Array<double, 1, C, mutable, aligned>)

which is the fully qualified name of a jitted function and the types with which it was called.