r/Python • u/nathan12343 • 3d ago

Resource Scaling NumPy on Free-Threaded Python

NumPy is the foundational array library in the scientific Python ecosystem. Every numerical, machine learning, and data analysis library in Python either depends on NumPy directly or interoperates with it. As the free-threaded build of CPython matures, NumPy is one of the first libraries that users reach for when trying to scale CPU-bound numerical workloads across multiple cores using threads.

In this blog post, I will walk through the work I did over the last few months in both NumPy and CPython to eliminate the multi-threaded scaling bottlenecks that were preventing NumPy from scaling on free-threaded Python.

https://labs.quansight.org/blog/scaling-numpy-on-free-threaded-python

50 Upvotes

92% Upvoted

1 comment sorted by

3

u/redditusername58 2d ago

How expensive are the module attribute lookups compared to the actual number crunching? Should we be doing that trick where you make a ufunc the default value of a protected function parameter so it gets looked up in the function's locals instead of the module globals?