Introduce a thread number reallocation table

[titouanc]

Some applications or libraries can spawn many short-lived threads. Because the size of the thread tracking table is fixed (to 1024 elements), the available slots could quickly get exhausted, and gil_load will simply print gil_load warning: too many threads, not all will be tracked.

To work around this problem, we can detect when a thread dies, and mark its thread number as reusable. In such case, as long as there are not more than 1024 concurrent threads (running at the same time), they can all be tracked.

Example application:

import threading
from time import sleep

import gil_load

THREADS_TO_SPAWN = 10_000

x = 1


def thread_function():
    """Thread function, expected to run for ~1s if no contention"""
    global x
    n = 100
    for i in range(n):
        # Let's create some GIL contention
        x = x ** 0.5
        x *= 2
        sleep(0.9/n)


gil_load.init()
gil_load.start()

threads = []
for i in range(THREADS_TO_SPAWN):
    if i % 1000 == 0:
        print(f"Spawning threads [{i // 1000}k/{THREADS_TO_SPAWN // 1000}k]...")
    if i % 10000 == 0:
        stats = gil_load.get()
        print(gil_load.format(stats))

    t = threading.Thread(target=thread_function, name=f"shortlived-{i}")
    threads.append(t)
    t.start()
    sleep(0.01)

print(f"Spawned {THREADS_TO_SPAWN} threads")

for i, t in enumerate(threads):
    t.join()

print("Joined all threads")
print(f"x = {x}")

gil_load.stop()
stats = gil_load.get()
print(gil_load.format(stats))

[chrisjbillington]

This is a great addition!

I think I understand the mechanism. By pushing and popping thread numbers to the reallocation table ring buffer, you ensure arbitrary gaps anywhere in the array of threads being tracked can be re-used, and everything is constant-time lookups. Very efficient.

I'll do a bit of testing, but am pretty much happy to merge this when you are happy with it.

[chrisjbillington]

Ah I see why this is a WIP, we'll need a mechanism to clear the stats that are collected in gil_load.pyx when a thread slot is reallocated to a new thread. Not sure if anything else is missing.

[titouanc]

Thank you @chrisjbillington for your first comments !

I just pushed an additional commit with a mechanism to reset the counters when a thread has terminated. Could you please have a look at it and tell me if this looks correct to you ?

I think I understand the mechanism. By pushing and popping thread numbers to the reallocation table ring buffer, you ensure arbitrary gaps anywhere in the array of threads being tracked can be re-used, and everything is constant-time lookups.

Yes indeed, and 15 days after opening the PR, I can now see that this wasn't evident at first. Would you like me to add some comments to explain this in the code ?

[titouanc]

Hello again @chrisjbillington !

Just wanted to check if you had the opportunity to have a look at this PR over the last weeks ?