No GIL,
Real Gains

Local like SQLite. Analytical like ClickHouse.

chDB is the ClickHouse engine as a pip install, running inside your Python process.

pip install chdb

import chdb
chdb.query(
  "SELECT count() FROM file('hits.parquet')")

+ = in your process

Two places where the engine touches Python

Eight engine threads. One gate into Python. Seven wait.

A string column is one Python object per cell. Reading a cell takes the lock.

How Python has lived with the GIL

Strategy

Who does it

What it costs

Release the GIL in C loops

NumPy, SciPy · PyTorch, TensorFlow ops

Ends the moment a Python object is touched; per-element callbacks re-take the lock

Move work to another process

multiprocessing, joblib · Dask, Ray · DataLoader num_workers · Gunicorn workers

Copies across by pickle, a model loaded per worker, startup time

Compile Python away

Cython nogil · Numba nogil=True · Rust via PyO3

Only code you rewrite; cannot call back into Python

Threads only for waiting

asyncio · ThreadPoolExecutor · aiohttp, requests

Zero gain for CPU-bound Python

Fewer, bigger calls

NumPy vectorize · Polars · Pandas vectorized ops, not Series.map

Only when the work vectorizes

Subinterpreters PEP 684

interpreters module, 3.12+

One GIL per interpreter; isolated state, no shared objects

All of the above, from C++

chDB (before this port)

Worked until Python was in the middle of the query: per-row UDFs and PyObject strings.

Release the GIL for the whole query: the C++ engine runs on every core while Python waits (that is why a pure-SQL query is 0.161 s on both builds). Python UDFs ran in a separate process. Strings were read one value at a time under the lock.

Most work has a workaround. Two shapes don't.

One agent turn: six Python steps, one lock

So the GIL shapes how Python builds agents

Removing the GIL: 29 years of attempts

Guido's rule (2007): single-threaded code must not get slower.

1996

Greg Stein

Worked. Single thread 2× slower.

AP

2009

Antoine Pitrou

New GIL: fairer, still there.

LH

2016

Larry Hastings

Gilectomy. Atomic refcounts too slow.

2017–2023

Eric Snow

Per-interpreter GIL. Not a removal.

2021–2025

Sam Gross

nogil → PEP 703 → 3.14t

Why nogil passed: no big idea, four small ones

Result: ~1–8% single-thread cost. Guido's rule, met.

One line, one promise: nothing here needs the GIL

PYBIND11_MODULE(_chdb, m, py::mod_gil_not_used())   // pybind11 ≥ 2.13

Touching an object used to be safe. Not anymore.

Shared state: every "safe because GIL" spot got a real lock

3.14t: a gate per thread. Eight in Python at once.

Same code. py::gil_scoped_acquire on every worker thread: on stock Python one shared lock, on 3.14t per-thread bookkeeping.

Same UDF, 8 threads: 0.85 s → 0.15 s

Strings: stop asking Python for a copy

if (PyUnicode_IS_COMPACT_ASCII(obj)) {               // logs, ids, keys
    data = PyUnicode_1BYTE_DATA(obj);                // pointer, no copy
    len  = PyUnicode_GET_LENGTH(obj);
} else data = PyUnicode_AsUTF8AndSize(obj, &len);  // everything else

Same scan, same 16 threads: 2.6× faster

5M object-dtype strings, one PyObject per cell · 16 engine threads · CPython 3.14.6 vs 3.14.6t

Or skip Python objects entirely

Background: since Pandas 3.0 the default string dtype is Arrow-backed when PyArrow is installed (PDEP-14). Other dtypes are still NumPy; the move to Arrow is not finished.

219 s → 20 s. Not the lock: zero-copy.

Works on stock Python today.

Pandas 3.0 with PyArrow gives you Arrow-backed string columns by default. Other dtypes are still NumPy.

Scorecard: a lane per thread

One agent turn on 3.14t: six steps, no lock

A Pandas chain becomes one SQL query

import chdb.datastore as pd
df = pd.read_parquet("events.parquet")
top = (df[df.status == 200]
         .groupby("user_id").amount.sum()
         .sort_values(ascending=False)
         .head(10))

SELECT user_id, sum(amount) AS amount
FROM file('events.parquet')
WHERE status = 200
GROUP BY user_id
ORDER BY amount DESC
LIMIT 10

How it runs: lazy chain → compiler → segments

Measured: 10M rows, 16 everyday ops. Faster on 14 of 16; bulk value-replace stays with Pandas

Where the ecosystem is on 3.14t

One undeclared import re-locks the whole process

$ python3.14t -c "import yourstack"
RuntimeWarning: The global interpreter lock (GIL) has been enabled
  to load module 'X', which has not declared that it can run safely
  without the GIL.

Try it yourself

uv python install 3.14t && uv venv -p 3.14t

python -c "import sys, yourstack; print(sys._is_gil_enabled())"

@chdb.func([INT64], INT64)
def slow(x): return sum(range(x % 500))
chdb.query("SELECT sum(slow(number)) FROM numbers_mt(2e6) "
           "SETTINGS max_threads=8")

The GIL was a shortcut from the single-core era.
Today it costs more than it saves.

3.14t removes it. The bugs it was hiding were always ours.