PythonConcurrency17 min read

The blocking call inside an async def

The most damaging bug in Python backends: one synchronous call in a coroutine stalls every other request on the loop. How to find and fix it.

Day 72 of the 488-day pathway. Published in full — nothing held back.

The blocking call in an async def — the single most damaging Python backend bug

The sentence to own: a blocking call inside async def does not make one request slow. It stops every request that worker is serving — because while your code runs, the loop is not calling select(), so nothing can be noticed as ready.

Day 014 showed you the browser freeze. Day 016 built the selectors loop. Day 048 showed await is a yield point. Today those three become one production incident, and its fix.

Part 1 — Seeing it

import asyncio, time

async def handler(n):
    time.sleep(1)                 # ⚠️⚠️    BLOCKING
    return n

async def main():
    t0 = time.perf_counter()
    await asyncio.gather(*(handler(i) for i in range(10)))
    print(time.perf_counter() - t0)      #    10.0 seconds — fully SEQUENTIAL

async def handler(n):
    await asyncio.sleep(1)        #   non-blocking
    # ⇒    1.0 second. Same code shape. Ten times the throughput.

   WHY — and you can now derive this rather than remember it:
    the loop is a while-loop that spends its life inside select() (Day 070, step 3).
    `time.sleep(1)` never yields ⇒    the loop never regains control ⇒ select() is
    never called ⇒    no socket can be observed as ready ⇒ NOTHING ELSE RUNS.

  ⇒    IT IS NOT A SLOWDOWN PROPORTIONAL TO THE WORK. IT IS A FULL STOP FOR EVERYONE.
       Ten unrelated users each waited a second longer because one handler slept.

The severity is what people underestimate. A slow synchronous endpoint in a threaded server makes that request slow. The same code in an async worker makes every concurrent request slow — so the blast radius is your whole worker, not one user.

#    HOW TO CATCH IT — turn this on in development and in your test suite:
asyncio.run(main(), debug=True)
# ⇒    "Executing <Task ... handler() at app.py:12> took 1.003 seconds"
#    ⇒   it NAMES THE COROUTINE AND THE LINE. This is the whole diagnostic.
loop.slow_callback_duration = 0.05      #   default 0.1s — lower it to catch smaller offenders

Part 2 — The catalogue — what is secretly blocking

   THE OBVIOUS ONES:
    time.sleep()                 ⇒ await asyncio.sleep()
    requests.get()               ⇒   httpx.AsyncClient / aiohttp
    psycopg2 / pymysql / redis   ⇒   asyncpg, SQLAlchemy 2 async, redis.asyncio
    subprocess.run()             ⇒ asyncio.create_subprocess_exec()

   THE ONES THAT HIDE — and these are where real incidents come from:
    ·    FILE I/O. open(), read(), write() ARE BLOCKING. There is no async file I/O on
         Linux in the stdlib; aiofiles is a thread pool wearing async syntax.
    ·    DNS. socket.getaddrinfo() BLOCKS — so even an "async" client can block on the
         FIRST connection to a new host, or when a DNS server is slow.
    ·    CPU work that does not look like it: json.dumps on a 50 MB payload, a big
         regex, Pillow, pandas, a comprehension over a million rows.
    ·    PASSWORD HASHING. bcrypt/argon2 are DELIBERATELY slow (Day 015) — ~100 ms of
         pure CPU,    on the event loop, per login. A login endpoint can throttle your
         entire worker.
    ·   logging to a slow or synchronous handler (a network log shipper, a full disk)
    ·   importing a module inside a handler (first call reads from disk)
    ·   any C extension that does not release the GIL and takes real time

The bcrypt one is the best interview example, because it is correct code doing exactly what it should — 100 ms of deliberate CPU — in exactly the wrong place. Ten logins per second and your worker is permanently saturated, while the CPU graph looks entirely reasonable.

The DNS one is the sneakiest, because everything else in the request is async and the profile shows the time inside your "async" HTTP client.

Part 3 — The fixes

#    1. USE AN ASYNC LIBRARY — always the first choice.
async with httpx.AsyncClient() as client:
    r = await client.get(url, timeout=5)

#    2. BLOCKING I/O YOU CANNOT REPLACE ⇒ push it to a THREAD:
data = await asyncio.to_thread(legacy_sync_call, arg)     #   3.9+
#   the thread blocks; the loop does not. The GIL is released during its I/O (Day 065).

#    3. CPU-BOUND WORK ⇒ push it to a PROCESS:
loop = asyncio.get_running_loop()
result = await loop.run_in_executor(process_pool, cpu_heavy, arg)
# ⚠️    NOT a thread — the GIL means a thread would still contend with the loop's thread
#    for bytecode execution, so the loop still stutters (Day 065's convoy effect).

#   4. TRULY HEAVY OR LONG WORK ⇒ get it out of the request entirely:
await queue.enqueue(job)          #    Celery / ARQ / a queue — return 202 (Day 010)

   THE DECISION, IN FOUR LINES:
    async library exists?              ⇒    use it
    blocking I/O, no async version?    ⇒    asyncio.to_thread
    CPU-bound, milliseconds?           ⇒    run_in_executor(PROCESS pool)
    CPU-bound, seconds, or unbounded?  ⇒    a background queue, and 202 Accepted

⚠ asyncio.to_thread uses the default executor, which is unbounded-ish and shared. Under load it will happily create many threads. For anything hot, create your own ThreadPoolExecutor with an explicit max_workers and pass it to run_in_executor (Day 069).

⚠ A process pool cannot be created lazily inside a request — ProcessPoolExecutor startup is expensive and forking from a threaded async process is unsafe (Day 068). Create it once at startup, in the lifespan handler.

Part 4 — FastAPI: the part that is genuinely misunderstood

@app.get("/a")
def sync_route():                 #    a PLAIN def — FastAPI runs it in a THREAD POOL
    time.sleep(1)                 #   SAFE. The loop is untouched.

@app.get("/b")
async def async_route():          #    an async def — runs ON THE EVENT LOOP
    time.sleep(1)                 # ⚠️⚠️    FREEZES EVERY REQUEST THIS WORKER IS SERVING

   THE RULE, AND IT IS THE OPPOSITE OF WHAT PEOPLE ASSUME:
       A SYNCHRONOUS `def` ROUTE IS SAFER THAN A BADLY-WRITTEN `async def` ROUTE.
    ⇒ FastAPI inspects your function (  Day 042's iscoroutinefunction) and:
         `def`       ⇒   runs it in a threadpool ⇒ blocking is contained to one thread
         `async def` ⇒    runs it on the loop ⇒ blocking stops everything
    ⇒    SO: IF YOU ARE NOT SURE EVERY CALL INSIDE IS ASYNC, USE `def`.
         You lose some efficiency. You do not lose the service.

This applies to dependencies too — a def dependency runs in the threadpool, an async def one runs on the loop. ⚠ And to middleware, and to lifespan: a blocking call at startup delays the whole application's readiness.

⚠ The threadpool is bounded (Starlette's default is 40). Forty concurrent slow def routes and the forty-first waits — so def is a safety net, not a scaling strategy.

   THE REVIEW CHECKLIST — five questions for any `async def`:
   □ 1. Does every I/O call have an `await` in front of it?
   □ 2.    Is there an `await` at all? An `async def` with no `await` is a plain
          blocking function running in the worst possible place.
   □ 3. Any `open()`, `json.dumps` on something large, hashing, image work, `re` on a big string?
   □ 4. Is any library here synchronous? (`requests`, `psycopg2`, `boto3`, `redis`)
   □ 5. Does every network call have a timeout? (Day 060)

Question 2 is the fastest code review in Python: search the function for await. If an async def contains none, it is doing something synchronously on the event loop, and the only question is how long it takes.

Common mistakes

Mistake	Correction
`time.sleep` / `requests` in `async def`	Freezes every concurrent request.
Thinking it is "just slower"	It is a full stop, and the blast radius is the worker.
Forgetting file I/O blocks	`open`/`read` are blocking. `to_thread` or `aiofiles`.
Forgetting DNS blocks	`getaddrinfo` blocks even inside an async client.
Password hashing on the loop	~100 ms of CPU per login, by design. `to_thread`.
`json.dumps` of a large payload on the loop	CPU work that does not look like it.
A thread pool for CPU work	The GIL — use a process pool.
Creating a process pool per request	Expensive, and unsafe from a threaded process.
Unbounded `to_thread` under load	Use your own bounded executor.
`async def` when unsure	Use `def` — FastAPI runs it in a threadpool.
Treating `def` routes as a scaling strategy	The threadpool is bounded (~40).
Not running with `debug=True`	It names the coroutine and line that blocked.

Interview questions

Q: What happens if you call time.sleep(1) inside an async def?

Every request that worker is serving stops for a second. The event loop is a single thread that spends its life inside select, and time.sleep never yields — so the loop never regains control, select is never called, and no socket can be observed as ready. It isn't a slowdown proportional to the work; it's a full stop, and the blast radius is the whole worker rather than one user. Ten unrelated users each waited an extra second because one handler slept.

Q: What blocks that people don't expect?

File I/O — open and read are blocking, and there's no real async file I/O on Linux in the stdlib, so aiofiles is a thread pool wearing async syntax. DNS resolution: getaddrinfo blocks, so even a fully async HTTP client can block on the first connection to a new host, and the profile misleadingly points inside the async client. And CPU work that doesn't look like CPU work — json.dumps on a large payload, a big regex, image processing. My favourite example is password hashing: bcrypt is deliberately slow, around a hundred milliseconds of pure CPU by design, so a login endpoint doing it on the loop saturates the worker at about ten logins a second while the CPU graph looks completely reasonable.

Q: How do you fix it?

Four options in order. Use an async library if one exists — that's always first. If it's blocking I/O with no async version, asyncio.to_thread, because the thread blocks and the loop doesn't, and the GIL is released during the I/O anyway. If it's CPU-bound and takes milliseconds, run_in_executor with a process pool — not a thread pool, because the GIL means a thread would still contend with the loop's thread for bytecode and the loop would stutter. And if it's CPU-bound and takes seconds, it doesn't belong in the request at all: put it on a queue and return 202.

Q: In FastAPI, is a def route or an async def route safer?

def, which surprises people. FastAPI inspects the function and runs a plain def in a threadpool, so blocking inside it is contained to one thread; an async def runs directly on the loop, so blocking inside it stops everything. So a synchronous route is safer than a badly-written async one, and the rule I'd give a team is: if you aren't certain every call inside is async, use def. You lose some efficiency and you don't lose the service. The caveat is that the threadpool is bounded — Starlette defaults to forty — so it's a safety net rather than a scaling strategy.

Q: How do you find these in an existing codebase?

Run with asyncio.run(debug=True), including in the test suite — it prints "Executing <Task ...> took 1.003 seconds" and names the coroutine and line, which is usually the entire diagnosis. Lower slow_callback_duration to catch smaller offenders. And the fastest static check is to search each async def for the word await: if a coroutine contains none, it's doing something synchronously in the worst possible place, and the only question is how long it takes.

Q: Why does this bug survive code review and testing?

Because it doesn't look wrong and it doesn't fail. The code runs, the tests pass — tests usually hit one endpoint at a time, so there's no concurrency to reveal it — and in development with one user the latency is fine. It only appears under concurrent load, as unrelated endpoints becoming slow together, which points investigators at the database or the network rather than at one handler. That combination is why it's the most damaging bug in Python backends rather than merely a common one.

Mini task

Run the ten-handler experiment with time.sleep and asyncio.sleep. Record 10 s and 1 s.
Turn on debug=True and reproduce the warning. Read the coroutine name and line it gives you.
Build two FastAPI routes — one def with time.sleep(1), one async def with the same — and hit each with 10 concurrent clients. Compare total time. (This is the whole day.)
Hash a password with bcrypt inside an async def and measure the loop's stall. Then move it to asyncio.to_thread.
json.dumps a 50 MB structure on the loop and measure the stall.
Read a large file with open() inside a handler while another handler tries to respond. Watch the second one wait.
Convert a requests call to httpx.AsyncClient and confirm concurrency returns.
Run CPU work through a thread pool and then a process pool from the loop. Measure the loop's responsiveness in both — this is why it must be processes.
Fire 100 concurrent requests at a bounded def route and find where the 41st waits.
Grep your own code for async def blocks containing no await.

Exit questions

Answer aloud, no notes.

What exactly happens when you block inside an async def? Explain via select().
Why is the blast radius the worker rather than one request?
Name four things that block but do not look like it.
Why is password hashing the best example?
Why does DNS make an "async" client block?
Give the four-line fix decision.
Why must CPU work go to a process pool rather than a thread pool?
What is the hazard with the default to_thread executor?
In FastAPI, which is safer — def or async def? Why?
What is the limit of that safety net?
Give the five-question review checklist.
Why does this bug survive review, testing and development?

Articulation drill

Record two minutes: "Your async service is slow under load. Every endpoint, not just one. Where do you look?"

Name the hypothesis first, because the symptom is diagnostic: when every endpoint degrades together, something is blocking the event loop — one handler is holding the single thread, so no socket can be observed as ready and every connection waits regardless of what it asked for.

Then the diagnostic: run with asyncio.run(debug=True), which prints the coroutine and line of any callback exceeding the threshold. And the fastest static check: search each async def for await — one with none is doing something synchronously in the worst possible place.

Then the usual suspects, ordered by how often they hide: a synchronous library that slipped in — requests, psycopg2, boto3; file I/O, which is always blocking; DNS resolution, which blocks inside an otherwise-async client; and CPU work that does not look like it — json.dumps on a large payload, or password hashing, which is a hundred milliseconds of deliberate CPU per login.

Then close on why it is worth a whole day: "the reason this is the most damaging bug in Python backends is that it does not look wrong and it does not fail. The code runs, tests pass — because tests hit one endpoint at a time and there is no concurrency to expose it — and in development with one user the latency is fine. It only appears under concurrent load, as unrelated endpoints slowing together, which sends people to look at the database. And the fix is often one word: await."

** The concurrency block is complete — Days 064–072.**

The five things this block establishes: measure CPU-bound versus I/O-bound before choosing anything · the GIL protects the interpreter, not your program · shared mutable state is the problem and threads only reveal it · processes give real cores and you pay per byte that crosses · and an event loop buys enormous concurrency for waiting and exactly zero parallelism for computing, which is why the deployment shape is processes containing event loops.

Next: the runtime and packaging block (073–077) — CPython internals and dis, imports and circular imports, uv and pyproject.toml, the stdlib worth knowing, and the Stage 1 capstone.

Previous: Day 071 · Tomorrow: Day 073 — CPython internals: bytecode, dis, the frame stack, and where the time actually goes