PythonConcurrency17 min read

FastAPI: def vs async def in production

When FastAPI runs your endpoint on a threadpool, when it does not, and why a plain def is sometimes the correct answer.

Day 134 of the 488-day pathway. Published in full — nothing held back.

def vs async def in production — the threadpool, the blocking call, and when synchronous is the right answer

The sentence to own: FastAPI runs a plain def route in a threadpool and an async def route on the event loop — so a blocking call costs you one thread in the first case and every connection the worker owns in the second, which is why a synchronous route is safer than a badly written asynchronous one.

Day 072 told you this. Day 106 showed you the mechanism from underneath. Today is the framework behaviour, the numbers, and the decision.

Part 1 — What FastAPI actually does

@app.get("/sync")
def sync_route():                                   #    plain def
    return db.query(...)                             #   blocking — and that is FINE here

@app.get("/async")
async def async_route():                             #    async def
    return await db.execute(...)                     #   must be awaitable all the way down

   THE RULE, AND IT IS ENTIRELY MECHANICAL:
    `def`        ⇒    RUN IN STARLETTE'S THREADPOOL via `run_in_threadpool`.
                     The event loop stays free.   Blocking is expected.
    `async def`  ⇒    RUN DIRECTLY ON THE EVENT LOOP.    ANY BLOCKING CALL
                     FREEZES EVERY CONNECTION THIS WORKER OWNS (Days 072, 106).

  ⇒    SO THE DANGEROUS COMBINATION IS NOT "SYNC CODE". IT IS **SYNC CODE IN
      AN `async def`** — and that is the single most damaging FastAPI mistake,
      because it looks modern and it is worse than the version that looks old.

   THE THREADPOOL IS BOUNDED, AND THE NUMBER MATTERS:
    Starlette's default is   **40 threads** (`anyio.to_thread` capacity).
    ⇒    SO A `def` ROUTE GIVES YOU 40 CONCURRENT REQUESTS PER WORKER, AND THE
         41ST QUEUES.   With four workers that is 160 —    perfectly adequate
         for most services, and NOT the thousands an event loop can hold.
    ⇒   raise it (`anyio.to_thread.current_default_thread_limiter().total_tokens
         = 100`) if your work is genuinely I/O-bound and slow —    but a
         threadpool of 500 is a memory problem (Day 106), not a solution.
    ⇒    AND THE POINT: `def` IS A SAFETY NET, NOT A SCALING STRATEGY. It
         bounds the damage; it does not give you async's concurrency.

⚠ A yield dependency defined with plain def also runs in the threadpool, and one defined with async def runs on the loop — so def get_db() doing blocking SQLAlchemy is safe, and async def get_db() doing the same thing is not. The rule applies to dependencies exactly as it does to routes.

Part 2 — Measuring the catastrophe

import time, asyncio

@app.get("/bad")
async def bad():                                     # ⚠️⚠️ blocking, on the loop
    time.sleep(1)
    return {"ok": True}

@app.get("/good-sync")
def good_sync():                                     #   blocking, in the threadpool
    time.sleep(1)
    return {"ok": True}

@app.get("/good-async")
async def good_async():                              #   non-blocking
    await asyncio.sleep(1)
    return {"ok": True}

$ hey -n 50 -c 50 http://localhost:8000/bad          #   50 concurrent
#    ~50 SECONDS TOTAL — fully serialised. One request at a time, forever.

$ hey -n 50 -c 50 http://localhost:8000/good-sync
#   ~2 seconds — 40 in parallel, then 10 more

$ hey -n 50 -c 50 http://localhost:8000/good-async
#    ~1 second — all 50 concurrently

   RUN THIS. THE NUMBERS ARE THE LESSON, AND THEY ARE MORE PERSUASIVE THAN ANY
    EXPLANATION:
    ·    50× WORSE THAN THE "OLD-FASHIONED" VERSION, from removing one keyword
    ·   and note the failure mode: it is not an error, a warning or a log line.
            IT IS LATENCY — so it shows up as "the API is slow under load", it
         gets diagnosed as needing more workers, and adding workers makes it
         slightly better and hides it further.
    ⇒    HEALTH CHECKS ALSO TIME OUT, because `/healthz` is on the same frozen
         loop ⇒   the orchestrator restarts a worker that was merely busy
         (Day 111).    A blocking call can therefore cause a restart loop.

Part 3 — The library audit

Blocking (do not call in `async def`)	Async equivalent
`requests`	`httpx.AsyncClient`
`psycopg2`, sync SQLAlchemy	`asyncpg`, SQLAlchemy async (Day 137)
`redis` (sync)	`redis.asyncio`
`time.sleep`	`asyncio.sleep`
`open()` / `.read()`	`anyio.open_file` — or accept it: local disk is fast
`boto3`	`aioboto3`, or a threadpool
`bcrypt` / `argon2`	⚠ CPU-bound — see below
`subprocess.run`	`asyncio.create_subprocess_exec`

#    WHEN THERE IS NO ASYNC VERSION — push it off the loop:
from fastapi.concurrency import run_in_threadpool

@app.post("/login")
async def login(payload: LoginIn, db: DB):
    user = await repo.get_by_email(db, payload.email)
    ok = await run_in_threadpool(ph.verify, user.hash, payload.password)   #    ~150ms of CPU
    ...

   PASSWORD HASHING IS THE EXAMPLE WORTH INTERNALISING, BECAUSE IT IS
    DELIBERATELY EXPENSIVE (Day 114): argon2 tuned to 150 ms is 150 ms of CPU.
    ⇒    CALLED DIRECTLY IN AN `async def`, IT FREEZES THE LOOP FOR 150 ms PER
         LOGIN.   Ten logins per second and the worker is 100% stalled.
    ⇒ ⚠️    AND `run_in_threadpool` ONLY HALF-HELPS FOR CPU WORK: the GIL
         (Day 065) means Python-level CPU still contends.   argon2 and bcrypt
         release the GIL in their C extensions, so it genuinely works for THEM
         —    but a pure-Python CPU loop moved to a thread still competes for
         the interpreter.
    ⇒    FOR REAL PYTHON-LEVEL CPU WORK: a process pool, or a queue (Day 068,
         and Celery later in this stage).   Not a thread.

asyncio.to_thread and run_in_threadpool are the same idea; use FastAPI's, because it respects the same capacity limiter as def routes — otherwise you have two unrelated thread budgets and no way to reason about the total.

Part 4 — The decision

   CHOOSE `async def` WHEN:
    ✔   EVERY I/O CALL IN THE PATH IS AWAITABLE — asyncpg, httpx, redis.asyncio
    ✔    the work is I/O-BOUND AND CONCURRENT: fan-out to three services,
         WebSockets, SSE, long polling ⇒   this is where async genuinely wins,
         and the win is large
    ✔ you need thousands of mostly-idle connections (Day 106)

     CHOOSE `def` WHEN:
    ✔    YOUR DRIVER IS SYNCHRONOUS AND YOU ARE NOT REWRITING IT.   Django
         ORM, sync SQLAlchemy, an internal library, a vendor SDK.
    ✔   the work is CPU-ish (hashing, image resizing, parsing)
    ✔    THE TEAM IS NOT CONFIDENT ABOUT ASYNC.   A `def` route with a
         blocking call is CORRECT. An `async def` route with the same call is a
         50× production incident, and code review does not reliably catch it.

  ⇒    THE POSITION WORTH DEFENDING, AND IT IS UNFASHIONABLE: **A CONSISTENTLY
      SYNCHRONOUS SERVICE IS BETTER THAN AN INCONSISTENTLY ASYNCHRONOUS ONE.**
        Mixed codebases are where this bug lives, because one function three
      layers down turns out to be blocking and nothing tells you.

⚠️⚠️    THE WORST OUTCOME IS THE HALF-MIGRATION: `async def` ROUTES CALLING A
     SYNCHRONOUS ORM.   You get async's fragility and none of its concurrency,
     and every route is the Day 072 bug.
     ⇒   if you must migrate:    MIGRATE THE DRIVER FIRST, THE ROUTES LAST.
          Sync routes over a sync driver is a working system; async routes over
          a sync driver is a broken one.
     ⇒   and while mixed, `run_in_threadpool` every sync call in an async path,
          explicitly, with a comment —    so a reviewer can see the ones that
          are not wrapped.

Part 5 — Catching it

#    THE DETECTOR: asyncio's own debug mode, which logs slow callbacks
import asyncio
asyncio.get_event_loop().set_debug(True)
asyncio.get_event_loop().slow_callback_duration = 0.1     #   warn above 100ms
# ⇒    "Executing <Task ...> took 1.005 seconds" — the blocking call, named.

   FOUR WAYS TO FIND A BLOCKING CALL, CHEAPEST FIRST:
   1.    ASYNCIO DEBUG MODE IN STAGING (or `PYTHONASYNCIODEBUG=1`).   It logs
        the coroutine that hogged the loop, with a duration.    Almost nobody
        turns this on, and it finds the bug in minutes.
   2.   `blockbuster` or `flake8-async` —    a linter that flags known blocking
        calls inside `async def` (Day 099's "let the machine catch it").
   3.    `py-spy dump` ON A STALLED WORKER (Day 082): the loop thread's stack
        shows `time.sleep` or a socket read instead of `epoll_wait`.
   4.   A LOAD TEST comparing 1 concurrent request with 50.    If total time
        scales linearly with concurrency, you are serialised.

  ⇒    AND THE REVIEW HABIT (Day 100B): IN ANY `async def`, EVERY I/O CALL
      MUST HAVE AN `await` IN FRONT OF IT OR BE WRAPPED.   A line doing I/O with
      no `await` is a blocking review comment, and it is greppable.

Common mistakes

Mistake	Correction
Sync driver inside `async def`	Freezes every connection the worker owns. 50× slower than `def`.
Making everything `async def` "for speed"	Without async drivers it is strictly worse.
Assuming `def` is the slow option	It is 40 concurrent per worker, and it is safe.
Treating the threadpool as unlimited	40 by default; the 41st queues.
Raising the threadpool to 500	A memory problem, not a fix.
`async def get_db()` with a sync session	The rule applies to dependencies too.
Hashing a password in `async def`	150 ms of loop freeze per login.
Expecting threads to fix CPU-bound Python	The GIL. Use a process or a queue.
`asyncio.to_thread` alongside `def` routes	Two unrelated thread budgets. Use `run_in_threadpool`.
A half-migration	Async fragility, no async benefit. Migrate the driver first.
Never enabling asyncio debug mode	It names the blocking call in minutes.

Interview questions

Q: What's the difference between def and async def in FastAPI?

A plain def route runs in Starlette's threadpool, so the event loop stays free and blocking is expected. An async def route runs directly on the loop, so any blocking call freezes every connection that worker owns. The consequence people get backwards is that the dangerous combination isn't synchronous code — it's synchronous code inside an async def. A def route with a blocking ORM call is correct; the same call in an async def is roughly fifty times slower under concurrency, and it looks more modern.

Q: How big is the threadpool, and what does that mean?

Forty threads by default, so a def route gives about forty concurrent requests per worker and the forty-first queues. With four workers that's 160, which is plenty for most services and nowhere near the thousands an event loop can hold. The framing I'd use is that def is a safety net rather than a scaling strategy: it bounds the damage from blocking code, it doesn't give you async's concurrency. Raising it helps a bit for slow I/O, but a pool of 500 threads is a memory problem rather than a solution.

Q: How would you find a blocking call in an async service?

Asyncio's debug mode with slow_callback_duration — it logs "executing this task took 1.005 seconds" and names the coroutine, and almost nobody turns it on. Then a linter like blockbuster or flake8-async that flags known blocking calls inside async def, so the machine catches it rather than review. py-spy dump on a stalled worker, where the loop thread's stack shows time.sleep or a socket read instead of epoll_wait. And a load test comparing one concurrent request with fifty — if total time scales linearly with concurrency, you're serialised. The symptom in production is pure latency, with no error and no log, which is why it survives so long.

Q: What about password hashing?

It's deliberately expensive — argon2 tuned to 150 milliseconds is 150 milliseconds of CPU — so calling it directly in an async def freezes the loop for that long on every login. Ten logins a second and the worker is fully stalled. I'd wrap it in run_in_threadpool, which genuinely works here because argon2 and bcrypt release the GIL in their C extensions. That distinction matters: for pure-Python CPU work a thread doesn't help, because the GIL means it still contends for the interpreter — that needs a process pool or a queue.

Q: Sync or async for a new service?

Async if every I/O call in the path can be awaited and the work is I/O-bound and concurrent — fan-out to several services, WebSockets, long-lived connections. Sync if the driver is synchronous and I'm not rewriting it, or if the team isn't confident about async. And I'd defend that second point: a consistently synchronous service is better than an inconsistently asynchronous one, because the failure mode of getting it wrong is a fifty-times regression with no error message, and review doesn't reliably catch a blocking call three layers down. The worst outcome is the half-migration — async routes over a sync ORM gives you async's fragility and none of its benefit. Migrate the driver first, the routes last.

Mini task

Build the three endpoints and run the load test. Write down all three numbers.
Repeat with -c 100 and confirm /bad scales linearly while /good-async does not.
Add /healthz and hit it while /bad is under load. Watch it time out.
Enable asyncio debug mode with slow_callback_duration = 0.1 and find the blocking call in the log.
py-spy dump on the stalled worker and read the loop thread's stack (Day 082).
Install blockbuster or flake8-async and run it over your codebase.
Find the threadpool limit at runtime, then raise it to 100 and re-run the /good-sync test.
Call argon2.verify directly in an async def and measure throughput. Then wrap it in run_in_threadpool and compare.
Run a pure-Python CPU loop in run_in_threadpool and confirm it does not scale (Day 065).
Write async def get_db() around a synchronous session and observe the same freeze.
Audit your dependencies against the Part 3 table. List every blocking call in an async path.

Exit questions

Answer aloud, no notes.

Where does a def route run? An async def route?
What is the dangerous combination, and why is it counterintuitive?
How big is the threadpool, and what does that give you?
Why is def a safety net rather than a scaling strategy?
Give the three load-test numbers and explain each.
Why does a blocking call cause health-check failures and restarts?
Does the rule apply to dependencies?
Why is password hashing a special case, and why does the threadpool help there?
Why does a thread not help pure-Python CPU work?
Give four ways to detect a blocking call.
When would you deliberately choose sync?
What is the worst migration order, and what is the right one?

Articulation drill

Record two minutes: "Should your FastAPI routes be async def?"

Answer with the mechanism first, because the choice follows from it: "only if everything in the path can actually be awaited. A plain def route runs in a threadpool, so blocking there is fine and costs one thread. An async def route runs on the event loop, so one blocking call freezes every connection that worker owns."

Then the number, because it's the persuasive part: "I've measured it — fifty concurrent requests against a route that sleeps one second: the async def version with a blocking sleep takes fifty seconds, fully serialised. The plain def version takes two. So removing one keyword made it fifty times worse, and the version that looks old-fashioned is the safe one. And the failure mode is pure latency — no error, no warning, no log line — so it gets diagnosed as needing more workers, and adding workers hides it further."

Then the operational consequence people miss: "it also takes down health checks, because /healthz is on the same frozen loop. So the orchestrator restarts a worker that was merely busy, and a blocking call can turn into a restart loop."

Close with the position: "so I'd go async when every driver is async and the work is genuinely I/O-bound and concurrent — fan-out, WebSockets, long-lived connections, where the win is real. And I'd happily stay synchronous otherwise, because a consistently sync service beats an inconsistently async one. The worst outcome is the half-migration: async routes over a sync ORM gives you async's fragility with none of the benefit. Migrate the driver first, the routes last — and turn on asyncio's debug mode in staging, because it names the blocking call in minutes and almost nobody enables it."

Previous: Day 133 · Tomorrow: Day 135 — SQLAlchemy 2.0: the engine, the session, and the unit of work you have to understand