Error "OperationalError: (duckdb.duckdb.TransactionException) TransactionContext Error: cannot start a transaction within a transaction"

[mriedmann]

Currently, when running FastAPI in dev-mode with (or without in-memory DuckDB), the following exception is flooding our logs (see details)

The dashboard works, but it appears that all concurrent requests are failing:

I tried quite a lot:

• Using SQLite as a backend does not work because of multithreading violations
• Using file-based vs. in-memory DuckDB makes no difference
• Using different SQLAlchemy pools (by changing the lib-code => no real solution)
  • SingleThreadedPool => No success at all
  • QueuedPool => Works only using file-based DuckDB (in-memory fails; see details). Then it works but slow
  • NullPool => Works also only if file-based (so no in-memory) but faster

So as a quick-fix it would be enough to make the pools configurable. We could work without fastapi dev --reload for now, only using file-based DuckDB.

It might be related to this change: #24

It would be wonderful to get this to work without error flooding. This project exactly does what we need, but we can't pollute our logs with that many errors.

Thank you for providing it!

Details

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1102, in _begin_impl
    self.engine.dialect.do_begin(self.connection)
  File "/home/****/.venv/lib/python3.10/site-packages/duckdb_engine/__init__.py", line 344, in do_begin
    connection.begin()
duckdb.duckdb.TransactionException: TransactionContext Error: cannot start a transaction within a transaction

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/****/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/****/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 176, in __call__
    with recv_stream, send_stream, collapse_excgroups():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_utils.py", line 82, in collapse_excgroups
    raise exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 178, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi_radar/middleware.py", line 54, in dispatch
    return await call_next(request)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 156, in call_next
    raise app_exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 141, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi_radar/api.py", line 466, in get_stats
    .one()
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2808, in one
    return self._iter().one()  # type: ignore
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2857, in _iter
    result: Union[ScalarResult[_T], Result[_T]] = self.session.execute(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2351, in execute
    return self._execute_internal(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2239, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2108, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
  File "<string>", line 2, in _connection_for_bind
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 1240, in _connection_for_bind
    transaction = conn.begin()
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 856, in begin
    self._transaction = RootTransaction(self)
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2692, in __init__
    self._connection_begin_impl()
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2710, in _connection_begin_impl
    self.connection._begin_impl(self)
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1104, in _begin_impl
    self._handle_dbapi_exception(e, None, None, None, None)
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2363, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1102, in _begin_impl
    self.engine.dialect.do_begin(self.connection)
  File "/home/****/.venv/lib/python3.10/site-packages/duckdb_engine/__init__.py", line 344, in do_begin
    connection.begin()
sqlalchemy.exc.OperationalError: (duckdb.duckdb.TransactionException) TransactionContext Error: cannot start a transaction within a transaction
(Background on this error at: https://sqlalche.me/e/20/e3q8)

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute
    cursor.execute(statement, parameters)
  File "/home/****/.venv/lib/python3.10/site-packages/duckdb_engine/__init__.py", line 150, in execute
    self.__c.execute(statement, parameters)
duckdb.duckdb.CatalogException: Catalog Error: Table with name radar_requests does not exist!
Did you mean "pg_sequences"?
LINE 2: FROM radar_requests 
             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/****/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/****/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 176, in __call__
    with recv_stream, send_stream, collapse_excgroups():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_utils.py", line 82, in collapse_excgroups
    raise exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 178, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi_radar/middleware.py", line 54, in dispatch
    return await call_next(request)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 156, in call_next
    raise app_exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 141, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/****/.venv/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/****/.venv/lib/python3.10/site-packages/fastapi_radar/api.py", line 466, in get_stats
    .one()
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2808, in one
    return self._iter().one()  # type: ignore
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2857, in _iter
    result: Union[ScalarResult[_T], Result[_T]] = self.session.execute(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2351, in execute
    return self._execute_internal(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 527, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2363, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
  File "/home/****/.venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute
    cursor.execute(statement, parameters)
  File "/home/****/.venv/lib/python3.10/site-packages/duckdb_engine/__init__.py", line 150, in execute
    self.__c.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (duckdb.duckdb.CatalogException) Catalog Error: Table with name radar_requests does not exist!
Did you mean "pg_sequences"?
LINE 2: FROM radar_requests 
             ^
[SQL: SELECT count(*) AS total_requests, avg(radar_requests.duration_ms) AS avg_response_time 
FROM radar_requests 
WHERE radar_requests.created_at >= $1]
[parameters: (datetime.datetime(2026, 1, 14, 16, 5, 21, 357717, tzinfo=datetime.timezone.utc),)]
(Background on this error at: https://sqlalche.me/e/20/f405)

[qinsehm1128]

是这样的,因为duckdb是单线程的,所以在提交中如果不用我的那个方案,重复开关链接进行查询,会导致多begin的问题,核心是duckdb只支持唯一的begin!不支持多个begin的,这个确实无法解决,如果接受不了这个方案,可以使用sqlite,没有任何问题的

[qinsehm1128]

如果可以的话，请你提供下后端报错日志,我这边空闲下来会进行优化的

[xelland042]

Hello, same problem

import uvicorn

from fastapi import FastAPI
from fastapi_radar import Radar

from contextlib import asynccontextmanager

from core.config import settings
from core.database import db_helper
from api import router as main_router

@asynccontextmanager
async def lifespan(app: FastAPI):
# before app starts
radar.create_tables()
yield
# before app finishes
await db_helper.dispose()

main_app = FastAPI(
lifespan=lifespan,
debug=True,
)

main_app.include_router(main_router)

radar = Radar(
main_app,
db_engine=db_helper.engine.sync_engine,
db_path="radar.duckdb",
retention_hours=24,
include_in_schema=False,
)

if name == "main":
uvicorn.run(
"main:main_app",
host=settings.run.host,
port=settings.run.port,
reload=True,
)

sqlalchemy.exc.OperationalError: (duckdb.duckdb.TransactionException) TransactionContext Error: cannot start a transaction within a transaction
(Background on this error at: https://sqlalche.me/e/20/e3q8)

How did you hix it?

[mriedmann]

是这样的,因为duckdb是单线程的,所以在提交中如果不用我的那个方案,重复开关链接进行查询,会导致多begin的问题,核心是duckdb只支持唯一的begin!不支持多个begin的,这个确实无法解决,如果接受不了这个方案,可以使用sqlite,没有任何问题的

Thx for your response. Your solution looks perfectly fine. Is there any chance to get your PR merged into the main project? @lxdklp @doganarif