Binder Error: Unique file handle conflict – since v1.5.0

[pSpitzner]

What happens?

I have a little helper to clone a duckdb but only for select schemas (gist).

The relevant bit of code is this:

    with duckdb.connect(str(source_path), read_only=True) as source_conn:

        available_schemas = [
            row[0]
            for row in source_conn.execute(
                "SELECT schema_name FROM information_schema.schemata"
            ).fetchall()
        ]
        valid_schemas = [s for s in schemas_to_copy if s in available_schemas]

        with duckdb.connect(str(target_path)) as target_conn:

            for schema in valid_schemas:
                tables = [
                    row[0]
                    for row in source_conn.execute(
                        "SELECT table_name FROM information_schema.tables WHERE table_schema = ?",
                        [schema],
                    ).fetchall()
                ]

                target_conn.execute(f"CREATE SCHEMA IF NOT EXISTS {schema}")
                for table in tables:
                    target_conn.execute(f"ATTACH '{source_path}' AS source_db")
                    target_conn.execute(
                        f"CREATE TABLE {schema}.{table} AS SELECT * FROM source_db.{schema}.{table}"
                    )
                    target_conn.execute("DETACH source_db")

Since v1.5.0, i am encountering a locking error:

Failed to copy schemas Binder Error: Unique file handle conflict: Cannot attach "source_db" - the database file "/Users/paul/demo/demo.duckdb" is already attached by database "demo"

I thought this was related to #19171 but that got fixed in 1.4.2

To Reproduce

Requires typer.

To run the gist:

python minimize_duckdb.py -v -i source.duckdb -o mini.duckdb -s desired_schema

OS:

docker container, debian, also tested on macos and windows

DuckDB Package Version:

1.5.0

Python Version:

3.12

Full Name:

Paul Spitzner

Affiliation:

linkFISH Consulting

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have not tested with any build

Did you include all relevant data sets for reproducing the issue?

Yes

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration to reproduce the issue?

• Yes, I have

[evertlammerts]

Thanks for reporting this! This is actually intended (stricter) behavior introduced in DuckDB v1.5.0 via duckdb/duckdb#18857, which added process-wide file handle tracking to prevent concurrent writes to the same database file (which could lead to data corruption).

In your script, this line opens source.duckdb in read-only mode:

with duckdb.connect(str(source_path), read_only=True) as source_conn:

And then from the target connection, this tries to attach the same file with the default access mode (which is read-write for local files):

target_conn.execute(f"ATTACH '{source_path}' AS source_db")

That's a read-only + read-write conflict on the same file, which is now correctly prevented.

The fix is to attach in read-only mode explicitly:

target_conn.execute(f"ATTACH '{source_path}' AS source_db (READ_ONLY)")

This worked in v1.4.x only because the cross-connection file handle tracking didn't exist yet — the underlying conflict was always there, it just wasn't enforced.

[pSpitzner]

Ah, good catch, thanks for the clarification!