[PERF]: Avoid redundant HTTP HEAD requests in cudf-polars by including known file size in SourceInfo

[TomAugspurger]

#22739 adds a feature to libcudf to specify a known file size, which lets kvikio skip one or two HTTP requests.

To use this in cudf-polars, we'd need to first list the size of the objects (or get them from polars, if/when it provides them) and pass that in when building the SourceInfo. This applies to both read_parquet_metadata and read_parquet.

109ad43 shows how it's done.

Blocked by #22739 and #22700.

[TomAugspurger]

The snippet shared at 109ad43 has a flaw. We have a serial loop over each path in paths:

    for path in paths:
        if path.startswith(("s3://", "https://", "http://")):
            # this should be the one and only HEAD request we perform.
            with kvikio.RemoteFile.open_s3_url(path) as remote_file:
                size = remote_file.nbytes()

At scale-100, that leads to ~1s of HEAD requests for one of the tables (lineitem probably).

So even though this itself is executing on a worker thread, since we parallelize things at the Scan node level, it's not enough.

We want to avoid over-parallelization though. We don't want each node to itself spawn thread pools to do the work. We'd ideally flatten the nested work to do and churn through all that at once.