issues with threads and example dataset

[bunop]

Dear all,

i'm trying to use threads with the example dataset described in https://palamaralab.github.io/software/threads/data/: first of all there's a problem in the example_data.map file, this is its content:

1       1       0       0

1       2       19.5    15000000

However threads version v0.2.1 and later requires to specify column names to in .map file (which accordingly your documentation need to be a SHAPEIT file):

$ threads infer --pgen example_data.pgen --map example_data.map --demography Ne10000.demo --ou
t example.threads                                                                                                                                
2025-11-25 17:36:21 INFO     Starting Threads-infer with the following parameters:                                                               
2025-11-25 17:36:21 INFO       pgen:                  example_data.pgen
...
2025-11-25 17:36:21 INFO     Using recombination rates from example_data.map
Traceback (most recent call last):
  File "/home/paolo/.conda/envs/threads/bin/threads", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 1462, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 1383, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 1850, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 1246, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 814, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/Projects/PalamaraLab-threads/src/threads_arg/__main__.py", line 62, in infer
    threads_infer(**kwargs)
  File "/home/paolo/Projects/PalamaraLab-threads/src/threads_arg/infer.py", line 198, in threads_infer
    genetic_positions, physical_positions = make_recombination_from_map_and_pgen(map, pgen, chrom)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/Projects/PalamaraLab-threads/src/threads_arg/utils.py", line 74, in make_recombination_from_map_and_pgen
    phys_pos, cm_pos, _chrom = read_map_file(map_file, expected_chrom)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/Projects/PalamaraLab-threads/src/threads_arg/utils.py", line 36, in read_map_file
    cm_pos = maps.cM.values.astype(np.float64)
             ^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/pandas/core/generic.py", line 6318, in __getattr__
    return object.__getattribute__(self, name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'DataFrame' object has no attribute 'cM'

Second, --map option seems to be mandatory in src/threads_arg/__main__.py: however if I don't known this information, should --recombination_rate option should be used as default value?

@main.command()
@click.option("--pgen", required=True, help="Path to input genotypes in pgen format")
@click.option("--map", required=True, help="Path to genotype map in SHAPEIT format")
@click.option("--recombination_rate", default=1.3e-8, type=float, help="Genome-wide recombination rate. Ignored if a map is passed")
@click.option("--demography", required=True, help="Path to input genotype")

Third: there's a issue in src/threads_arg/infer.py: the function make_constant_recombination_from_pgen(pgen, recombination_rate, chrom) is called with 3 parameters while it expects only 2:

Traceback (most recent call last):
  File "/home/paolo/.conda/envs/threads/bin/threads", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 1462, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 1383, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 1850, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 1246, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/.conda/envs/threads/lib/python3.12/site-packages/click/core.py", line 814, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/paolo/Projects/PalamaraLab-threads/src/threads_arg/__main__.py", line 62, in infer
    threads_infer(**kwargs)
  File "/home/paolo/Projects/PalamaraLab-threads/src/threads_arg/infer.py", line 201, in threads_infer
    genetic_positions, physical_positions = make_constant_recombination_from_pgen(pgen, recombination_rate, chrom)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: make_constant_recombination_from_pgen() takes 2 positional arguments but 3 were given

I will open a PR with some hotfixes: could you check please? how I need to change the example_data.map to have a working example?

many thanks in advance

[ArniFreyrG]

Hey Paolo, thanks for catching all of these.

The updated map format has three named columns: pos, chr, and cM. For example:

pos     chr     cM
0   20      0
1000000   20      26

(see also https://github.com/odelaneau/shapeit4/tree/master/maps) for human maps)

We had updated the maps for the test data but neglected the example data, sorry about that.

The other two issues are fully addressed by your PR, thanks for that

[ArniFreyrG]

Closing this: constant recombination rate fixed with #118 and I opened up #119 and #120 for pgen/region verification and example data updates