misc feature requests

[ofrei]

CLI syntax refinements

• Consider console-command aliases without the .py suffix and with the input
  format as a subcommand:
  • project_payload.py --input-format bfile ... -> project_payload bfile ...
  • prepare_variants.py --input-format bim ... -> prepare_variants bim ...
  • same question for prepare_variants_sharded.py.
• Decide singular/plural spelling before adding aliases. Current repo/docs use
  project_payload.py; the idea was phrased as project_payloads.py /
  project_payloads.
• If implemented, preserve the existing .py entry points as compatibility
  aliases unless a deliberate breaking-release plan says otherwise.

Payload input naming

• Revisit project_payload.py --input ...bim / --input ...pvar. For payload
  projection, a .bim or .pvar path is being used as a PLINK payload prefix
  locator, which reads strangely because the actual payload is the file set
  (.bed/.bim/.fam or .pgen/.pvar/.psam).
• Same issue exists in the underlying apply tools:
  apply_vmap_to_bfile.py and apply_vmap_to_pfile.py.
• Possible direction: expose payload-prefix terminology at the wrapper level
  while keeping compatibility with existing --input / --source-prefix
  flags.

derived_from contract

• Tighten the normative contract for derived_from in .vmap metadata.
• Consider allowing project_payload.py --input to be omitted when the source
  payload path can be resolved from the mapping object's derived_from.
• Sumstats are trickier because payload resolution may require both raw input
  and metadata. Consider whether derived_from should be a copyable single
  string encoding a pair such as input=<path>;metadata=<path>, or whether the
  object metadata should use structured fields.
• Clarify how derived_from propagates through preparation, sharded
  preparation, matching, intersection, and projection.

Combined sumstats prepare/project wrapper

• Consider a wrapper for the common summary-statistics sequence:
  prepare_variants.py followed by project_payload.py.
• Optional --target could constrain between prepare and project:
  prepare raw sumstats -> optionally match/intersect to target -> project
  cleaned or raw sumstats output.
• Define whether this is only a convenience wrapper around existing retained
  artifacts or whether it owns its own retained intermediate namespace,
  --resume, and --force behavior.

[ofrei]

Fixed in https://github.com/precimed/genomatch/releases/tag/v0.4.0

Object metadata row counts

• Add row-count metadata to .vmap and .vtable sidecars so tools can choose
  memory-efficient strategies without scanning every object first.
• Prefer a generic key such as variants_count over domain-specific nsnps;
  size is shorter but less explicit.
• Define whether the count is required for all newly written objects, optional
  for backward compatibility, and validated against actual row count when read
  or only trusted as producer metadata.

intersect_variants output semantics

• Consider changing intersect_variants.py to retain the artifact type of the
  first input:
  • first input .vmap -> intersected .vmap
  • first input .vtable -> intersected .vtable
• Preserve current exact intersection semantics: key by chrom:pos:a1:a2,
  output order follows the first input, and output id comes from the first
  input.
• Preserve bounded-memory behavior. Desired implementation direction:
  load the smallest input object fully, stream the others, and still stream or
  replay the first input as needed to preserve first-input order, IDs, and
  provenance. Use metadata row counts when available.
• This would change current source-of-truth text, which says
  intersect_variants.py emits .vtable.

--only-mapped-target simplification

• Consider removing --only-mapped-target from public apply_vmap_* surfaces.
• Wrapper direction to evaluate:
  • project_payload.py --full-target: call apply_vmap_to_* directly on the
    full target mapping.
  • default mapped-only projection: first call
    intersect_variants.py <vmap> <target-table-like-object> with the .vmap
    first, then call apply_vmap_to_* on that intersected .vmap.
• This depends on intersect_variants.py preserving first-input .vmap
  provenance and IDs.
• Caveat: mapped-only projection is not necessarily equivalent to intersecting
  the prepared source .vmap with the target if that changes row order. If
  default projection should follow target order, mapped-only filtering must
  happen after matching to target, or via a tool that preserves matched
  target-row order while dropping source_index == -1 rows.
• Check whether the simplification is worth the extra retained intermediate,
  changed performance profile, and compatibility cost for direct
  apply_vmap_* users.