Column names use `_TODO_DUP` suffix for duplicate columns

[asg017]

Note

Much of this ticket was composed with Claude

Description

FEC cover records (F3X, F3, F3P) contain duplicate column names - certain fields appear twice at different positions (e.g. total_receipts appears as both a summary field and a computed total). libfec disambiguates these by appending _TODO_DUP to the second occurrence:

• col_a_total_receipts (position 24) and col_a_total_receipts_TODO_DUP (position 45)
• col_a_total_disbursements (position 24) and col_a_total_disbursements_TODO_DUP (position 66)
• col_a_total_contributions and col_a_total_contributions_TODO_DUP
• Same pattern for col_b_* variants

FastFEC uses the same column name for both positions, meaning the header row has duplicate column names. This makes FastFEC's output ambiguous when reading by column name rather than position.

6 column pairs affected per F3X filing, 12 total with col_a and col_b variants.

Affected Filing

Filing ID: 1935668 (237KB, Chevron PAC F3XN filing)

Reproduction

libfec fastfec ~/.cache/libfec/cache/1935668.fec /tmp/out_libfec
fastfec --no-stdin ~/.cache/libfec/cache/1935668.fec /tmp/out_fastfec

Observe the Difference

# Show the column name differences in the header row:
diff \
  <(head -1 /tmp/out_libfec/1935668/F3XN.csv | tr ',' '\n') \
  <(head -1 /tmp/out_fastfec/1935668/F3XN.csv | tr ',' '\n')
# 45c45
# < col_a_total_receipts_TODO_DUP
# ---
# > col_a_total_receipts
# 66c66
# < col_a_total_disbursements_TODO_DUP
# ---
# > col_a_total_disbursements
# 68c68
# < col_a_total_contributions_TODO_DUP
# ---
# > col_a_total_contributions
# (+ 3 more col_b variants)

# Count all TODO_DUP columns:
head -1 /tmp/out_libfec/1935668/F3XN.csv | tr ',' '\n' | grep TODO_DUP
# col_a_total_receipts_TODO_DUP
# col_a_total_disbursements_TODO_DUP
# col_a_total_contributions_TODO_DUP
# col_b_total_receipts_TODO_DUP
# col_b_total_disbursements_TODO_DUP
# col_b_total_contributions_TODO_DUP

Decision

Options:

1. Match FastFEC: Use identical column names (accept ambiguity)
2. Better suffix: Replace _TODO_DUP with something cleaner like _2 or _repeat
3. Keep as-is: The _TODO_DUP suffix prevents silent data confusion in column-name-based tools

The _TODO_DUP suffix appears to be defined in fec-parser-macros column mapping files.