Skip to content

fastfec: Trailing commas on rows with empty trailing fields #14

Description

@asg017

Note

Much of this ticket was composed with Claude

libfec's CSV writer emits trailing commas for empty fields at the end of records. FastFEC strips trailing empty fields, producing no trailing comma. Both are valid CSV, but the outputs won't byte-match.

This affects header.csv (where report_id, report_number, comment are often empty) and data rows where trailing fields are empty (common in itemization schedules which have many optional fields at the end).

Affected Filing

Filing ID: 1935668 (237KB, Chevron PAC F3XN filing)

Reproduction

libfec fastfec ~/.cache/libfec/cache/1935668.fec /tmp/out_libfec
fastfec --no-stdin ~/.cache/libfec/cache/1935668.fec /tmp/out_fastfec

Observe the Difference

# header.csv trailing commas:
diff /tmp/out_libfec/1935668/header.csv /tmp/out_fastfec/1935668/header.csv
# 2c2
# < HDR,FEC,8.5,FECFile,8.5,,0,
# ---
# > HDR,FEC,8.5,FECFile,8.5,,0

# Itemization rows - show trailing commas:
tail -1 /tmp/out_libfec/1935668/SA11AI.csv | rev | cut -c1-5 | rev
# ,,,,,   <-- trailing commas
tail -1 /tmp/out_fastfec/1935668/SA11AI.csv | rev | cut -c1-20 | rev
# See Below   <-- no trailing commas

# Count affected rows across a filing:
diff \
  <(sed 's/,*$//' /tmp/out_libfec/1935668/SA11AI.csv) \
  <(sed 's/,*$//' /tmp/out_fastfec/1935668/SA11AI.csv) \
  | grep -c '^[<>]'
# 0  (no differences after stripping trailing commas)

Suggested Fix

The csv::Writer in Rust writes all fields including empty trailing ones. To match FastFEC, either:

  1. Post-process each record to trim trailing empty fields before writing
  2. Or accept the difference as cosmetic (both are valid CSV)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions