Skip to content

feat(copilot): add wordpress loader to extract#186

Merged
jacobdadams merged 5 commits intomainfrom
wordpress-loader
Mar 31, 2026
Merged

feat(copilot): add wordpress loader to extract#186
jacobdadams merged 5 commits intomainfrom
wordpress-loader

Conversation

@jacobdadams
Copy link
Copy Markdown
Member

Adding a wordpress loader to extract.py to load posts and/or media items from public Wordpress REST API endpoints.

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.03%. Comparing base (f87f4c5) to head (4a4474c).
⚠️ Report is 138 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #186      +/-   ##
==========================================
+ Coverage   94.52%   96.03%   +1.50%     
==========================================
  Files           7        7              
  Lines        1133     1361     +228     
  Branches      148      155       +7     
==========================================
+ Hits         1071     1307     +236     
+ Misses         52       44       -8     
  Partials       10       10              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a WordPress REST API loader to palletjack.extract so callers can retrieve posts (or other endpoint records) into pandas DataFrames and fetch/parse media item metadata into typed models.

Changes:

  • Introduces WordpressRestLoader with automatic pagination via X-WP-TotalPages, optional ACF field flattening, and configurable timeout/User-Agent.
  • Adds WordpressMediaItem / WordpressMediaDetails / WordpressMediaSize dataclasses to parse /wp/v2/media responses.
  • Expands tests/test_extract.py with coverage for the new loader and media parsing (plus minor SFTP test formatting changes).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/palletjack/extract.py Adds WordPress REST loader and media-response dataclasses.
tests/test_extract.py Adds tests for WordPress pagination, params, ACF expansion, and media-item parsing.

Comment thread src/palletjack/extract.py Outdated
Comment thread src/palletjack/extract.py Outdated
Comment thread src/palletjack/extract.py Outdated
jacobdadams and others added 2 commits March 26, 2026 17:53
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread src/palletjack/extract.py Outdated
Comment thread src/palletjack/extract.py Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@jacobdadams
Copy link
Copy Markdown
Member Author

@copilot, add test cases to cover the changes you suggested in the last commit.

…and_acf

Agent-Logs-Url: https://github.com/agrc/palletjack/sessions/1564c949-48f9-453a-975a-0768d658066d

Co-authored-by: jacobdadams <38168030+jacobdadams@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 30, 2026

@copilot, add test cases to cover the changes you suggested in the last commit.

Added 5 new test cases in commit 4a4474c covering all the changes from the last commit:

  • test_expand_acf_no_op_on_empty_responseexpand_acf=True on empty endpoint response returns empty DataFrame without raising
  • test_get_from_endpoint_dict_response_treated_as_single_record — dict JSON response is treated as a single row
  • test_get_from_endpoint_unexpected_json_type_raises — non-list/dict JSON raises ValueError with a clear message
  • test_get_from_endpoint_missing_total_pages_header_defaults_to_1 — absent X-WP-TotalPages header defaults to 1 page
  • test_get_from_endpoint_malformed_total_pages_header_defaults_to_1 — non-numeric header value logs a warning and defaults to 1 page

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@jacobdadams jacobdadams requested a review from stdavis March 30, 2026 19:05
@jacobdadams jacobdadams merged commit 105ba02 into main Mar 31, 2026
11 checks passed
@jacobdadams jacobdadams deleted the wordpress-loader branch March 31, 2026 22:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants