fix(scraper): extract real artist slugs and parse current uploader markup - #7
Merged
Merged
Conversation
…rkup Tapping an artist on the video page led to a model list without the current video — usually a 404 rendered as an error screen. Two defects: 1. The artist regex targeted the old markup (class="item btn_link" with the name span directly inside). The site now renders vote-chip pills (class="item btn_link video_meta_pill" with a wrap_image div before the span), so artist extraction never matched and the chip disappeared entirely. 2. Navigation derived the model slug from the display name (toLowerCase + dashes), which 404s for names like "OpenNSFW (VA)" whose real site slug is "opennsfw". parseVideoDetail now collects every model pill into an artists[] (name + slug taken from the site's own href), which the post page renders as one chip per artist. The sidebar's "Top Artists" links (class="item", no btn_link) are excluded, and each name is bound to its own anchor so adjacent pills can't bleed into each other. The uploader regex is tightened the same way (name bound to the member anchor, avatar alt first, plain text fallback for avatar-less members) instead of lazily scanning for the next alt attribute in the whole document. Artist chips promote a known role suffix from the model's display name to the label prefix — "OpenNSFW (VA)" renders as "VA: OpenNSFW", "HentAudio (Audio)" as "Audio: HentAudio"; plain names keep the "Artist: Name" prefix. Unknown parentheticals stay part of the name.
💯 Test Coverage
😎 Tests Results
👀 Tests Details • (52%)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tapping an artist on the video page led to a model list without the current video — usually a 404 rendered as an error screen. Two defects:
parseVideoDetail now collects every model pill into an artists[] (name + slug taken from the site's own href), which the post page renders as one chip per artist. The sidebar's "Top Artists" links (class="item", no btn_link) are excluded, and each name is bound to its own anchor so adjacent pills can't bleed into each other. The uploader regex is tightened the same way (name bound to the member anchor, avatar alt first, plain text fallback for avatar-less members) instead of lazily scanning for the next alt attribute in the whole document.
Artist chips promote a known role suffix from the model's display name to the label prefix — "OpenNSFW (VA)" renders as "VA: OpenNSFW", "HentAudio (Audio)" as "Audio: HentAudio"; plain names keep the "Artist: Name" prefix. Unknown parentheticals stay part of the name.