Repository navigation
Remove donor leftovers: twin engines' --mode help, access badge, issue templates - #14
Merged
Merged
Conversation
…lates - selenium/puppeteer: --mode help and docstring described MediaMarkt's "/product/ page with brand, EAN"; now mirror Playwright's listing / product (/listing/<id>) / shop text. - README: access badge "needs a residential exit" -> "needs Scraping Browser API" (README measures residential alone as refused, t=bv). - bug_report.yml: --out grills -> mugs; Farfetch's "96 products" -> the README's 184-row run. - site_changed.yml: JSON-LD-first / lowest_price_30d / 90% options replaced with data-listing-id anchor and per-page-kind floors. - SECURITY.md: Akamai -> DataDome. output_writer.py: MediaMarkt tv-audio URL example -> a taxonomy hub. CONTRIBUTING: "the registration form" generalised. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tern comment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SECURITY.md said the project has no releases or version tags; it has both (gh release list). Supported versions now names the latest release and main, in the wording bbb-scraper and foodpanda-scraper already use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jehrr
force-pushed
the
fix/donor-leftovers
branch
from
September 28, 2026 19:51
8e80775 to
a6e97d2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Leftovers from the repos etsy-scraper was bootstrapped from (mediamarkt, farfetch) that describe another site as if it were this one.
Fixed
selenium_scraper.py,puppeteer_scraper.py--help(user-visible) —--modesaid "listing (default) or product. product reads one /product/ page and adds brand, EAN…", which is MediaMarkt's product mode. All three engines takelisting|product|shopand Etsy's detail page is/listing/<id>. The twins now carry the Playwright engine's help text and module-docstring mode list verbatim.403, t=bv, says "a residential exit was not sufficient on its own here", and names the Scraping Browser API as the path that works. Badge order was already the family order.bug_report.yml— example--out grills→--out mugs; expected placeholder was Farfetch's "96 products, as the README says a category page yields" → the README's 184-row 3-page search run.site_changed.yml— "the parser tries JSON-LD first", anoriginal_price or lowest_price_30doverlay option (no such column; TROUBLESHOOTING says so) and "DOM price confirmation below 90%" →data-listing-idprimary anchor with JSON-LD enriching (8 entries for 64 tiles, fromproduct_parser.py), and the per-page-kindCONFIRMATION_FLOOR.SECURITY.md— "how Akamai or reCAPTCHA behave" → DataDome.output_writer.py— comment's example of a no-listings page was MediaMarkt's/de/category/tv-audio-202.html→ a taxonomy hub (TROUBLESHOOTING's own wording).smoke_test.py— privacy-pattern comment said "MediaMarkt's pages embed…" without naming it a sibling; now "On a sibling site (mediamarkt-scraper)…".CONTRIBUTING.md— "the registration form" → "a registration or login form".Kept, deliberately
diff_runs.py"The sibling repos guard this withsource… mediamarkt.at against mediamarkt.de" andproduct_parser.py"Unlike … the sibling mediamarkt-scraper" / "unlike in farfetch-scraper" — explicit provenance with a lesson.smoke_test.py:detect_page_state("<html><body>MediaMarkt</body></html>")(a deliberately foreign page asserting "built out of none of Etsy's assets is blocked"),is_supported_host("https://www.amazon.de/x")(another marketplace is refused), and the "dash-cents is a MediaMarkt convention, measured absent on Etsy" pin — all intentional.akamaientry in the vendor marker table; captcha_solver farfetch notes; craigslist.gitignorenote; released CHANGELOG sections.Verification
python3 smoke_test.pyexit 0 before and after (same two engine groups skipped locally);ci_checks.py --secret-checkand--help-check --sample-checkexit 0; issue templates parse as YAML.🤖 Generated with Claude Code