Scribd Scraper & PDF Generator

A lightweight, asynchronous Python tool to scrape Scribd documents and assemble them into a single, high-quality PDF.

Features

• No Account or Browser Automation Required: Fetches public embed manifests and page assets directly over HTTP.
• Bot Detection Bypass: Uses browser TLS impersonation via curl_cffi to avoid Cloudflare/bot challenges.
• Asynchronous Concurrent Downloads: Downloads page scans concurrently with polite rate-limiting via semaphores.
• Accurate Pagination: Automatically detects intentionally blank divider pages and generates matching blank canvas pages to maintain exact document pagination.
• High-Quality PDF Compilation: Combines images and blank pages into a unified PDF using Pillow.
• Interactive CLI: Real-time progress bars and status reporting powered by rich.

---

Prerequisites

• Python 3.10+ (tested on Python 3.11)
• Python packages:
  • curl_cffi (HTTP client with browser TLS impersonation)
  • pillow (image processing and PDF generation)
  • rich (CLI terminal formatting and progress bar)
  • pytest (optional, for running tests)

Installation

If setting up in a new environment, install the required dependencies:

pip install curl_cffi pillow rich pytest

---

Usage

1. Basic Usage

Scrape a Scribd document by providing its full URL or numeric document ID:

python main.py https://www.scribd.com/document/730511888/Iswara-owner-manual

If no URL is provided, it defaults to the target document configured in main.py.

2. Specify Output File

Save the resulting PDF to a custom filename or folder:

python main.py https://www.scribd.com/document/730511888/Iswara-owner-manual -o output/Iswara_Manual.pdf

3. Adjust Concurrency

Increase or decrease concurrent download threads (default is 5):

python main.py https://www.scribd.com/document/730511888/Iswara-owner-manual -c 10

4. Clean Cache After Compilation

By default, downloaded images are cached in an output .cache_<id> directory for faster resume. To automatically remove the cache folder after PDF compilation:

python main.py https://www.scribd.com/document/730511888/Iswara-owner-manual --clean-cache

---

CLI Options

Argument	Flag	Default	Description
url	(positional)	Default URL in main.py	Scribd document URL or numeric ID
--output	-o	output/<title>.pdf	Output PDF file path or directory
--concurrency	-c	5	Number of simultaneous downloads
--clean-cache		False	Purge temporary image cache after PDF generation

---

Project Structure

ScribdScraper/
├── main.py            # CLI entry point, argument parsing, orchestration
├── resolver.py        # URL parsing, embed HTML fetching, JSONP manifest parsing
├── downloader.py      # Async image downloader & blank page generator
├── pdf_builder.py     # Multi-page PDF assembly using Pillow
├── models.py          # Data classes (PageInfo, DocumentMetadata)
└── tests/
    └── test_scraper.py # Unit tests for models, URL extraction, and PDF generation

---

Running Tests

Execute the automated test suite using pytest:

python -m pytest -v tests/