A lightweight, personal, self-hosted frontend and local caching proxy for pixiv artworks p.ufal.my.id
pixiv python archiver
PHP 46%
Python 29%
JavaScript 13%
CSS 9%
Shell 1%
Dockerfile <1%
<1%

README.md

localpixiv #

localpixiv is a lightweight, personal, self-hosted frontend and local caching proxy for pixiv artworks.

It allows users to browse, search, and view pixiv illustrations locally with fast loading times, offline persistence, customized content filtering, and responsive gallery interfaces.


Features #

  • Local Image Caching: Automatically converts and caches pixiv artworks in modern WebP format for fast delivery and minimal storage usage.
  • On-Demand Responsive Thumbnails: Generates 320px or 640px WebP previews for the homepage, gallery, and viewer recommendations without storing duplicate thumbnail files on the server.
  • Artwork link previews: Fetches page-zero metadata for uncached artwork, serves a small pixiv thumbnail on preview pages, and uses a larger pixiv master image for Open Graph cards.
  • RSS Feed: Selects up to 50 recently archived artworks with image-only entries containing every available artwork page.
  • Inline Library Stats: Shows the total number of archived artworks, disk space used by the cache and database, and the latest archive activity directly on the homepage.
  • Random Discovery: Opens a randomly selected artwork from the local archive on every visit.
  • Admin Access Control: Provides a persistent admin login and an admin-only switch controlling whether visitors may archive uncached artworks.
  • Content Filtering: Server-side gallery exclusions and client-side homepage/recommendation filters for R-18, AI-generated content, and sensitivity levels. These are viewing preferences; they do not restrict access to direct image URLs or RSS.
  • Gallery & Viewer: Paginated grid browsing, exact artist filtering, full-resolution manga/illustration reader, and sidebar recommendations.
  • Instant Autocomplete: Gallery search offers real-time native pixiv tag suggestions with stored English/Japanese translations.
  • Comprehensive CLI Suite: Rich terminal management tool (./localpixiv) to fetch artworks, verify cache integrity, repair webp assets, and purge orphan entries.
  • Cloudflare Worker Proxy Included: Bundled worker script (worker/worker.js) to seamlessly bypass pixiv image hotlinking restrictions.
  • Self-Contained Deployment: Zero external database dependencies; powered by SQLite WAL mode and ready for Docker containerization.

Requirements #

Native / Bare-Metal Requirements #

  • PHP 8.1 or higher with pdo_sqlite and mbstring; PHP-FPM for the supplied Caddy setup. The Docker image also builds GD, but current image conversion and thumbnails use Pillow and libvips.
  • Python 3.10 or higher
  • libvips 8.11 or higher (/usr/bin/vips, commonly provided by the libvips-tools package)
  • Caddy, Nginx, or Apache web server
  • SQLite 3

Python Dependencies #

  • pixivpy3 >= 3.7.2
  • Pillow >= 10.0.0
  • requests >= 2.31.0

Setup Guide #

Before deploying, read the development and deployment notes and known issues. The current bootstrap does not create the full analytics schema, and the bundled Caddyfile does not block private configuration/database paths. An existing working installation may have schema or server protections not supplied by this checkout.

Step 1: Deploy the Image Proxy (Cloudflare Worker) #

pixiv restricts direct image downloads from i.pximg.net by enforcing strict Referer headers. A lightweight Cloudflare Worker script is included in the worker/ directory.

  1. Go to the Cloudflare Dashboard > Workers & Pages > Create Application > Create Worker.
  2. Name your worker (e.g. pixiv-proxy) and deploy it.
  3. Click Edit Code, paste the code from worker/worker.js, and click Deploy.
  4. Your proxy URL will look like: https://pixiv-proxy.yourname.workers.dev/?url=
  5. Save this URL to config/proxy.

Step 2: Obtain your pixiv Refresh Token #

localpixiv uses OAuth authentication via pixivpy3 to access illustration metadata from the pixiv API.

To obtain your refresh token:

  1. Use an established open-source token extractor tool such as pixiv-auth or the browser login PKCE flow.
  2. Once generated, your 64-character refresh token will look like: 1234567890abcdef...
  3. Save this token to config/token.

Step 3: Launch localpixiv #

  1. Clone the repository:

    git clone https://tangled.org/ufal.my.id/localpixiv.git
    cd localpixiv
    
  2. Initialize configuration files:

    mkdir -p config db cache/images
    cp -n config.example/* config/
    
  3. Insert your proxy URL and token into config/proxy and config/token:

    echo "https://your-worker-subdomain.workers.dev/?url=" > config/proxy
    echo "your_pixiv_refresh_token_here" > config/token
    

    Configure config/credentials with an admin username and a PHP-compatible password hash:

    USER=admin
    PASS=$2y$...your bcrypt hash...
    

    Generate the hash with PHP's password_hash() function. Keep config/public set to off until public archiving is intentionally enabled from the homepage Settings panel.

  4. Start the container:

    docker compose up -d
    

    When updating an existing installation after the Docker dependencies change, rebuild the image:

    docker compose up -d --build
    
  5. Open your browser and navigate to:

    http://localhost:8080
    

Option B: Native Bare-Metal Setup #

  1. Clone the repository and navigate to the project root:

    git clone https://tangled.org/ufal.my.id/localpixiv.git
    cd localpixiv
    
  2. Set up Python virtual environment and install dependencies:

    python3 -m venv venv
    ./venv/bin/pip install -r requirements.txt
    

    Install libvips using your operating system's package manager. On Debian or Ubuntu:

    sudo apt install libvips-tools
    

    On Fedora:

    sudo dnf install vips-tools
    
  3. Initialize configuration files from templates:

    mkdir -p config db cache/images
    cp -n config.example/* config/
    
  4. Insert your proxy URL and token into config/proxy and config/token, then configure config/credentials as described in the Docker setup above:

    echo "https://your-worker-subdomain.workers.dev/?url=" > config/proxy
    echo "your_pixiv_refresh_token_here" > config/token
    
  5. Configure web server permissions: Ensure your PHP-FPM user (e.g., www-data, apache, or http) has read and write permissions to db/, cache/, and config/public. The public-archive checkbox must be able to update config/public:

    chmod -R 775 db cache
    chmod 664 config/public
    

    The PHP-FPM user also needs read access to the other configuration files. On Fedora, if PHP-FPM runs as apache while the project belongs to your own user, a narrow ACL can grant only the additional write access required by this setting:

    setfacl -m u:apache:rw config/public
    
  6. Start PHP-FPM separately using your host's service configuration. Adapt Caddyfile so its document root points to this checkout and its PHP-FPM upstream matches that service, then start the web server:

    caddy run --config Caddyfile
    

Responsive Thumbnail Behavior #

The homepage's recently archived cards, the gallery grid, and the viewer recommendation sidebar use the first locally archived page of each artwork as their cover. Instead of sending that full-resolution file to these compact interfaces, gallery.php generates a square WebP thumbnail on demand. The browser chooses between the 320px and 640px variants through srcset, depending on each component's rendered size and the display pixel density.

Generated thumbnails are streamed directly from libvips to the HTTP response and are not written to cache/images or another persistent server directory. They are cached privately by the browser for 24 hours and validated with an ETag afterward. The same thumbnail URL is reused across the homepage, gallery, and recommendations, allowing the browser to reuse its cached response. Opening an artwork still uses the original full-resolution WebP from the local cache.

If the local cover is unavailable, each interface retains its existing fallback to the configured pixiv image proxy.

The downloading screen and the public-archive-unavailable screen use pixiv's square_medium page-zero image through the Worker. Open Graph uses the existing local WebP for archived artwork or pixiv's master1200 JPEG through the Worker for uncached artwork. These previews do not use libvips or create persistent thumbnail files.


RSS Feed #

The homepage menu below Recently Archived includes an RSS link after Settings, Stats, and Random. The feed is available at /rss and selects up to 50 of the most recently archived artworks, skipping entries with no available image URL.

Each entry uses the title format <artwork title> by <artist> - <site title>, links to the local artwork viewer, and uses the archived timestamp as its publication date. Its body contains every page with an available local or source URL and no text. Local images are resized on demand to at most 640px wide without upscaling, preserving their complete aspect ratio. Feed responses are cached privately for five minutes and validated with an ETag. The feed does not apply browser content filters.

Library Statistics #

The homepage includes a Stats toggle immediately after Settings. It loads the total number of archived artworks, the combined disk space occupied by cache/ and db/cache.sqlite, and the latest archive activity directly into the footer card. The activity timestamp uses the visitor's local timezone. Settings and Stats are mutually exclusive, so opening either panel closes the other. The /stats route is the JSON data source used by this on-demand panel rather than a separate page.

The Random link immediately after Stats opens one randomly selected artwork from the local database. It respects the homepage R-18 and AI filters: artwork from a category set to Hide is excluded from the random candidate pool. Sensitive and recommendation-specific rules are not applied. Its /random redirect is temporary and non-cacheable so each visit performs a fresh selection.

Admin & Public Archive Access #

The /admin page accepts the username and password configured in config/credentials. A successful login sets a browser cookie that expires after 30 days; the server does not independently enforce session expiry. Returning to /admin while authenticated displays the active admin name and a Logout button.

Authenticated admins receive an Allow public archive checkbox in the homepage Settings panel. When disabled, anonymous visitors can open complete archived artwork. For an uncached or incomplete artwork, they see its thumbnail, title, and artist with Public archive is not available and a Home button. The server may fetch and publicly cache metadata for this page, but it does not download the artwork or create an archive row. If an uncached ID has no available metadata, the page has HTTP 404 status and limited artwork details. Logged-in admins can continue archiving regardless of the public setting.

While an admin is signed in, each artwork viewer also shows Delete beside the pixiv link. The first click changes the label to Are you sure?; the second click permanently removes that artwork's database records and cached images through delete.py, then returns to the homepage. The action is submitted as an ID-scoped, CSRF-protected POST request and is unavailable to visitors who are not signed in.


CLI Management (./localpixiv) #

The ./localpixiv script provides unified command-line management for downloading and maintaining the local repository:

Command Description
./localpixiv <artwork_id> Fetch and cache one artwork, then insert its metadata through insert.py.
./localpixiv delete <id> Remove one artwork and its cached images after confirmation.
./localpixiv delete <id> -y Remove one artwork without the CLI prompt; used after another trusted confirmation flow.
./localpixiv delete purge Purge all artwork entries and cached media (requires confirmation).
./localpixiv cleanup Scan cache/database inconsistencies and interactively delete selected numbered results.
./localpixiv repair Scan all entries, show repairable artwork, and repair it after confirmation.
./localpixiv repair <id> Inspect and repair one existing database entry after confirmation.
./localpixiv info <id> Report database, cache, page, original-image, and WebP completeness for one ID.
./localpixiv viewreset <id> Reset lifetime, weekly, and daily view-counter data for one artwork.
./localpixiv viewreset purge Reset view-counter data for every artwork after confirmation.

cleanup reports cache-only folders, database-only rows, and incomplete artwork as one numbered list. Its selection prompt accepts a number, comma-separated numbers, an inclusive range such as 2-5, or all, followed by one final confirmation. repair converts an available JPG/PNG first and otherwise downloads the missing page from its stored source URL; it then synchronizes local_photos.


Configuration Reference #

All settings are stored as plain text files in the config/ directory:

File Default Description
config/title localpixiv Website header title and HTML page title.
config/favicon /static/favicon.png template Site-root favicon path (a leading / is added). External URLs are not supported; the referenced default image is not included.
config/footer (Markdown string) Custom footer text rendered with basic markdown support.
config/credentials Placeholder Admin username and PHP password_hash() output in USER=... / PASS=... format.
config/public off Allows anonymous visitors to archive uncached artwork when set to on.
config/proxy (Placeholder URL) Cloudflare Worker reverse proxy URL for pixiv image downloading.
config/site_url Empty Optional public site origin, such as https://pixiv.example.com, for absolute Open Graph URLs behind a reverse proxy. When empty, request host/scheme headers are used.
config/token Placeholder pixiv API OAuth refresh token used by pixivpy3; replace the example value before fetching.

Credits & Acknowledgements #

  • pixiv: Artwork metadata and illustration sources provided via pixiv API.
  • pixivpy (PixivPy API): Python pixiv API client library by Upupming and contributors.
  • Pillow: Python Imaging Library for WebP conversions.
  • libvips: Low-memory, on-demand responsive thumbnail generation.
  • Caddy Server: Modern, secure, and lightweight web server.
  • Development & AI Disclosure: Developed with the assistance of Antigravity CLI powered by Gemini 3.1 Pro and Gemini 3.7 Flash models and ChatGPT desktop powered by GPT 5.6 Sol and GPT-6 Astra model.

License #

This project is licensed under the terms of the MIT License.