Command line interface for accessing the Open Web Index shards and slices.

feat: Phase 1.2 - Add config schema for pre-sign support master

Config Schema Updates: - Added `use_presign` boolean field per source (default: true) - Added `presign_api_url` string field (default: https://dashboard.ows.eu/api) - Backward compatible: sources without fields use direct S3 - Mixed mode supported: some sources with pre-sign, others direct S3 Example Configuration: - docs/examples/warc-cfg-presign-example.json - Shows it4i, lrz with pre-sign enabled - Shows csc with direct S3 (use_presign=false) - Includes fallback credentials for all sources - Documents _comment field for user guidance Testing: - 9 new config schema tests (100% passing) - Tests cover: presign enabled/disabled, mixed mode, backward compat - Tests verify example config is valid JSON - Tests document default value behavior Documentation: - docs/warc-config-presign.md (comprehensive guide) - Covers: schema, authentication, migration, troubleshooting - Shows examples for all modes: presign, direct S3, mixed - Includes .env-rc sourcing instructions - Documents security best practices Key Features: - Gradual migration path (source-by-source) - Fallback to direct S3 if pre-sign fails - CLI parameter or env var for token - Batch URL pre-fetching optimization 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>


+619
3 changed files