OWILIX - Open Web Index CLI #

A command-line tool for managing OpenWebIndex datasets across local and remote repositories.

Quick Start #

One-Line Install (Recommended) #

curl -fsSL https://openwebindex.net/owilix/install.sh | sh

This script:

• Installs uv if not present
• Creates a virtual environment at ~/.venv/owilix
• Installs the latest owilix
• Adds to your PATH
• Checks for config migrations

Update #

OWILIX_UPGRADE=1 curl -fsSL https://openwebindex.net/owilix/install.sh | sh

If your terminal paste inserts \|, remove the backslash so the command contains a normal pipe (|).

If the installer fails with a python-http-irods-client URL/dependency error, use the manual uv install shown below.

Manual Install #

# Install uv if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create a virtual environment and install owilix
uv venv --python 3.11 ~/.venv/owilix
source ~/.venv/owilix/bin/activate
uv pip install \
  "python-http-irods-client @ git+https://opencode.it4i.eu/lexis-platform/data/python-http-irods-client.git@1.1.4" \
  owilix \
  --index-url https://opencode.it4i.eu/api/v4/projects/92/packages/pypi/simple \
  --extra-index-url https://opencode.it4i.eu/api/v4/projects/107/packages/pypi/simple

# Add to PATH (optional)
echo 'export PATH="$HOME/.venv/owilix/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

Verify Installation #

owilix --version
owilix remote doctor
owilix remote ls all

After updates, check if config needs migration: owilix config version

Help and Reporting #

• join us in the mattermost community channel
• Or report issues at the Open Code Gitlab

Documentation #

Full documentation: openwebsearcheu-public.pages.it4i.eu/owi-cli

• Installation

• Configuration

• Migration

• Commands & Usage

• Query Commands

• Development

Release Notes #

For the latest changes and version history, see:

• CHANGELOG.md

Migration from v0.x #

# Check current version
owilix config version

# Migrate config (creates backup)
owilix config migrate

# Or delete and start fresh
rm ~/.owi/owilix.cfg
owilix remote doctor

Key Commands #

# Remote operations
owilix remote ls all              # List all datasets
owilix remote ls lrz:latest       # List latest from specific DC
owilix remote pull all:latest     # Download latest datasets

# Local operations
owilix local ls all               # List local datasets
owilix local info <id>            # Show dataset details

# Configuration
owilix config version             # Show version info
owilix config list                # Show full config
owilix config add-repo my-s3 -t s3a --endpoint https://s3.example.com

# Query (DuckDB)
owilix query less --remote main:latest  # Quick data preview
owilix query sites --remote main:latest # Site statistics

# Remote full-text search
owilix remote search "open web search" --language eng --limit 10
owilix remote search "graz" --language deu --fetch --select "url,title,quality_score"

# Optional local HTTP API preview
uv pip install "owilix[http]"
owilix http serve                 # Binds to 127.0.0.1:8080 by default

The HTTP API is a preview local service. Do not expose it on a public or shared network without an external access-control layer.

Docker #

# Pull and run
docker run --rm -v ~/.owi:/home/owi/.owi \
  opencode.it4i.eu:5050/openwebsearcheu-public/owi-cli:latest --version

# Set alias
alias owilix='docker run --rm -it -v ~/.owi:/home/owi/.owi opencode.it4i.eu:5050/openwebsearcheu-public/owi-cli:latest'

Screencast #

Demo: screencast video

Data License #

By using this tool you agree to the Open Web Index License (OWIL) and our Terms of Use.

Software License #

OWILIX is licensed under the Apache 2.0 License.