diff --git a/CHANGELOG.md b/CHANGELOG.md index eed39bf..5574ccf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,129 @@ # CHANGELOG +## v0.23.0 (2025-08-28) + +### Bug Fixes + +- Build script for opensearch indices improved. + ([`018d1ae`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/018d1ae0b3d940510c773a449ad60be0c3288a78)) + +- Documentation on quering timeout + ([`b5ad205`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/b5ad20532ba858266d9d325eb858c15e2ff004b0)) + +- Documentation on quering timeout + ([`23af57e`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/23af57e51666d9bc1b06d3b1dd55ae235f45e393)) + +- Eta correction for warc command; documentation update. + ([`5c27fbc`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/5c27fbc4fb1e73a19053a78f7a66e001ff7830b3)) + +- Eta correction for warc command; documentation update. + ([`90a429e`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/90a429ef07f82c9af512496f4f9ed43d12152858)) + +- Fixed a bug in OWIDuckDBCopyExecutor to make paths local. currently, we can only support local + duckdb copying. + ([`1597ce5`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/1597ce5a7869e08c55b6d8eb18e979b5c4c1b44d)) + +- Help with multi argument commands like query now working properly. e.g. use query help + graph_reports + ([`c470ac7`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/c470ac74ef9318e67b9ad466847bf6bca1e43fee)) + +- Improved retry logic / error reporting; connection_timout set to None default for query + ([`8322af0`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/8322af0b0572390958b5dfdadaab739487f7f001)) + +- Py4lexis 2.3.1 required unimplemented methods in fsspecirods of created / modified. + ([`0d1280a`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/0d1280a1f764283093c3102e2513917e674cc241)) + +- Query_warc parallelism over datacenters + ([`b060f31`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/b060f3105cf5c115e8b9e6df0699581c8a8bdfd5)) + +- Query_warc parallelism over datacenters + ([`46d6091`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/46d60917d01658dadf90c723cb484c346c4aa767)) + +- Setting lexis session in IRODSOWI session. + ([`e48c858`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/e48c858fa16378a7c5e0c0244b8b362102d3f1aa)) + +- Slice working again. + ([`9abaaa2`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/9abaaa2ce7222823bb8c9fb407ff9147d1474349)) + +- Speed improvement for random access warc download. Now 1-3 MB/s/Thread + ([`ebcf7e8`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/ebcf7e8fb9de153085e449e2bc9355c2be141303)) + +- Speed improvement for random access warc download. Now 2-3 MB/s/Thread + ([`40cce72`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/40cce729e0410f0af1a831bf8f9cd6ac8e6b9a15)) + +- Switched to lexis 3.2.1. + ([`03002f1`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/03002f1f13b7818aba717217a3ffddb0ace286f8)) + +- Warc query - added queue to warc downloads to enable better dataset distribution + ([`689bb86`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/689bb86691cd23a35924c3912a7e966735ffc4d0)) + +### Chores + +- Cleanup of sql folder + ([`d24a461`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/d24a461f23dfe449db2b1eee5dc7a005bf65eecc)) + +- Docker build script and update of build.sh + ([`0ec2fdd`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/0ec2fdd3582644cb7eada9101175f982e75c99e8)) + +- Test improvement + ([`1b59e30`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/1b59e3095a7bb312b2f432c6818fd1f19ee8988c)) + +- Test improvement + ([`4c1f8ba`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/4c1f8ba9e877506fcadd0a3700ada699855b7c1d)) + +- Update of gitignore + ([`348939e`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/348939ea803cb2097acadf585eb3ecff8cae3201)) + +### Documentation + +- Warc_query doc improved. + ([`5f9ffab`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/5f9ffabfdbd8639c3b48a60262e503a919022045)) + +### Features + +- Added graph statistics per dataset. Analysis can be written back to original source + ([`64a5ba2`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/64a5ba2fe7bd05ac1bf9f47df1b8f16e187f23f3)) + +- Added mime type analysis to json analyse + ([`3fe1610`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/3fe161059e99dba86ed125f0d0d51f6a3c29f908)) + +- Added support for multiple file glob patterns (As python list) + ([`83c9bb8`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/83c9bb8c36e25b6c73ba9d328d742e18d87416cc)) + +- Added workflow statistic summary for lexis workflows (no working properly on non-lexis workflows) + ([`64952ed`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/64952edf2014a848e0bfcd41f26feec80cb3be8a)) + +- Excluding data cetners in aggregated repository list function + ([`01c7466`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/01c74665766d53cb3b03042de2d3762fe1a8bd07)) + +- Excluding data cetners in aggregated repository list function + ([`0dc5a36`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/0dc5a36f8742cf5b5665308cf4aa2ef8263b84b8)) + +- Improved repository configuration. --remotes +repo_name selectes repo_name in addition to repos in + the selected_remote config part + ([`f630c2c`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/f630c2c5364e5ed8cbd4286348ea269fd63c78d5)) + +- Improved repository configuration. --remotes +repo_name selectes repo_name in addition to repos in + the selected_remote config part + ([`b78c613`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/b78c613858c99e086833b7ab6072df4c61cd56da)) + +- Irods+cache filesystem available to improve query performance + ([`95a72e8`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/95a72e8549a83447c170f9537c501a97888527f4)) + +- Slicing is working again and now also slices the ciff index + ([`49a3be9`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/49a3be9e9f457f27effad939e8577af0a8132517)) + +- Slicing of ciff files can be switchted off now, as it might take too long. + ([`2e1ba75`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/2e1ba755991a8cb9fa83659d7f8a525ed62add34)) + +- Warc cache working with parallel downloads + ([`a9a6fa4`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/a9a6fa4bfad51402e866adb867ef3f6ffefbba59)) + +- Warc cache working with parallel downloads + ([`7f68942`](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/commit/7f6894272b9c2312ff2f0ccef279f95b822d9d3a)) + + ## v0.22.2 (2025-05-26) ### Bug Fixes diff --git a/owilix/_version.py b/owilix/_version.py index 78ebff1..eb5e816 100644 --- a/owilix/_version.py +++ b/owilix/_version.py @@ -1,3 +1,3 @@ # These version placeholders will be replaced later during substitution. -__version__ = "0.22.2" +__version__ = "0.23.0" __version_tuple__ = (0, 15, 0) diff --git a/pyproject.toml b/pyproject.toml index 1160e01..16aa9d5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "owilix" -version = "0.22.2" +version = "0.23.0" description = "OWILIX - the Command Line Interface for slicing and consuming the Open Web Index. " authors = ["Michael Granitzer "] license = "MIT"