diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 86dd8d1..b86e9ad 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -5,7 +5,7 @@ arrow = "1.2.4" # https://github.com/arrow-kt/arrow/releases kotlinx-coroutines = "1.8.1" # https://github.com/Kotlin/kotlinx.coroutines/releases kotlinx-datetime = "0.5.0" # https://github.com/Kotlin/kotlinx-datetime/releases gradle-testkit = "8.6" # https://gradle.org/releases/ -parameterize = "0.3.0" # https://github.com/BenWoodworth/Parameterize/releases +parameterize = "0.3.2" # https://github.com/BenWoodworth/Parameterize/releases ktor = "2.3.11" # https://ktor.io/docs/releases.html#release-details [plugins] -- 2.51.2 From e50e6c9b59d9e8dc17237bd09807c9af8ae77bc1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ivan=20=E2=80=9CCLOVIS=E2=80=9D=20Canet?= Date: Sat, 31 Aug 2024 21:40:04 +0200 Subject: [PATCH 2/3] docs(website): Add a page on parameterized testing --- docs/website/docs/features/index.md | 10 +++ docs/website/docs/features/parameterize.md | 96 ++++++++++++++++++++++ docs/website/mkdocs.yml | 1 + 3 files changed, 107 insertions(+) create mode 100644 docs/website/docs/features/parameterize.md diff --git a/docs/website/docs/features/index.md b/docs/website/docs/features/index.md index 2cd8971..c80ef26 100644 --- a/docs/website/docs/features/index.md +++ b/docs/website/docs/features/index.md @@ -38,11 +38,21 @@ fun SuiteDsl.testUsers() = suite("Test users") { // … } } + + parameterize { //(2)! + val a by parameterOf(1, 2, 3, 4) + val b by parameter('a'..'z') + + test("Test $a $b") { + // … + } + } } } ``` 1. Because tests are declared as regular function calls, any Kotlin language feature, like loops, can be used to declare complex suites. No need to learn annotation-based test parameterization anymore! +2. For more complex cases, the `parameterize` DSL can be used to declare many tests quickly. Here, 104 tests will be declared (4 × 26). [Learn more](parameterize.md). To learn more about tests and suites, see [the dedicated page](../tutorials/syntax.md). diff --git a/docs/website/docs/features/parameterize.md b/docs/website/docs/features/parameterize.md new file mode 100644 index 0000000..7973b10 --- /dev/null +++ b/docs/website/docs/features/parameterize.md @@ -0,0 +1,96 @@ +# Parameterized tests + +When we want to test edge cases or invariants, it is common to execute the same test, but with different data. For the sake of this example, let's say we want to test a function that checks whether a number is odd: +```kotlin +fun Int.isOdd(): Boolean = + this % 2 == 0 +``` + +We want to ensure that the function works for a bunch of values. A naive solution would be to test all values in a single test: +```kotlin +fun SuiteDsl.testIsOdd() { + test("Test isOdd") { + val even = listOf(0, 2, 4, -2, 984) + val odd = listOf(1, 3, -1, 873) + + for (number in even) + check(!number.isOdd()) + + for (number in odd) + check(number.isOdd()) + } +} +``` + +This approach works, but it has a flaw: if a test fails, it's not possible to know at a glance which other values would be successful or not. Our brain is great at recognizing patterns, so seeing all failed cases often gives insight on what could be wrong. + +## Single parameter + +Since Prepared is [DSL-based](../tutorials/syntax.md), we can use any language feature to programmatically declare multiple tests. For example, we can lift the loop out of the test: +```kotlin +fun SuiteDsl.testIsOdd() = suite("isOdd") { + val even = listOf(0, 2, 4, -2, 984) + val odd = listOf(1, 3, -1, 873) + + for (number in even) { + test("$number should not be odd") { + check(!number.isOdd()) + } + } + + for (number in odd) { + test("$number should be odd") { + check(number.isOdd()) + } + } +} +``` + +Each test will be reported as independent failures, so we can quickly get an overview of what works and what doesn't. + +Instead of using loops, we can use any other language feature that helps code reuse: utility functions, the [`repeat` helper](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/repeat.html), etc. + +!!! tip "Consider creating an enclosing suite" + When declaring tests programmatically, we recommend creating an enclosing `suite` for all the generated tests, to ensure reports are easy to read. You can see this being done in the very first line of the example above. + +## Multiple parameters + +When we want to test combinations of multiple parameters, the previous approach can quickly become unwieldy: +```kotlin +fun SuiteDsl.foo() = suite("Foo") { + for (a in listOf(1, 2, 3, 4, 5)) { + for (b in listOf("b", "", "aaaaaaa", a.toString())) { + for (c in listOf(true, false, null)) { + test("foo $a $b $c") { + foo(a, b, c) + } + } + } + } +} +``` +The added indentation levels make the intent of the code harder to understand. Instead, we can use the [Parameterize](https://github.com/BenWoodworth/parameterize) library to simplify complex declarations. + +!!! info "Configuration" + Add a dependency on `dev.opensavvy.prepared:compat-parameterize` to use the features in this section. + + See the [reference](https://opensavvy.gitlab.io/groundwork/prepared/api-docs/compat/compat-parameterize/index.html). + +The previous example can be rewritten as: +```kotlin +fun SuiteDsl.foo() = suite("Foo") { + parameterize { + val a by parameterOf(1, 2, 3, 4, 5) + val b by parameterOf("b", "", "aaaaaa", a.toString()) + val c by parameterOf(true, false, null) + + test("foo $a $b $c") { + foo(a, b, c) + } + } +} +``` + +As you can see, this is much easier to read, and the intent of the test is conveyed much more clearly. + +To learn more about this module, and the way it interacts with [prepared values](prepared-values.md), see [the reference](https://opensavvy.gitlab.io/groundwork/prepared/api-docs/compat/compat-parameterize/index.html). diff --git a/docs/website/mkdocs.yml b/docs/website/mkdocs.yml index 3efc598..48e2f66 100644 --- a/docs/website/mkdocs.yml +++ b/docs/website/mkdocs.yml @@ -98,6 +98,7 @@ nav: - features/random.md - features/time.md - features/files.md + - features/parameterize.md - Best practices: - practices/index.md -- 2.51.2 From 857cde847f08e98b5c855f5af807fe766065c778 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ivan=20=E2=80=9CCLOVIS=E2=80=9D=20Canet?= Date: Sat, 31 Aug 2024 21:44:39 +0200 Subject: [PATCH 3/3] docs(website): Reorganize the features into basic features and external control --- docs/website/docs/features/random.md | 2 +- docs/website/docs/features/time.md | 2 +- docs/website/mkdocs.yml | 8 +++++--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/website/docs/features/random.md b/docs/website/docs/features/random.md index c54c411..dc94cc4 100644 --- a/docs/website/docs/features/random.md +++ b/docs/website/docs/features/random.md @@ -1,4 +1,4 @@ -# Randomness control +# Randomness Randomness can be useful to generate arbitrary test data (e.g. for fuzzing, property testing, generating default values…). However, tests that exhibit randomness are quickly hard to debug: re-running them may give a different result! diff --git a/docs/website/docs/features/time.md b/docs/website/docs/features/time.md index 77efa3e..2bef0ea 100644 --- a/docs/website/docs/features/time.md +++ b/docs/website/docs/features/time.md @@ -1,4 +1,4 @@ -# Date and time control +# Date and time Tests run in virtual time: a fixed timescale which is controlled by the developer. diff --git a/docs/website/mkdocs.yml b/docs/website/mkdocs.yml index 48e2f66..aedc449 100644 --- a/docs/website/mkdocs.yml +++ b/docs/website/mkdocs.yml @@ -95,11 +95,13 @@ nav: - features/shared-values.md - features/finalizers.md - features/async.md - - features/random.md - - features/time.md - - features/files.md - features/parameterize.md + - Controlling external data: + - features/time.md + - features/random.md + - features/files.md + - Best practices: - practices/index.md