doc/website: v0.24 release notes (#2860)

* doc/website: v0.24 release notes * slash * doc: add testing pages --------- Co-authored-by: Ariel Mashraki <[email protected]>
ariga · Jun 14, 2024 · 44a6f88 · 44a6f88
1 parent 078008e
commit 44a6f88
Show file tree

Hide file tree

Showing 7 changed files with 1,046 additions and 2 deletions.
diff --git a/doc/md/testing/migrate.mdx b/doc/md/testing/migrate.mdx
@@ -0,0  1,270 @@
 ---
 title: Testing Migrations
 slug: /testing/migrate
 ---
 
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 import LoginRequired from '../components/login-required.mdx'
 
 The `atlas migrate test` command allows you to write tests for your schema migrations. This feature
 enables you to test logic in your migrations in a minimal and straightforward way. The command is similar to
 [`atlas schema test`](/testing/schema) but is focused on testing schema migrations.
 
 :::info Login Required
 Migrations testing works only for logged-in users, free and paid. Use the following command to use this feature:
 
 ```shell
 atlas login
 ```
 
 :::
 
 ### Introduction
 
 Atlas migrations testing is inspired by the popular databases in the way they test their public (and private) APIs. The structure
 is defined in HCL files (suffixed with a `.test.hcl`), but the underlying testing logic is written in plain SQL. The
 following document describes the different structure options, flags, and capabilities supported by the testing framework.
 
 #### Flags
 
 * `--dir` the URL to the migration directory, by default it is `file://migrations`.
 * `--dev-url` - a [URL](/concepts/url) to the [_Dev-Database_](../concepts/dev.mdx) to run the tests on.
 * `--run` (optional) - run only tests matching the given regexp.
 
 #### Examples
 
 <Tabs>
 <TabItem value="dir" label="Test Directories">
 
 ```shell
 atlas migrate test --dev-url "docker://postgres/15/dev" .
 ```
 
 </TabItem>
 <TabItem value="file" label="Test Files">
 
 ```shell
 atlas migrate test --env local migrate.test.hcl
 ```
 
 </TabItem>
 </Tabs>
 
 ### The `test "migrate"` block
 
 The `test "migrate" "<name>"` block describes a test case. The second label defines the test case name, and the following
 arguments are supported:
 
 - `skip` (bool) - indicates whether the test case should be executed or skipped (can be computed). Defaults to `false`.
 
 Every test case starts with the zero state of the migration directory, and calls to the `migrate` command
 blocks migrate it to the specified version. At the end of the execution, Atlas cleans up the dev-database and prepares
 it for the next test case, regardless of the test result.
 
 #### Example
 
 ```hcl
 test "migrate" "20240613061102" {
   # Migrate to version 20240613061046.
   migrate {
     to = "20240613061046"
   }
   # Insert some test data.
   exec {
     sql = "insert into users (name) values ('Ada Lovelace')"
   }
   # Migrate to version 20240613061102.
   migrate {
       to = "20240613061102"
   }
   # Verify the correctness of the data migration.
   exec {
     sql = "select first_name,last_name from users"
     output = "Ada, Lovelace"
   }
 }
 ```
 :::info
 * Unlike [schema test](/testing/schema) cases, migration tests cannot be run in parallel.
 * The `skip` argument or the `--run` flag can be used to test only migrations that are relevant to the current
   development. For example, testing only the latest migration.
 :::
 
 A test case is composed of zero or more commands that are executed in order, and it is aborted if any of the commands fail.
 The supported commands are:
 
 ### `migrate` command
 
 The `migrate` command expects a migration version to migrate to. The strategy for testing a specific version is to
 migrate to the version before it, insert test data, migrate to the tested version to ensure it passes, and then run
 assertions to verify its correctness.
 
 ```hcl title="migrate.test.hcl"
 test "migrate" "20240613061102" {
   # Migrate to one version before the tested version.
   migrate {
     to = "20240613061046"
   }
   # Insert some test data.
   exec {
     sql = "insert into users (name) values ('Ada Lovelace')"
   }
   # Migrate to the tested version.
   migrate {
       to = "20240613061102"
   }
   # Verify the correctness of the data migration.
   exec {
     sql = "select first_name,last_name from users"
     output = "Ada, Lovelace"
   }
 }
 ```
 
 ### `exec` command
 
 The `exec` command expects an SQL statement to pass. If `output` or `match` is defined, the output of the SQL statement
 is compared to the expected value.
 
 * `sql` (string) - the SQL statement to execute.
 * `format` (optional) - the format of the output (default: `csv`). Can be `table` or `csv`.
 * `output` (optional) - the expected output of the SQL statement.
 * `match` (optional) - a regular expression to match the output of the SQL statement.
 
 ```hcl title="migrate.test.hcl"
 test "migrate" "20240613061046" {
   migrate {
     to = "20240613061046"
   }
   # Expected exec to pass.
   exec {
     sql = <<SQL
 INSERT INTO t VALUES (1, 'one');
 INSERT INTO t VALUES (2, 'two');
 SQL
   }
   # Expected exec to pass and output
   # be equal to the expected table.
   exec {
     sql = "SELECT a, b FROM t;"
     format = table
     output = <<TAB
  a |  b
 --- -----
  1 | one
  2 | two
 TAB
   }
 }
 ```
 
 ### `catch` command
 
 The `catch` command expects an SQL statement to fail. If `error` is defined, the error message is compared to the expected
 value.
 
 * `sql` (string) - the SQL statement to execute.
 * `error` (optional) - matches the error message.
 
 ```hcl title="migrate.test.hcl"
 test "migrate" "20240613061046" {
   migrate {
     to = "20240613061046"
   }
   catch {
     sql   = "SELECT 1 "
     error = "incomplete input"
   }
 }
 ```
 
 ### `assert` command
 
 The `assert` command expects an SQL statement to pass and the output to be a single row ( column) with a true value.
 
 * `sql` (string) - the SQL statement to execute.
 * `error_message` (optional) - the error message to display if the assertion fails.
 
 ```hcl title="migrate.test.hcl"
 test "migrate" "20240613061046" {
   migrate {
     to = "20240613061046"
   }
   assert {
     sql = "SELECT json_valid('{')"
     error_message = "expects a valid JSON"
   }
 }
 ```
 
 ### `log` command
 
 The `log` command logs a message to the test output. It can be useful to report the progress of the test case.
 
 * `message` (string) - the message to log.
 
 ```hcl title="migrate.test.hcl"
 test "migrate" "20240613061046" {
   migrate {
     to = "20240613061046"
   }
   log {
     message = "Seeded the database"
   }
 }
 ```
 
 ### Input Variables
 
 Test files can be parameterized using variables, and their values can be set through the [`atlas.hcl`](../atlas-schema/projects)
 config file. For example, given this test file:
 
 ```hcl title="migrate.test.hcl"
 variable "seed_file" {
   type = string
 }
 
 test "migrate" "seed" {
   migrate {
     to = "20240613061046"
   }
   exec {
     sql = "SELECT seed('${var.seed_file}')"
   }
 }
 ```
 
 Test config can be defined on the `env` block (or globally) and executed using the `--env` flag:
 
 ```hcl title="atlas.hcl"
 env "dev" {
   src = "<schema to test>"
   dev = "<docker-based dev-url>"
 
 // highlight-start
   # Test configuration for local development.
   test {
     migrate {
       src = ["migrate.test.hcl"]
       vars = {
         seed_file = "filename.sql"
         variable2 = var.name
         variable3 = data.external.value
       }
     }
   }
 // highlight-end
 }
 ```
 
 ```
 atlas migrate test --env dev
 ```
 
 :::info Input Variables
 
 [Input variables](../atlas-schema/input-variables) can be defined statically per environment, injected from the CLI using
 the `--var` flag, or computed from a [data source](../atlas-schema/projects#data-sources).
 
 :::