From 28beec84deece800af7a4c6ac6676ae9e966ab3f Mon Sep 17 00:00:00 2001 From: atlowChemi Date: Sun, 10 May 2026 09:25:09 +0300 Subject: [PATCH] test_runner: add tags option and tag-name filter Adds a `tags` option to test(), it(), suite(), and describe() that accepts an array of string labels. Tags are canonicalized to lowercase and inherited from suites to nested tests by union. Reporter events expose the tag set on every test, and TestContext exposes the test's tags via `context.tags`. Filtering is done by literal tag name through the new `--experimental-test-tag-filter=` flag (or `testTagFilters` on run()). The flag may be specified more than once; tests must contain every filter to run. Untagged tests are excluded under any positive filter. The tagging mechanism is gated behind a one-shot ExperimentalWarning. Signed-off-by: atlowChemi --- doc/api/cli.md | 18 +++ doc/api/test.md | 134 ++++++++++++++++++ doc/node.1 | 4 + lib/internal/test_runner/harness.js | 2 + lib/internal/test_runner/runner.js | 31 ++++ lib/internal/test_runner/tag_filter.js | 117 +++++++++++++++ lib/internal/test_runner/test.js | 67 +++++++-- lib/internal/test_runner/tests_stream.js | 19 ++- lib/internal/test_runner/utils.js | 37 ++++- src/node_options.cc | 5 + src/node_options.h | 1 + test/fixtures/test-runner/tagged.js | 22 +++ test/parallel/test-runner-tag-filter-cli.mjs | 108 ++++++++++++++ test/parallel/test-runner-tags-events.mjs | 96 +++++++++++++ .../test-runner-tags-experimental-warning.mjs | 97 +++++++++++++ .../parallel/test-runner-tags-inheritance.mjs | 118 +++++++++++++++ test/parallel/test-runner-tags-validation.mjs | 117 +++++++++++++++ 17 files changed, 977 insertions(+), 16 deletions(-) create mode 100644 lib/internal/test_runner/tag_filter.js create mode 100644 test/fixtures/test-runner/tagged.js create mode 100644 test/parallel/test-runner-tag-filter-cli.mjs create mode 100644 test/parallel/test-runner-tags-events.mjs create mode 100644 test/parallel/test-runner-tags-experimental-warning.mjs create mode 100644 test/parallel/test-runner-tags-inheritance.mjs create mode 100644 test/parallel/test-runner-tags-validation.mjs diff --git a/doc/api/cli.md b/doc/api/cli.md index 80f2fa74818b92..8f7d6185ac464c 100644 --- a/doc/api/cli.md +++ b/doc/api/cli.md @@ -1413,6 +1413,23 @@ Enable module mocking in the test runner. This feature requires `--allow-worker` if used with the [Permission Model][]. +### `--experimental-test-tag-filter=` + + + +> Stability: 1.0 - Early development + +Run only tests whose tag set contains ``. Tests declare tags via the +`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags +inherit from suites to nested tests by union. Filtering is +case-insensitive. + +The flag may be specified more than once; tests must contain **every** +filter value to run. See [Test tags][] for details on declaring and +inheriting tags. + ### `--experimental-vm-modules` + +> Stability: 1.0 - Early development + +Tags annotate tests and suites with arbitrary string labels. The +[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters` +option on [`run()`][]) selects tests whose tag set contains every +provided filter value. + +Tags are an alternative to encoding metadata into test names. They are +useful for cross-cutting axes such as subsystem, speed bucket, flakiness, +or environment, where a name pattern would be brittle. + +### Authoring tagged tests + +Pass a `tags` array on any of `test()`, `it()`, `suite()`, or `describe()`. +Tags inherit from a suite to its child tests by union—a test inside a +suite tagged `['db']` that declares its own `tags: ['integration']` +effectively has both tags. + +```mjs +import { describe, it } from 'node:test'; + +describe('database', { tags: ['db'] }, () => { + it('reads a row'); // tags: ['db'] + it('writes a row', { tags: ['integration'] }); // tags: ['db', 'integration'] + it('reconnects after disconnect', { tags: ['flaky'] }); // tags: ['db', 'flaky'] +}); +``` + +```cjs +const { describe, it } = require('node:test'); + +describe('database', { tags: ['db'] }, () => { + it('reads a row'); // tags: ['db'] + it('writes a row', { tags: ['integration'] }); // tags: ['db', 'integration'] + it('reconnects after disconnect', { tags: ['flaky'] }); // tags: ['db', 'flaky'] +}); +``` + +Tag values must be non-empty strings. Tags are matched case-insensitively; +the canonical form is lowercase. Duplicates within a single `tags` array +are collapsed on the lowercased form, preserving the first-seen +declaration order. + +Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their +own tags. They run as part of their owning suite, which carries the +suite's tags. + +### Filtering by tag + +Each [`--experimental-test-tag-filter`][] value is a literal tag name. A +test runs only when its tag set contains that name. The flag may be +specified more than once; tests must match **every** filter to run. The +same applies to the `testTagFilters` array on [`run()`][]. Filters are +case-insensitive and AND'd with [`--test-name-pattern`][], +[`--test-skip-pattern`][], and `.only` filtering. + +Untagged tests are excluded under any non-empty filter, since the filter +requires the tag to be present. + +### Reading tags from inside a test + +The [`TestContext`][] object exposes the test's tags as a frozen array +through [`context.tags`][], so tests can branch on their own metadata. + +### Errors + +A tag value that violates the validation rules above throws +`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs. +A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. + ## Extraneous asynchronous activity Once a test function finishes executing, the results are reported as quickly @@ -750,6 +826,8 @@ test runner functionality: * `--test` - Prevented to avoid recursive test execution * `--experimental-test-coverage` - Managed by the test runner +* `--experimental-test-tag-filter` - Filter values are validated by the parent + process and re-emitted to child processes * `--watch` - Watch mode is handled at the parent level * `--experimental-default-config-file` - Config file loading is handled by the parent * `--test-reporter` - Reporting is managed by the parent process @@ -1568,6 +1646,9 @@ added: - v18.9.0 - v16.19.0 changes: + - version: REPLACEME + pr-url: https://github.com/nodejs/node/pull/63221 + description: Added the `testTagFilters` option. - version: - v25.6.0 - v24.14.0 @@ -1656,6 +1737,10 @@ changes: For each test that is executed, any corresponding test hooks, such as `beforeEach()`, are also run. **Default:** `undefined`. + * `testTagFilters` {string|string\[]} A tag name, or an array of tag names, + used to filter tests by their declared tags. Tests must contain every + listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][] + on the command line. See [Test tags][]. **Default:** `undefined`. * `timeout` {number} A number of milliseconds the test execution will fail after. If unspecified, subtests inherit this value from their parent. @@ -1799,6 +1884,9 @@ added: - v18.0.0 - v16.17.0 changes: + - version: REPLACEME + pr-url: https://github.com/nodejs/node/pull/63221 + description: Added the `tags` option. - version: - v20.2.0 - v18.17.0 @@ -1842,6 +1930,10 @@ changes: * `skip` {boolean|string} If truthy, the test is skipped. If a string is provided, that string is displayed in the test results as the reason for skipping the test. **Default:** `false`. + * `tags` {string\[]} An array of string labels associated with the test. + Used together with [`--experimental-test-tag-filter`][] to filter which + tests run. Tags inherit from suites to nested tests by union. See + [Test tags][]. **Default:** `[]`. * `todo` {boolean|string} If truthy, the test marked as `TODO`. If a string is provided, that string is displayed in the test results as the reason why the test is `TODO`. **Default:** `false`. @@ -3430,6 +3522,9 @@ Emitted when code coverage is enabled and all tests have completed. `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3453,6 +3548,9 @@ The corresponding declaration ordered events are `'test:pass'` and `'test:fail'` `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3494,6 +3592,9 @@ defined. `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3520,6 +3621,9 @@ Emitted when a test is enqueued for execution. `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3579,6 +3683,9 @@ since the parent runner only knows about file-level tests. When using `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -3618,6 +3725,9 @@ defined. `undefined` if the test was run through the REPL. * `name` {string} The test name. * `nesting` {number} The nesting level of the test. + * `tags` {string\[]} The flattened lowercased tags declared on the test + and its ancestor suites, in declaration order. Empty for untagged tests. + See [Test tags][]. * `testId` {number} A numeric identifier for this test instance, unique within the test file's process. Consistent across all events for the same test instance, enabling reliable correlation in custom reporters. @@ -4121,6 +4231,20 @@ The attempt number of the test. This value is zero-based, so the first attempt i the second attempt is `1`, and so on. This property is useful in conjunction with the `--test-rerun-failures` option to determine which attempt the test is currently running. +### `context.tags` + + + +> Stability: 1.0 - Early development + +* Type: {string\[]} + +A frozen array of the test's flattened lowercased tags, in declaration +order, including any tags inherited from ancestor suites. Empty when the +test has no tags. See [Test tags][]. + ### `context.workerId`