Skip to content

Commit dd77ccb

Browse files
juliangruberjuliangruber
authored
test_runner: support only tests (#3)
* test_runner: support only tests * docs
1 parent 76a33c6 commit dd77ccb

10 files changed

Lines changed: 522 additions & 64 deletions

‎README.md‎

Lines changed: 284 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
[![CI](https://github.com/juliangruber/node-core-test/actions/workflows/ci.yml/badge.svg)](https://github.com/juliangruber/node-core-test/actions/workflows/ci.yml)
44

5-
This is a user-land port of [`node:test`](https://github.com/nodejs/node/blob/b476b1b91ef8715f096f815db5a0c8722b613678/doc/api/test.md),
5+
This is a user-land port of [`node:test`](https://github.com/nodejs/node/blob/54819f08e0c469528901d81a9cee546ea518a5c3/doc/api/test.md),
66
the experimental test runner introduced in Node.js 18. This module makes it
77
available in Node.js 14 and later.
88

@@ -16,13 +16,48 @@ Differences from the core implementation:
1616
- Doesn't hide its own stack frames
1717
- Internally uses `._private` property names instead of `#private` fields,
1818
for compatibility
19+
- Uses `String` instead of `Symbol`, for compatibility
1920

20-
## Usage
21+
## Docs
2122

22-
```js
23-
const assert = require('assert')
24-
const test = require('node-core-test')
23+
> https://github.com/nodejs/node/blob/54819f08e0c469528901d81a9cee546ea518a5c3/doc/api/test.md
24+
25+
# Test runner
26+
27+
<!--introduced_in=REPLACEME-->
28+
29+
> Stability: 1 - Experimental
30+
31+
<!-- source_link=lib/test.js -->
32+
33+
The `node:test` module facilitates the creation of JavaScript tests that
34+
report results in [TAP][] format. To access it:
35+
36+
```mjs
37+
import test from 'node-core-test'
38+
```
39+
40+
```cjs
41+
const test = require('node-core-test');
42+
```
43+
44+
Tests created via the `test` module consist of a single function that is
45+
processed in one of three ways:
2546

47+
1. A synchronous function that is considered failing if it throws an exception,
48+
and is considered passing otherwise.
49+
2. A function that returns a `Promise` that is considered failing if the
50+
`Promise` rejects, and is considered passing if the `Promise` resolves.
51+
3. A function that receives a callback function. If the callback receives any
52+
truthy value as its first argument, the test is considered failing. If a
53+
falsy value is passed as the first argument to the callback, the test is
54+
considered passing. If the test function receives a callback function and
55+
also returns a `Promise`, the test will fail.
56+
57+
The following example illustrates how tests are written using the
58+
`test` module.
59+
60+
```js
2661
test('synchronous passing test', t => {
2762
// This test passes because it does not throw an exception.
2863
assert.strictEqual(1, 1)
@@ -69,48 +104,256 @@ test('callback failing test', (t, done) => {
69104
})
70105
```
71106

72-
```bash
73-
$ node example.js
74-
TAP version 13
75-
ok 1 - synchronous passing test
76-
---
77-
duration_ms: 0.001514889
78-
...
79-
not ok 2 - synchronous failing test
80-
---
81-
duration_ms: 0.002878527
82-
failureType: 'testCodeFailure'
83-
error: 'Expected values to be strictly equal:\n\n1 !== 2\n'
84-
stack: |-
85-
Test.run (/Users/julian/dev/juliangruber/node-core-test/lib/test.js:347:17)
86-
Test.processPendingSubtests (/Users/julian/dev/juliangruber/node-core-test/lib/test.js:153:27)
87-
Test.postRun (/Users/julian/dev/juliangruber/node-core-test/lib/test.js:390:19)
88-
Test.run (/Users/julian/dev/juliangruber/node-core-test/lib/test.js:352:10)
89-
processTicksAndRejections (node:internal/process/task_queues:96:5)
90-
91-
...
92-
(run it yourself to see the full output)
93-
...
94-
95-
1..7
96-
# tests 7
97-
# pass 3
98-
# fail 4
99-
# skipped 0
100-
# todo 0
101-
$ echo $?
107+
As a test file executes, TAP is written to the standard output of the Node.js
108+
process. This output can be interpreted by any test harness that understands
109+
the TAP format. If any tests fail, the process exit code is set to `1`.
110+
111+
## Subtests
112+
113+
The test context's `test()` method allows subtests to be created. This method
114+
behaves identically to the top level `test()` function. The following example
115+
demonstrates the creation of a top level test with two subtests.
116+
117+
```js
118+
test('top level test', async t => {
119+
await t.test('subtest 1', t => {
120+
assert.strictEqual(1, 1)
121+
})
122+
123+
await t.test('subtest 2', t => {
124+
assert.strictEqual(2, 2)
125+
})
126+
})
102127
```
103128

104-
You can also run the tests using the [tap](https://npm.im/tap) CLI, or any
105-
other CLI that expects TAP output, for improved DX:
129+
In this example, `await` is used to ensure that both subtests have completed.
130+
This is necessary because parent tests do not wait for their subtests to
131+
complete. Any subtests that are still outstanding when their parent finishes
132+
are cancelled and treated as failures. Any subtest failures cause the parent
133+
test to fail.
106134

107-
```bash
108-
$ tap example.js
135+
## Skipping tests
136+
137+
Individual tests can be skipped by passing the `skip` option to the test, or by
138+
calling the test context's `skip()` method. Both of these options support
139+
including a message that is displayed in the TAP output as shown in the
140+
following example.
141+
142+
```js
143+
// The skip option is used, but no message is provided.
144+
test('skip option', { skip: true }, t => {
145+
// This code is never executed.
146+
})
147+
148+
// The skip option is used, and a message is provided.
149+
test('skip option with message', { skip: 'this is skipped' }, t => {
150+
// This code is never executed.
151+
})
152+
153+
test('skip() method', t => {
154+
// Make sure to return here as well if the test contains additional logic.
155+
t.skip()
156+
})
157+
158+
test('skip() method with message', t => {
159+
// Make sure to return here as well if the test contains additional logic.
160+
t.skip('this is skipped')
161+
})
162+
```
163+
164+
### `only` tests
165+
166+
If Node.js is started with the `--test-only` command-line option, it is
167+
possible to skip all top level tests except for a selected subset by passing
168+
the `only` option to the tests that should be run. When a test with the `only`
169+
option set is run, all subtests are also run. The test context's `runOnly()`
170+
method can be used to implement the same behavior at the subtest level.
171+
172+
```js
173+
// Assume Node.js is run with the --test-only command-line option.
174+
// The 'only' option is set, so this test is run.
175+
test('this test is run', { only: true }, async t => {
176+
// Within this test, all subtests are run by default.
177+
await t.test('running subtest')
178+
179+
// The test context can be updated to run subtests with the 'only' option.
180+
t.runOnly(true)
181+
await t.test('this subtest is now skipped')
182+
await t.test('this subtest is run', { only: true })
183+
184+
// Switch the context back to execute all tests.
185+
t.runOnly(false)
186+
await t.test('this subtest is now run')
187+
188+
// Explicitly do not run these tests.
189+
await t.test('skipped subtest 3', { only: false })
190+
await t.test('skipped subtest 4', { skip: true })
191+
})
192+
193+
// The 'only' option is not set, so this test is skipped.
194+
test('this test is not run', () => {
195+
// This code is not run.
196+
throw new Error('fail')
197+
})
109198
```
110199

111-
## API
200+
## Extraneous asynchronous activity
201+
202+
Once a test function finishes executing, the TAP results are output as quickly
203+
as possible while maintaining the order of the tests. However, it is possible
204+
for the test function to generate asynchronous activity that outlives the test
205+
itself. The test runner handles this type of activity, but does not delay the
206+
reporting of test results in order to accommodate it.
207+
208+
In the following example, a test completes with two `setImmediate()`
209+
operations still outstanding. The first `setImmediate()` attempts to create a
210+
new subtest. Because the parent test has already finished and output its
211+
results, the new subtest is immediately marked as failed, and reported in the
212+
top level of the file's TAP output.
213+
214+
The second `setImmediate()` creates an `uncaughtException` event.
215+
`uncaughtException` and `unhandledRejection` events originating from a completed
216+
test are handled by the `test` module and reported as diagnostic warnings in
217+
the top level of the file's TAP output.
218+
219+
```js
220+
test('a test that creates asynchronous activity', t => {
221+
setImmediate(() => {
222+
t.test('subtest that is created too late', t => {
223+
throw new Error('error1')
224+
})
225+
})
226+
227+
setImmediate(() => {
228+
throw new Error('error2')
229+
})
230+
231+
// The test finishes after this line.
232+
})
233+
```
234+
235+
## `test([name][, options][, fn])`
236+
237+
- `name` {string} The name of the test, which is displayed when reporting test
238+
results. **Default:** The `name` property of `fn`, or `'<anonymous>'` if `fn`
239+
does not have a name.
240+
- `options` {Object} Configuration options for the test. The following
241+
properties are supported:
242+
- `concurrency` {number} The number of tests that can be run at the same time.
243+
If unspecified, subtests inherit this value from their parent.
244+
**Default:** `1`.
245+
- `only` {boolean} If truthy, and the test context is configured to run
246+
`only` tests, then this test will be run. Otherwise, the test is skipped.
247+
**Default:** `false`.
248+
- `skip` {boolean|string} If truthy, the test is skipped. If a string is
249+
provided, that string is displayed in the test results as the reason for
250+
skipping the test. **Default:** `false`.
251+
- `todo` {boolean|string} If truthy, the test marked as `TODO`. If a string
252+
is provided, that string is displayed in the test results as the reason why
253+
the test is `TODO`. **Default:** `false`.
254+
- `fn` {Function|AsyncFunction} The function under test. This first argument
255+
to this function is a [`TestContext`][] object. If the test uses callbacks,
256+
the callback function is passed as the second argument. **Default:** A no-op
257+
function.
258+
- Returns: {Promise} Resolved with `undefined` once the test completes.
259+
260+
The `test()` function is the value imported from the `test` module. Each
261+
invocation of this function results in the creation of a test point in the TAP
262+
output.
263+
264+
The `TestContext` object passed to the `fn` argument can be used to perform
265+
actions related to the current test. Examples include skipping the test, adding
266+
additional TAP diagnostic information, or creating subtests.
267+
268+
`test()` returns a `Promise` that resolves once the test completes. The return
269+
value can usually be discarded for top level tests. However, the return value
270+
from subtests should be used to prevent the parent test from finishing first
271+
and cancelling the subtest as shown in the following example.
272+
273+
```js
274+
test('top level test', async t => {
275+
// The setTimeout() in the following subtest would cause it to outlive its
276+
// parent test if 'await' is removed on the next line. Once the parent test
277+
// completes, it will cancel any outstanding subtests.
278+
await t.test('longer running subtest', async t => {
279+
return new Promise((resolve, reject) => {
280+
setTimeout(resolve, 1000)
281+
})
282+
})
283+
})
284+
```
285+
286+
## Class: `TestContext`
287+
288+
An instance of `TestContext` is passed to each test function in order to
289+
interact with the test runner. However, the `TestContext` constructor is not
290+
exposed as part of the API.
291+
292+
### `context.diagnostic(message)`
293+
294+
- `message` {string} Message to be displayed as a TAP diagnostic.
295+
296+
This function is used to write TAP diagnostics to the output. Any diagnostic
297+
information is included at the end of the test's results. This function does
298+
not return a value.
299+
300+
### `context.runOnly(shouldRunOnlyTests)`
301+
302+
- `shouldRunOnlyTests` {boolean} Whether or not to run `only` tests.
303+
304+
If `shouldRunOnlyTests` is truthy, the test context will only run tests that
305+
have the `only` option set. Otherwise, all tests are run. If Node.js was not
306+
started with the [`--test-only`][] command-line option, this function is a
307+
no-op.
308+
309+
### `context.skip([message])`
310+
311+
- `message` {string} Optional skip message to be displayed in TAP output.
312+
313+
This function causes the test's output to indicate the test as skipped. If
314+
`message` is provided, it is included in the TAP output. Calling `skip()` does
315+
not terminate execution of the test function. This function does not return a
316+
value.
317+
318+
### `context.todo([message])`
319+
320+
- `message` {string} Optional `TODO` message to be displayed in TAP output.
321+
322+
This function adds a `TODO` directive to the test's output. If `message` is
323+
provided, it is included in the TAP output. Calling `todo()` does not terminate
324+
execution of the test function. This function does not return a value.
325+
326+
### `context.test([name][, options][, fn])`
327+
328+
- `name` {string} The name of the subtest, which is displayed when reporting
329+
test results. **Default:** The `name` property of `fn`, or `'<anonymous>'` if
330+
`fn` does not have a name.
331+
- `options` {Object} Configuration options for the subtest. The following
332+
properties are supported:
333+
- `concurrency` {number} The number of tests that can be run at the same time.
334+
If unspecified, subtests inherit this value from their parent.
335+
**Default:** `1`.
336+
- `only` {boolean} If truthy, and the test context is configured to run
337+
`only` tests, then this test will be run. Otherwise, the test is skipped.
338+
**Default:** `false`.
339+
- `skip` {boolean|string} If truthy, the test is skipped. If a string is
340+
provided, that string is displayed in the test results as the reason for
341+
skipping the test. **Default:** `false`.
342+
- `todo` {boolean|string} If truthy, the test marked as `TODO`. If a string
343+
is provided, that string is displayed in the test results as the reason why
344+
the test is `TODO`. **Default:** `false`.
345+
- `fn` {Function|AsyncFunction} The function under test. This first argument
346+
to this function is a [`TestContext`][] object. If the test uses callbacks,
347+
the callback function is passed as the second argument. **Default:** A no-op
348+
function.
349+
- Returns: {Promise} Resolved with `undefined` once the test completes.
350+
351+
This function is used to create subtests under the current test. This function
352+
behaves in the same fashion as the top level [`test()`][] function.
112353

113-
https://github.com/nodejs/node/blob/b476b1b91ef8715f096f815db5a0c8722b613678/doc/api/test.md
354+
[tap]: https://testanything.org/
355+
[`testcontext`]: #class-testcontext
356+
[`test()`]: #testname-options-fn
114357

115358
## Kudos
116359

‎lib/options.js‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
// https://github.com/nodejs/node/blob/54819f08e0c469528901d81a9cee546ea518a5c3/lib/internal/options.js
2+
3+
'use strict'
4+
5+
const minimist = require('minimist')
6+
7+
const argv = minimist(process.argv.slice(2))
8+
9+
function getOptionValue (optionName) {
10+
return argv[optionName.slice(2)] // remove leading --
11+
}
12+
13+
module.exports = {
14+
getOptionValue
15+
}

0 commit comments

Comments
 (0)