Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions doc/api/deprecations.md
Original file line number Diff line number Diff line change
Expand Up @@ -4847,6 +4847,20 @@ Type: Documentation-only
`node:sqlite`'s `StatementSync` class was renamed to `Statement`.
`StatementSync` is kept as a deprecated alias. Use `Statement` instead.

### DEP0212: `util.TextDecoder` and `util.TextEncoder`

<!-- YAML
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/67461
description: Documentation-only deprecation.
-->

Type: Documentation-only

`node:util`'s `util.TextDecoder` and `util.TextEncoder` are deprecated.
Use global `TextDecoder` and `TextEncoder` instead.

Comment thread
LiviaMedeiros marked this conversation as resolved.
Resolved
[DEP0142]: #dep0142-repl_builtinlibs
[DEP0156]: #dep0156-aborted-property-and-abort-aborted-event-in-http
[NIST SP 800-38D]: https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38d.pdf
Expand Down
178 changes: 174 additions & 4 deletions doc/api/globals.md
Original file line number Diff line number Diff line change
Expand Up @@ -1192,7 +1192,133 @@ only if the Node.js binary was compiled with including support for the
added: v11.0.0
-->

The WHATWG `TextDecoder` class. See the [`TextDecoder`][] section.
An implementation of the [WHATWG Encoding Standard][] `TextDecoder` API.

```js
const decoder = new TextDecoder();
const u8arr = new Uint8Array([72, 101, 108, 108, 111]);
console.log(decoder.decode(u8arr)); // Hello
```

### WHATWG supported encodings

Per the [WHATWG Encoding Standard][], the encodings supported by the
`TextDecoder` API are outlined in the tables below. For each encoding,
one or more aliases may be used.

Different Node.js build configurations support different sets of encodings.
(see [Internationalization][])

#### Encodings supported by default (with full ICU data)

| Encoding | Aliases |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `'ibm866'` | `'866'`, `'cp866'`, `'csibm866'` |
| `'iso-8859-2'` | `'csisolatin2'`, `'iso-ir-101'`, `'iso8859-2'`, `'iso88592'`, `'iso_8859-2'`, `'iso_8859-2:1987'`, `'l2'`, `'latin2'` |
| `'iso-8859-3'` | `'csisolatin3'`, `'iso-ir-109'`, `'iso8859-3'`, `'iso88593'`, `'iso_8859-3'`, `'iso_8859-3:1988'`, `'l3'`, `'latin3'` |
| `'iso-8859-4'` | `'csisolatin4'`, `'iso-ir-110'`, `'iso8859-4'`, `'iso88594'`, `'iso_8859-4'`, `'iso_8859-4:1988'`, `'l4'`, `'latin4'` |
| `'iso-8859-5'` | `'csisolatincyrillic'`, `'cyrillic'`, `'iso-ir-144'`, `'iso8859-5'`, `'iso88595'`, `'iso_8859-5'`, `'iso_8859-5:1988'` |
| `'iso-8859-6'` | `'arabic'`, `'asmo-708'`, `'csiso88596e'`, `'csiso88596i'`, `'csisolatinarabic'`, `'ecma-114'`, `'iso-8859-6-e'`, `'iso-8859-6-i'`, `'iso-ir-127'`, `'iso8859-6'`, `'iso88596'`, `'iso_8859-6'`, `'iso_8859-6:1987'` |
| `'iso-8859-7'` | `'csisolatingreek'`, `'ecma-118'`, `'elot_928'`, `'greek'`, `'greek8'`, `'iso-ir-126'`, `'iso8859-7'`, `'iso88597'`, `'iso_8859-7'`, `'iso_8859-7:1987'`, `'sun_eu_greek'` |
| `'iso-8859-8'` | `'csiso88598e'`, `'csisolatinhebrew'`, `'hebrew'`, `'iso-8859-8-e'`, `'iso-ir-138'`, `'iso8859-8'`, `'iso88598'`, `'iso_8859-8'`, `'iso_8859-8:1988'`, `'visual'` |
| `'iso-8859-8-i'` | `'csiso88598i'`, `'logical'` |
| `'iso-8859-10'` | `'csisolatin6'`, `'iso-ir-157'`, `'iso8859-10'`, `'iso885910'`, `'l6'`, `'latin6'` |
| `'iso-8859-13'` | `'iso8859-13'`, `'iso885913'` |
| `'iso-8859-14'` | `'iso8859-14'`, `'iso885914'` |
| `'iso-8859-15'` | `'csisolatin9'`, `'iso8859-15'`, `'iso885915'`, `'iso_8859-15'`, `'l9'` |
| `'koi8-r'` | `'cskoi8r'`, `'koi'`, `'koi8'`, `'koi8_r'` |
| `'koi8-u'` | `'koi8-ru'` |
| `'macintosh'` | `'csmacintosh'`, `'mac'`, `'x-mac-roman'` |
| `'windows-874'` | `'dos-874'`, `'iso-8859-11'`, `'iso8859-11'`, `'iso885911'`, `'tis-620'` |
| `'windows-1250'` | `'cp1250'`, `'x-cp1250'` |
| `'windows-1251'` | `'cp1251'`, `'x-cp1251'` |
| `'windows-1252'` | `'ansi_x3.4-1968'`, `'ascii'`, `'cp1252'`, `'cp819'`, `'csisolatin1'`, `'ibm819'`, `'iso-8859-1'`, `'iso-ir-100'`, `'iso8859-1'`, `'iso88591'`, `'iso_8859-1'`, `'iso_8859-1:1987'`, `'l1'`, `'latin1'`, `'us-ascii'`, `'x-cp1252'` |
| `'windows-1253'` | `'cp1253'`, `'x-cp1253'` |
| `'windows-1254'` | `'cp1254'`, `'csisolatin5'`, `'iso-8859-9'`, `'iso-ir-148'`, `'iso8859-9'`, `'iso88599'`, `'iso_8859-9'`, `'iso_8859-9:1989'`, `'l5'`, `'latin5'`, `'x-cp1254'` |
| `'windows-1255'` | `'cp1255'`, `'x-cp1255'` |
| `'windows-1256'` | `'cp1256'`, `'x-cp1256'` |
| `'windows-1257'` | `'cp1257'`, `'x-cp1257'` |
| `'windows-1258'` | `'cp1258'`, `'x-cp1258'` |
| `'x-mac-cyrillic'` | `'x-mac-ukrainian'` |
| `'gbk'` | `'chinese'`, `'csgb2312'`, `'csiso58gb231280'`, `'gb2312'`, `'gb_2312'`, `'gb_2312-80'`, `'iso-ir-58'`, `'x-gbk'` |
| `'gb18030'` | |
| `'big5'` | `'big5-hkscs'`, `'cn-big5'`, `'csbig5'`, `'x-x-big5'` |
| `'euc-jp'` | `'cseucpkdfmtjapanese'`, `'x-euc-jp'` |
| `'iso-2022-jp'` | `'csiso2022jp'` |
| `'shift_jis'` | `'csshiftjis'`, `'ms932'`, `'ms_kanji'`, `'shift-jis'`, `'sjis'`, `'windows-31j'`, `'x-sjis'` |
| `'euc-kr'` | `'cseuckr'`, `'csksc56011987'`, `'iso-ir-149'`, `'korean'`, `'ks_c_5601-1987'`, `'ks_c_5601-1989'`, `'ksc5601'`, `'ksc_5601'`, `'windows-949'` |

#### Encodings supported when Node.js is built with the `small-icu` option

| Encoding | Aliases |
| ------------ | ------------------------------- |
| `'utf-8'` | `'unicode-1-1-utf-8'`, `'utf8'` |
| `'utf-16le'` | `'utf-16'` |
| `'utf-16be'` | |

#### Encodings supported when ICU is disabled

| Encoding | Aliases |
| ------------ | ------------------------------- |
| `'utf-8'` | `'unicode-1-1-utf-8'`, `'utf8'` |
| `'utf-16le'` | `'utf-16'` |

The `'iso-8859-16'` encoding listed in the [WHATWG Encoding Standard][]
is not supported.

### `new TextDecoder([encoding[, options]])`

* `encoding` {string} Identifies the `encoding` that this `TextDecoder` instance
supports. **Default:** `'utf-8'`.
* `options` {Object}
* `fatal` {boolean} `true` if decoding failures are fatal.
This option is not supported when ICU is disabled
(see [Internationalization][]). **Default:** `false`.
* `ignoreBOM` {boolean} When `true`, the `TextDecoder` will include the byte
order mark in the decoded result. When `false`, the byte order mark will
be removed from the output. This option is only used when `encoding` is
`'utf-8'`, `'utf-16be'`, or `'utf-16le'`. **Default:** `false`.

Creates a new `TextDecoder` instance. The `encoding` may specify one of the
supported encodings or an alias.

The `TextDecoder` class is also available on the global object.

### `textDecoder.decode([input[, options]])`

* `input` {ArrayBuffer|DataView|TypedArray} An `ArrayBuffer`, `DataView`, or
`TypedArray` instance containing the encoded data.
* `options` {Object}
* `stream` {boolean} `true` if additional chunks of data are expected.
**Default:** `false`.
* Returns: {string}

Decodes the `input` and returns a string. If `options.stream` is `true`, any
incomplete byte sequences occurring at the end of the `input` are buffered
internally and emitted after the next call to `textDecoder.decode()`.

If `textDecoder.fatal` is `true`, decoding errors that occur will result in a
`TypeError` being thrown.

### `textDecoder.encoding`

* Type: {string}

The encoding supported by the `TextDecoder` instance.

### `textDecoder.fatal`

* Type: {boolean}

The value will be `true` if decoding errors result in a `TypeError` being
thrown.

### `textDecoder.ignoreBOM`

* Type: {boolean}

The value will be `true` if the decoding result will include the byte order
mark.

## Class: `TextDecoderStream`

Expand All @@ -1214,7 +1340,51 @@ A browser-compatible implementation of [`TextDecoderStream`][].
added: v11.0.0
-->

The WHATWG `TextEncoder` class. See the [`TextEncoder`][] section.
An implementation of the [WHATWG Encoding Standard][] `TextEncoder` API. All
instances of `TextEncoder` only support UTF-8 encoding.

```js
const encoder = new TextEncoder();
const uint8array = encoder.encode('this is some data');
```

The `TextEncoder` class is also available on the global object.

### `textEncoder.encode([input])`

* `input` {string} The text to encode. **Default:** an empty string.
* Returns: {Uint8Array}

UTF-8 encodes the `input` string and returns a `Uint8Array` containing the
encoded bytes.

### `textEncoder.encodeInto(src, dest)`

<!-- YAML
added: v12.11.0
-->

* `src` {string} The text to encode.
* `dest` {Uint8Array} The array to hold the encode result.
* Returns: {Object}
* `read` {number} The read Unicode code units of src.
* `written` {number} The written UTF-8 bytes of dest.

UTF-8 encodes the `src` string to the `dest` Uint8Array and returns an object
containing the read Unicode code units and written UTF-8 bytes.

```js
const encoder = new TextEncoder();
const src = 'this is some data';
const dest = new Uint8Array(10);
const { read, written } = encoder.encodeInto(src, dest);
```

### `textEncoder.encoding`

* Type: {string}

The encoding supported by the `TextEncoder` instance. Always set to `'utf-8'`.

## Class: `TextEncoderStream`

Expand Down Expand Up @@ -1480,9 +1650,11 @@ A browser-compatible implementation of [`WritableStreamDefaultWriter`][].
[CommonJS modules]: modules.md
[ECMAScript module]: esm.md
[HTML Standard]: https://html.spec.whatwg.org/multipage/workers.html
[Internationalization]: intl.md
[Navigator API]: https://html.spec.whatwg.org/multipage/system-state.html#the-navigator-object
[RFC 5646]: https://www.rfc-editor.org/rfc/rfc5646.txt
[Refable protocol]: process.md#processrefmayberefable
[WHATWG Encoding Standard]: https://encoding.spec.whatwg.org/
[Web Crypto API]: webcrypto.md
[`--experimental-eventsource`]: cli.md#--experimental-eventsource
[`--localstorage-file`]: cli.md#--localstorage-filefile
Expand Down Expand Up @@ -1514,9 +1686,7 @@ A browser-compatible implementation of [`WritableStreamDefaultWriter`][].
[`Request`]: #class-request
[`Response`]: #class-response
[`TextDecoderStream`]: webstreams.md#class-textdecoderstream
[`TextDecoder`]: util.md#class-utiltextdecoder
[`TextEncoderStream`]: webstreams.md#class-textencoderstream
[`TextEncoder`]: util.md#class-utiltextencoder
[`TransformStreamDefaultController`]: webstreams.md#class-transformstreamdefaultcontroller
[`TransformStream`]: webstreams.md#class-transformstream
[`URLPattern`]: url.md#class-urlpattern
Expand Down
6 changes: 3 additions & 3 deletions doc/api/intl.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ programs. Some of them are:
* The [WHATWG URL parser][]'s [internationalized domain names][] (IDNs) support
* [`require('node:buffer').transcode()`][]
* More accurate [REPL][] line editing
* [`require('node:util').TextDecoder`][]
* [`TextDecoder`][]
* [`RegExp` Unicode Property Escapes][]

Node.js and the underlying V8 engine use
Expand Down Expand Up @@ -57,7 +57,7 @@ option:
| [WHATWG URL Parser][] | partial (no IDN support) | full | full | full |
| [`require('node:buffer').transcode()`][] | none (function does not exist) | full | full | full |
| [REPL][] | partial (inaccurate line editing) | full | full | full |
| [`require('node:util').TextDecoder`][] | partial (basic encodings support) | partial/full (depends on OS) | partial (Unicode-only) | full |
| [`TextDecoder`][] | partial (basic encodings support) | partial/full (depends on OS) | partial (Unicode-only) | full |
| [`RegExp` Unicode Property Escapes][] | none (invalid `RegExp` error) | full | full | full |

The "(not locale-aware)" designation denotes that the function carries out its
Expand Down Expand Up @@ -231,8 +231,8 @@ to be helpful:
[`String.prototype.normalize()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/normalize
[`String.prototype.toLowerCase()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/toLowerCase
[`String.prototype.toUpperCase()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/toUpperCase
[`TextDecoder`]: globals.md#class-textdecoder
[`require('node:buffer').transcode()`]: buffer.md#buffertranscodesource-fromenc-toenc
[`require('node:util').TextDecoder`]: util.md#class-utiltextdecoder
[btest402]: https://github.com/srl295/btest402
[full-icu]: https://www.npmjs.com/package/full-icu
[internationalized domain names]: https://en.wikipedia.org/wiki/Internationalized_domain_name
4 changes: 2 additions & 2 deletions doc/api/module.md
Original file line number Diff line number Diff line change
Expand Up @@ -1036,7 +1036,7 @@ not possible to replace the value of a Node.js builtin (core) module.
* The specific {TypedArray} object is a {Uint8Array}.
If the source value of a text-based format (i.e., `'json'`, `'module'`)
is not a string, it is converted to a string using [`util.TextDecoder`][].
is not a string, it is converted to a string using [`TextDecoder`][].
### Examples
Expand Down Expand Up @@ -1413,6 +1413,7 @@ returned object contains the following keys:
[`NODE_V8_COVERAGE=dir`]: cli.md#node_v8_coveragedir
[`Object.freeze()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze
[`SourceMap`]: #class-modulesourcemap
[`TextDecoder`]: globals.md#class-textdecoder
[`module.constants.compileCacheStatus`]: #moduleconstantscompilecachestatus
[`module.enableCompileCache()`]: #moduleenablecompilecacheoptions
[`module.flushCompileCache()`]: #moduleflushcompilecache
Expand All @@ -1421,7 +1422,6 @@ returned object contains the following keys:
[`module.registerHooks()`]: #moduleregisterhooksoptions
[`module.setSourceMapsSupport()`]: #modulesetsourcemapssupportenabled-options
[`os.tmpdir()`]: os.md#ostmpdir
[`util.TextDecoder`]: util.md#class-utiltextdecoder
[accepted final formats]: #accepted-final-formats-returned-by-load
[deregistration of customization hooks]: #deregistration-of-customization-hooks
[hook functions]: #hook-functions-accepted-by-moduleregisterhooks
Expand Down
Loading
Loading