diff options
author | Jessica Quynh Tran <jessica.quynh.tran@gmail.com> | 2017-06-03 23:11:32 +0300 |
---|---|---|
committer | Myles Borins <mylesborins@google.com> | 2017-09-12 04:18:28 +0300 |
commit | c90c68e8a01a8edcec8a2cb5d70cddbf349113ba (patch) | |
tree | 9abf41f2067b5285e3865a13580176006eb17c3c /doc/api/stream.md | |
parent | 4882f079f126e094443e1d962ca9857cc3e10a20 (diff) |
doc: document bytes to chars after setEncoding
This commit documents and edge-case behavior in readable streams.
It is expected that non-object streams are measured in bytes
against the highWaterMark. However, it was discovered in issue
thereafter begin to measure the buffer's length in characters.
PR-URL: https://github.com/nodejs/node/pull/13442
Refs: https://github.com/nodejs/node/issues/6798
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Diffstat (limited to 'doc/api/stream.md')
-rw-r--r-- | doc/api/stream.md | 25 |
1 files changed, 20 insertions, 5 deletions
diff --git a/doc/api/stream.md b/doc/api/stream.md index efa69532ef8..f635558707b 100644 --- a/doc/api/stream.md +++ b/doc/api/stream.md @@ -68,8 +68,8 @@ buffer that can be retrieved using `writable._writableState.getBuffer()` or The amount of data potentially buffered depends on the `highWaterMark` option passed into the streams constructor. For normal streams, the `highWaterMark` -option specifies a total number of bytes. For streams operating in object mode, -the `highWaterMark` specifies a total number of objects. +option specifies a [total number of bytes][hwm-gotcha]. For streams operating +in object mode, the `highWaterMark` specifies a total number of objects. Data is buffered in Readable streams when the implementation calls [`stream.push(chunk)`][stream-push]. If the consumer of the Stream does not @@ -1517,9 +1517,9 @@ constructor and implement the `readable._read()` method. #### new stream.Readable([options]) * `options` {Object} - * `highWaterMark` {number} The maximum number of bytes to store in - the internal buffer before ceasing to read from the underlying - resource. Defaults to `16384` (16kb), or `16` for `objectMode` streams + * `highWaterMark` {number} The maximum [number of bytes][hwm-gotcha] to store + in the internal buffer before ceasing to read from the underlying resource. + Defaults to `16384` (16kb), or `16` for `objectMode` streams * `encoding` {string} If specified, then buffers will be decoded to strings using the specified encoding. Defaults to `null` * `objectMode` {boolean} Whether this stream should behave @@ -2157,6 +2157,19 @@ object mode has an interesting side effect. Because it *is* a call to However, because the argument is an empty string, no data is added to the readable buffer so there is nothing for a user to consume. +### `highWaterMark` discrepency after calling `readable.setEncoding()` + +The use of `readable.setEncoding()` will change the behavior of how the +`highWaterMark` operates in non-object mode. + +Typically, the size of the current buffer is measured against the +`highWaterMark` in _bytes_. However, after `setEncoding()` is called, the +comparison function will begin to measure the buffer's size in _characters_. + +This is not a problem in common cases with `latin1` or `ascii`. But it is +advised to be mindful about this behavior when working with strings that could +contain multi-byte characters. + [`'data'`]: #stream_event_data [`'drain'`]: #stream_event_drain [`'end'`]: #stream_event_end @@ -2195,6 +2208,8 @@ readable buffer so there is nothing for a user to consume. [fs write streams]: fs.html#fs_class_fs_writestream [http-incoming-message]: http.html#http_class_http_incomingmessage [zlib]: zlib.html +[hwm-gotcha]: #stream_highWaterMark_discrepency_after_calling_readable_setencoding +[Readable]: #stream_class_stream_readable [stream-_flush]: #stream_transform_flush_callback [stream-_read]: #stream_readable_read_size_1 [stream-_transform]: #stream_transform_transform_chunk_encoding_callback |