{"id":83,"date":"2019-09-23T13:35:11","date_gmt":"2019-09-23T13:35:11","guid":{"rendered":"https:\/\/www.kindsonthegenius.com\/nodejs\/?p=83"},"modified":"2020-08-06T09:56:18","modified_gmt":"2020-08-06T09:56:18","slug":"node-js-streams","status":"publish","type":"post","link":"https:\/\/kindsonthegenius.com\/nodejs\/node-js-streams\/","title":{"rendered":"Node.js -Streams"},"content":{"rendered":"<p>We would cover Streams in Node.js under the following sub-topics:<\/p>\n<ol>\n<li><a href=\"#t1\">Introduction to Streams in Node.js<\/a><\/li>\n<li><a href=\"#t2\">Reading from Streams<\/a><\/li>\n<li><a href=\"#t3\">Writing to Streams<\/a><\/li>\n<li><a href=\"#t4\">Piping a Stream<\/a><\/li>\n<li><a href=\"#t5\">Chaining Streams<\/a><\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<p><strong id=\"t1\">1. Introduction to Streams in Node.js<\/strong><\/p>\n<p>In Node.js, streams are used to read data from a source to a destination in a continuous manner. We would learn of four types of streams in Node.js. They are:<\/p>\n<ul>\n<li><b>Readable<\/b> \u2212 This is a stream\u00a0used for read operation.<\/li>\n<li><b>Writable<\/b>\u00a0\u2212 Stream which is used for write operation.<\/li>\n<li><b>Duplex<\/b> \u2212 This can be used for both read and write operations.<\/li>\n<li><b>Transform<\/b> \u2212 A type of duplex stream where the output is determined based on the input.<\/li>\n<\/ul>\n<p>Each of the type so streams listed above is an instance of EvenEmitter (<a href=\"https:\/\/kindsonthegenius.com\/nodejs\/node-js-eventemitter\/\" target=\"_blank\" rel=\"noopener noreferrer\">EventEmitters were discussed in Part 8<\/a>). They can fire several events at different times. For instance some of the most common events are:<\/p>\n<ul>\n<li><b>data<\/b> \u2212 This is fired whenever there is data is available to read.<\/li>\n<li><b>end<\/b> \u2212 This is fired whenever there is no more data to read.<\/li>\n<li><b>error<\/b> \u2212 This is fired whenever there is any error receiving or writing data.<\/li>\n<li><b>finish<\/b> \u2212 This is fired whenever all the data has been flushed to underlying system.<\/li>\n<\/ul>\n<p>Let&#8217;s now examine the operations you can perform with streams. We begin with reading.<\/p>\n<p>&nbsp;<\/p>\n<p><strong id=\"t2\">2. Reading from a Stream<\/strong><\/p>\n<p>I have created a file named testinput.txt. The content is as follows:<br \/>\n<!-- HTML generated using hilite.me --><\/p>\n<pre style=\"margin: 0; line-height: 125%;\">Find all tutorial in kindsonthegenius.com\r\nAlso watch the video lessons.\r\n<\/pre>\n<p>Then, in VS Code, write and run the following program:<\/p>\n<pre style=\"margin: 0; line-height: 125%;\"><span style=\"color: #008800; font-weight: bold;\">var<\/span> fs <span style=\"color: #333333;\">=<\/span> require(<span style=\"background-color: #fff0f0;\">\"fs\"<\/span>);\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> data <span style=\"color: #333333;\">=<\/span> <span style=\"background-color: #fff0f0;\">''<\/span>;\r\n\r\n<span style=\"color: #888888;\">\/\/ Create a readable stream object<\/span>\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> readStream <span style=\"color: #333333;\">=<\/span> fs.createReadStream(<span style=\"background-color: #fff0f0;\">'D:\/nodefiles\/testinput.txt'<\/span>);\r\n\r\n<span style=\"color: #888888;\">\/\/ Set the character encoding to be utf8. <\/span>\r\nreadStream.setEncoding(<span style=\"background-color: #fff0f0;\">'UTF8'<\/span>);\r\n\r\n<span style=\"color: #888888;\">\/\/ Handle stream data event<\/span>\r\nreadStream.on(<span style=\"background-color: #fff0f0;\">'data'<\/span>, <span style=\"color: #008800; font-weight: bold;\">function<\/span>(chunk) {\r\n   data <span style=\"color: #333333;\">+=<\/span> chunk;\r\n});\r\n\r\n<span style=\"color: #888888;\">\/\/ Handle the end event<\/span>\r\nreadStream.on(<span style=\"background-color: #fff0f0;\">'end'<\/span>,<span style=\"color: #008800; font-weight: bold;\">function<\/span>() {\r\n   console.log(data);\r\n});\r\n\r\n<span style=\"color: #888888;\">\/\/Handle the error event<\/span>\r\nreadStream.on(<span style=\"background-color: #fff0f0;\">'error'<\/span>, <span style=\"color: #008800; font-weight: bold;\">function<\/span>(err) {\r\n   console.log(err.stack);\r\n});\r\n\r\nconsole.log(<span style=\"background-color: #fff0f0;\">\"End of Program\"<\/span>);\r\n<\/pre>\n<p>&nbsp;<\/p>\n<p>In the program above we created and stream. Then we read from the stream.<\/p>\n<p>&nbsp;<\/p>\n<p><strong id=\"t3\">3. Writing to a Stream<\/strong><\/p>\n<p>Now, we are going write to a stream. The code below create a writable stream. The it writes some text into it.<\/p>\n<pre style=\"margin: 0; line-height: 125%;\"><span style=\"color: #008800; font-weight: bold;\">var<\/span> fs <span style=\"color: #333333;\">=<\/span> require(<span style=\"background-color: #fff0f0;\">\"fs\"<\/span>);\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> data <span style=\"color: #333333;\">=<\/span> <span style=\"background-color: #fff0f0;\">'kindsonthegenius.com is the best place to learn'<\/span>;\r\n\r\n<span style=\"color: #888888;\">\/\/ Create a writable stream<\/span>\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> writeStream <span style=\"color: #333333;\">=<\/span> fs.createWriteStream(<span style=\"background-color: #fff0f0;\">'D:\/nodefiles\/testoutput.txt'<\/span>);\r\n\r\n<span style=\"color: #888888;\">\/\/ Write the data to the stream with character encoding to be utf8<\/span>\r\nwriteStream.write(data,<span style=\"background-color: #fff0f0;\">'UTF8'<\/span>);\r\n\r\n<span style=\"color: #888888;\">\/\/ Indicate the end of file<\/span>\r\nwriteStream.end();\r\n\r\n<span style=\"color: #888888;\">\/\/ Handle stream finis event<\/span>\r\nwriteStream.on(<span style=\"background-color: #fff0f0;\">'finish'<\/span>, <span style=\"color: #008800; font-weight: bold;\">function<\/span>() {\r\n   console.log(<span style=\"background-color: #fff0f0;\">\"Data was written succesfully.\"<\/span>);\r\n});\r\n\r\n<span style=\"color: #888888;\">\/\/ Handle the Error event<\/span>\r\nwriteStream.on(<span style=\"background-color: #fff0f0;\">'error'<\/span>, <span style=\"color: #008800; font-weight: bold;\">function<\/span>(err) {\r\n   console.log(<span style=\"background-color: #fff0f0;\">'Error Occured: '<\/span> <span style=\"color: #333333;\">+<\/span> err.stack);\r\n});\r\n\r\nconsole.log(<span style=\"background-color: #fff0f0;\">\"End of Program\"<\/span>);\r\n<\/pre>\n<p>&nbsp;<\/p>\n<p>This program creates the file specified. Then open it and writes into it. So run this code, then check the directory to see that a new file was created.<\/p>\n<p>Also introduce some error, just to make it fire the error event.<\/p>\n<p>&nbsp;<\/p>\n<p><strong id=\"t4\">4. Piping a Stream to another Stream<\/strong><\/p>\n<p>Sometimes a pipe and a stream works together. You use a pipe to connect an output of a stream as input to another stream. This called piping. Now, you can pipe as many streams as you want. In the code below, we would read data from a readable stream and write it into another stream.<\/p>\n<p>In the code below, we create two streams: readStream and writeStream. Then we pipe the readStream into the writeStream.<\/p>\n<pre style=\"margin: 0; line-height: 125%;\"><span style=\"color: #888888;\">\/\/ Program to demonstrate Piping operation<\/span>\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> fs <span style=\"color: #333333;\">=<\/span> require(<span style=\"background-color: #fff0f0;\">\"fs\"<\/span>);\r\n\r\n<span style=\"color: #888888;\">\/\/ Create a readable stream object<\/span>\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> readerStream <span style=\"color: #333333;\">=<\/span> fs.createReadStream(<span style=\"background-color: #fff0f0;\">'D:\/nodefiles\/testinput.txt'<\/span>);\r\n\r\n<span style=\"color: #888888;\">\/\/ Create a writable stream object<\/span>\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> writerStream <span style=\"color: #333333;\">=<\/span> fs.createWriteStream(<span style=\"background-color: #fff0f0;\">'D:\/nodefiles\/testoutput.txt'<\/span>);\r\n\r\n<span style=\"color: #888888;\">\/\/ Pipe the readstream into the writeStream<\/span>\r\nreaderStream.pipe(writerStream);\r\n\r\nconsole.log(<span style=\"background-color: #fff0f0;\">\"End of Program\"<\/span>);\r\n<\/pre>\n<p>&nbsp;<\/p>\n<p><strong id=\"t5\">5. Chaining Streams<\/strong><\/p>\n<p>Similar to piping, is another mechanism called chaining.<\/p>\n<p>Chaining allows you to connect the output of a stream to another stream. Therefore, you can chain multiple stream operations. Oftentimes, chaining is used along with piping.<\/p>\n<p>For example, we are going to use piping and chaining to first compress a file. Then we decompress the same file.<\/p>\n<p>&nbsp;<\/p>\n<p>The code is given below:<br \/>\n<!-- HTML generated using hilite.me --><\/p>\n<pre style=\"margin: 0; line-height: 125%;\"><span style=\"color: #008800; font-weight: bold;\">var<\/span> fs <span style=\"color: #333333;\">=<\/span> require(<span style=\"background-color: #fff0f0;\">\"fs\"<\/span>);\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> zlib <span style=\"color: #333333;\">=<\/span> require(<span style=\"background-color: #fff0f0;\">'zlib'<\/span>); <span style=\"color: #888888;\">\/\/compression module<\/span>\r\n\r\n<span style=\"color: #888888;\">\/\/ Compress the file testinput.txt to testinput.txt.gz<\/span>\r\nfs.createReadStream(<span style=\"background-color: #fff0f0;\">'D:\/nodefiles\/testinput.txt'<\/span>)\r\n   .pipe(zlib.createGzip())\r\n   .pipe(fs.createWriteStream(<span style=\"background-color: #fff0f0;\">'D:\/nodefiles\/testinput.txt.gz'<\/span>));\r\n  \r\nconsole.log(<span style=\"background-color: #fff0f0;\">\"File was Compressed.\"<\/span>);\r\n<\/pre>\n<p>&nbsp;<\/p>\n<p>If you run the code above succefully, then it would compress the specified file. Now you can check the directory. You&#8217;ll notice that a compressed file has been created. The folder in my system is shown below.<\/p>\n<p><a href=\"https:\/\/kindsonthegenius.com\/nodejs\/wp-content\/uploads\/2019\/09\/Streams-in-Node.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-85\" src=\"https:\/\/kindsonthegenius.com\/nodejs\/wp-content\/uploads\/2019\/09\/Streams-in-Node.jpg\" alt=\"Streams in Node.js\" width=\"581\" height=\"329\" srcset=\"https:\/\/kindsonthegenius.com\/nodejs\/wp-content\/uploads\/2019\/09\/Streams-in-Node.jpg 682w, https:\/\/kindsonthegenius.com\/nodejs\/wp-content\/uploads\/2019\/09\/Streams-in-Node-300x170.jpg 300w\" sizes=\"auto, (max-width: 581px) 100vw, 581px\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>Now, we would run a code to decompress the same file. The code is given below:<\/p>\n<pre style=\"margin: 0; line-height: 125%;\"><span style=\"color: #008800; font-weight: bold;\">var<\/span> fs <span style=\"color: #333333;\">=<\/span> require(<span style=\"background-color: #fff0f0;\">\"fs\"<\/span>);\r\n<span style=\"color: #008800; font-weight: bold;\">var<\/span> zlib <span style=\"color: #333333;\">=<\/span> require(<span style=\"background-color: #fff0f0;\">'zlib'<\/span>);\r\n\r\n<span style=\"color: #888888;\">\/\/ Decompress the file testinput.txt.gz to testinput.txt<\/span>\r\nfs.createReadStream(<span style=\"background-color: #fff0f0;\">'D:\/nodefiles\/testinput.txt.gz'<\/span>)\r\n   .pipe(zlib.createGunzip())\r\n   .pipe(fs.createWriteStream(<span style=\"background-color: #fff0f0;\">'D:\/nodefiles\/testinput.txt'<\/span>));\r\n  \r\nconsole.log(<span style=\"background-color: #fff0f0;\">\"File was Decompressed.\"<\/span>);\r\n<\/pre>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We would cover Streams in Node.js under the following sub-topics: Introduction to Streams in Node.js Reading from Streams Writing to Streams Piping a Stream Chaining &hellip; <\/p>\n","protected":false},"author":1,"featured_media":86,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-83","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/posts\/83","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/comments?post=83"}],"version-history":[{"count":3,"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/posts\/83\/revisions"}],"predecessor-version":[{"id":123,"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/posts\/83\/revisions\/123"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/media\/86"}],"wp:attachment":[{"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/media?parent=83"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/categories?post=83"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kindsonthegenius.com\/nodejs\/wp-json\/wp\/v2\/tags?post=83"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}