In this lab we will put together a simple shell. We will interact with the file system and network while learning some useful features of JavaScript.

• Working with standard input
• Implementing a print working directory command
• Interacting with the filesystem: reading a directory
• Interacting with HTTP: downloading a file

Commands will be provided to our shell through the process' standard input. By default, Node.js does not activate the standard input stream. The first thing we will do is enable standard input and echo back what we read.

Create a new file called shell.js in a brand new directory and add the following:

{% highlight javascript %} process.stdin.on('data', function (input) { console.log(input); }); {% endhighlight %}

Before we go any further, let's experiment with what the above code does. To run the file:

{% highlight bash %} $ node shell.js {% endhighlight %}

Type anything and press enter. Notice that the input is line buffered (the data is sent in to Node.js every time you press enter). To shut down the process press CTRL+C.

The output of your program might look something like this:

{% highlight javascript %} foo <Buffer 61 73 64 0a> bar <Buffer 62 61 72 0a> {% endhighlight %}

You can see that we're printing out a Buffer object. That's because the input variable passed in to our function (input) { ... } callback does not contain the string value of your input directly.

It's worth noting that, at this point, the buffer exists completely outside of the JavaScript memory space (Buffer is an object from the C++ Node.js internals). Interacting with this buffer will move data from between the native (C++) and JavaScript boundary. For example, calling input.toString() will create a new JavaScript string containing the entire contents of the Buffer. An optional encoding can be specified as the first argument of the toString function (for example, input.toString('utf8')).

Since we're working with relatively short strings, let's go ahead and call input.toString() on the Buffer object. Here's what your program should look like now:

{% highlight javascript %} process.stdin.on('data', function (input) { console.log(input.toString()); }); {% endhighlight %}

Now starting up the shell and typing any value will result in the expected output ending with the new line character.

The next step is to trim and parse the input string. The commands in our simple shell will take the form:

{% highlight bash %} command [args...] {% endhighlight %}

We can use a handy regular expression to separate the arguments from the command: /(\w+)(.*)/. We will then parse our arguments by splitting on white space.

{% highlight javascript %} process.stdin.on('data', function (input) { var matches = input.toString().match(/(\w+)(.*)/); var command = matches[1].toLowerCase(); var args = matches[2].trim().split(/\s+/); }); {% endhighlight %}

The result of 'some string'.split(/\s+/) is an array ['some', 'string'].

This example could have been done with 'some string'.split(' '), except that would not account for other types of white space or multiple white space characters. For example:

'some  string'.split(' ') would result in ['some', '', 'string'].

Feel free to check out the result of the above code block by logging out the value of command and args. You may want to add a little more logic to make this resilient to malformed input. We will leave that exercise up to you.

pwd is a simple program to print out the current working directory. Let's implement this in our shell.

{% highlight javascript %} var commands = { 'pwd': function () { console.log(process.cwd()); } };

process.stdin.on('data', function (input) { var matches = input.toString().match(/(\w+)(.*)/); var command = matches[1].toLowerCase(); var args = matches[2].trim().split(/\s+/);

commandscommand; }); {% endhighlight %}

To clarify what's happening above, here's sample output of executing the regular expression at the Node.js REPL. The input is cmd_name arg1 arg2.

{% highlight javascript %}

var input = 'cmd_name arg1 arg2' 'cmd_name arg1 arg2' var matches = input.match(/(\w+)(.*)/) matches [ 'cmd_name arg1 arg2', // matches[0] 'cmd_name', // matches[1] ' arg1 arg2', // matches[2] index: 0, // matches[3] input: 'cmd_name arg1 arg2'] // matches[4] {% endhighlight %}

We are accessing matches[1] because it's the first group (groups are specified with the parenthesis). If you are unfamiliar with regular expressions, a good source to learn more is at http://regular-expressions.info.

Now, give your shell a try!

Start up the shell with the node command and execute our one and only command:

{% highlight bash %} $ node shell.js pwd /users/you/simple-shell/ {% endhighlight %}

The command ls [directory] prints the contents of a directory. If the directory argument is not specified it will print the contents of the current working directory.

First, we will import the fs module at the top of the file. The fs module is the Node.js core module for file system operations.

{% highlight javascript %} var fs = require('fs'); {% endhighlight %}

To implement ls, add a new property to our commands object named 'ls':

{% highlight javascript %} var commands = { 'pwd': function () { console.log(process.cwd()); }, // <----------------------- Don't forget this comma! 'ls': function (args) { // New property added here. fs.readdir(args[0] || process.cwd(), function (err, entries) { entries.forEach(function (e) { console.log(e); }); }); } }; {% endhighlight %}

Let's talk for a moment about args[0] || process.cwd():

Unlike many other languages, JavaScript doesn't care if you access an index out of bounds of an array. If an element does not exist at the requested index, undefined will be returned. Since undefined is considered false, Using the x || y syntax will test the existence of x and if it is false (doesn't exist), it will evaluate to y. This is a common pattern for assigning a default value.

There are plenty of other commands available to access the file system. Feel free to peruse the documentation and implement any other commands that interest you.

Here's what your program should look like:

{% highlight javascript %} var fs = require('fs');

var commands = { 'pwd': function () { console.log(process.cwd()); }, 'ls': function (args) { fs.readdir(args[0] || process.cwd(), function (err, entries) { entries.forEach(function (e) { console.log(e); }); }); } };

process.stdin.on('data', function (input) { var matches = input.toString().match(/(\w+)(.*)/); var command = matches[1].toLowerCase(); var args = matches[2].trim().split(/\s+/);

commandscommand; }); {% endhighlight %}

The commands object can get a bit hairy when we're nesting so many brackets deep. We can instead define our commands object like so:

{% highlight javascript %} var commands = {};

commands['pwd'] = function () { console.log(process.cwd()); };

commands['ls'] = function (args) { fs.readdir(args[0] || process.cwd(), function (err, entries) { entries.forEach(function (e) { console.log(e); }); }); }; {% endhighlight %}

In this example we're defining a container object commands and assigning anonymous functions to properties on that object. The major difference from before is that the function definitions are not part of the object initializer. There's no difference in behavior between the two approaches.

Similarly to the fs module, Node.js core contains a http module which can be used to act as a HTTP client or server. In the next lab you will be creating a HTTP server, but for now, we will focus on creating a simple wget command to download a file.

Import the http module up top next to where you imported fs:

{% highlight javascript %} var http = require('http'); {% endhighlight %}

Now, we will use http.get(url, callback) to make a HTTP GET request and download the results to the file system.

{% highlight javascript %} var commands = { 'pwd': ... , // omitted for brevity 'ls': ... , // omitted for brevity 'wget': function (args) { var url = args[0]; var file = args[1] || 'download';

http.get(url, function (res) {
  var writeStream = fs.createWriteStream(file);
  res.pipe(writeStream);

  res.on('end', function () {
    console.log(url + ' downloaded to file \'' + file + '\'');
  });
});

} }; {% endhighlight %}

Now in order to test this try to run the following: (it's very important to include http://)

node shell.js wget http://google.com

The above command will download the contents of the home page of google to download.

Let's talk about what's happening in the callback provided to http.get callback.

First, we're creating a writable stream to our file system using fs.createWriteStream, named file (the second argument, or 'download' by default). This will create or overwrite the file.

Next, let's talk about res.pipe(writeStream);. The res (response) object given to us isn't the full HTTP response. Since the response may be large, Node.js gives us the response as soon as it can be useful: after the HTTP handshake has occurred. We have access to the complete set of HTTP headers at this point, but do not have access to the data from the response.

The response data is streamed, much like the standard input stream we opened to read commands from the terminal. The res object emits res.on('data') and res.on('end') events, which can be directly piped out to our writable stream, saving chunks of data to the file system and then ending the write.

If you get stuck, check the manual on the HTTP module.