logo_nodejs


NodeJS is an open source, asynchronous-event driven, non blocking I/O (input/output) JavaScript runtime environment. It is available for anyone to review and even edit on GitHub. Most developers, including myself until recently, hardly ever explore the source code. This is quite surprising since you can easily download the code for free and even make changes to it. Doing this will give you valuable insights into the inner workings of NodeJS and it will also make you a much better developer. It will also save you from a lot of frustration and hours of wasted effort.

This is a 4 part article organized as follows:

  1. Introduction to NodeJS and the V8 Engine, Modules, Export and Require
  2. Events and Event Emitter, asynchronous code, libuv, streams
  3. HTTP server
  4. Node Package Manager (NPM), Express and MEAN stack overview

Introduction to NodeJS
The fundamental concept behind NodeJS is to bring JavaScript to the server side. For many years JavaScript has been well known as a client side (browser) scritpting language. On the other hand, server side technology is a totally different ball game. Developing code for the client or browser normally involves dealing with multiple platforms, displaying images, text and video etc. In contrast, the server side is concerned with accessing databases, managing user sessions, implementing complex business logic and also managing network traffic. In order for JavaScript to make this transition successfully certain features need to be added.

  1. Should be able to better organize code into reusable pieces
  2. Should be able to access databases
  3. Should be able to handle HTTP and other internet protocols
  4. Should be able to read/write files on the server’s OS
  5. Should be able to handle requests and send responses

Let’s start by looking at the source code on GitHub. The NodeJS repository has 3 very important folders:

  1. node/deps: contains all dependencies e.g. V8, openSSL, http_parser
  2. node/lib: the Javascript core including files like module.js, http.js, _http_server.js
  3. node/src: the C++ wrappers e.g. fs_even_wrap.cc and node.cc

node/deps
NodeJS is made up of several other libraries such as V8, libuv, OpenSSL etc. Out of these, the most important is the V8 engine developed by Google to power the Chrome browser. The NodeJS core consists of two main components – the C++ core and the JavaScript core.
The V8 Engine is written in C++, it complies with ECMA-262 and it basically interprets Javascript code into a lower level machine code that your computer can understand. It also enables any C++ application to have access to Javascript. This design feature is what allows NodeJS to extend Javascript features to the server side.
The V8 code itself implements all the Javascript language features such as objects, functions, data types and operators. It defines how functions like JSON parser and JSON stringify should work.(json-parser.cc, json-parser.h, json-stringify.cc, json-stringify.h in the deps/v8/src folder). If you want more details about V8 visit the Google Developer V8 reference

node/lib
This folder contains the NodeJS Javascript core which includes the wrappers for all the new features not found in the original Javascript language. In other words the wrappers allow us to extend Javascript with native C++ functionality. All the wrappers use process.binding() to expose the internal C++ functions. This folder also contains the V8 module with APIs from the V8 Engine such as getHeapSpaceStatistics() which returns details on the current heap space. There’s also code for handling streams and buffers, features that are critical for web servers. These will be discussed further in the next blog post.

node/src
This folder contains the C++ NodeJS core which includes the C++ hooks or access points for all JavaScript wrappers in the node/lib folder described above. Since NodeJS provides C++ functions to JavaScript, the files in this folder contain the C++ functions and objects that can be accessed through NodeJS using JavaScript. The files here are in pairs the header files with .h extension and the C++ files have the .cc extension. Some of the functionality provided to NodeJS from C++ include streams, buffers and http_parser. (http_parser is a C utility that interprets a stream of bytes as HTTP headers, body encoding etc). There’s also code to handle other internet protocols like TCP (Transport Control Protocol) and UDP (User Datagram Protocol). In simple terms these protocols allow NodeJS server to handle requests and responses on the internet.

Modules with require and exports
How does NodeJS allow you to create reusable code? How can you organize your code in large complex applications? NodeJS was designed in accordance with the CommonJS standard. This standard specification was developed to address how an application’s modules should be defined in order to be reusable. There are other alternative standards such as AMD (Asynchronous Module Definition). By using modules, it is possible to organize code into smaller sections that can communicate and share data with each other. Even though modules help to create a better structure, they also introduce dependencies. Meaning that a module will need to access code stored in another module before it can perform any tasks. This is where require and exports come into the picture.

Now, let us take a peek at the node > lib > module.js file. Most of the dependencies are imported using require(), a few use process.binding() to access features in the C++ core. Basically, NodeJS leverages the V8 JavaScript engine which provides hooks for adding new functionality.

If you take a closer look at the code, you will notice the Module function which has an exports property.

function Module (id, parent){
...
...
this.exports = {};
...
...
}

This Module function then has some more properties added to its prototype, using Javascript’s prototype inheritance. Some important properties are
Module.prototype.load – this loads the file and appends the .js extension if not provided
Module.prototype.require – which in turn calls the Module._load function
Module.wrap function – this is perhaps the most important because it creates a wrapper function around the code in your module. It uses the compiledWrapper and runInThisContext functions.

var wrapper = Module.wrap(content);

  var compiledWrapper = vm.runInThisContext(wrapper, {
    filename: filename,
    lineOffset: 0,
    displayErrors: true
  });
...
...
...

var dirname = path.dirname(filename);
  var require = internalModule.makeRequireFunction.call(this);
  var args = [this.exports, require, this, filename, dirname];
  var depth = internalModule.requireDepth;
  if (depth === 0) stat.cache = new Map();
  var result = compiledWrapper.apply(this.exports, args);

The compiledWrapper function is invoked in line 17 above using Javascript’s built-in apply method. The arguments are defined in line 14: exports and require are included.

What is exports?
As the name implies the exports property exposes parts of the module to the rest of the application. Normally, each module is an isolated or protected scope similar to a Class in OOP. This design principle is meant to prevent conflicts between different modules. Imagine if your module is being re-used by another application or even within the same application. The last thing you want is for someone else to over write a function or variable changing the behavior of your module, this could lead to unpredictable results. Using exports allows you to create an API (Application Programming Interface) that other developers can access. In other words you create a unique namespace for your module that other modules can reference.

What do you require?
The require function simply loads the file or module specified in the argument, as an example

require ('./app/src/modules/login');

This code will look in the app/src/modules folder and load the login.js file. NodeJS assumes the .js extension by default, if there is no file with that extension then it tries it looks in the login folder for an index.js file, if that file is not there then it looks for login.json and then login.node files in the modules folder.

If you have come this far then hopefully NodeJS is a little bit less intimidating than it was before. Feel free to explore the source code even further, it will only make you a better developer. In the next blog post we will look into Event, Event Emitter, streams and so forth. Stay tuned…