The following document outlines Mapbox's approach to writing native abstractions for Node.js (anecdotally referred to as addons).

Node is integral to the Mapbox APIs. Sometimes at scale, though, Node becomes a bottleneck for performance. Node is single-threaded, which blocks execution. C++ on the other hand allows you to execute operations without clogging up the event loop. Passing heavy operations into C++ and subsequently into C++ workers can greatly improve the overall runtime of the code.

Native Abstractions for Node.js (NAN)

To swing between Node and C++, the Node community maintains a project called NAN (Native Abstractions for Node.js) that simplifies running different versions of Node and subsequently, V8. NAN is a header-only C++ library that provides a set of Macros for developing Node.js addons. Check out the usage guidelines.

such as node-mapnik, node-osrm, and node-osmium. More examples of how to port C++ libraries to node can be found at nodejs.org/api/addons.html. See https://nodesource.com/blog/c-add-ons-for-nodejs-v4/ for a detailed summary of the origins of Nan.

Create an addon is a viable solution for the following reasons:

1. To port a C++ project to Node to expose a new interface for the tool (like Mapnik & Node Mapnik)
2. Improve performance at scale where Node becomes the bottleneck.

A Node.js addon is still a Node module. Users still interact with it as if they are writing Javascript (i.e. var awesome = require('awesome')), but the library will tend to pass much of the logic into C++ workers, which are highly performant, then return information back into a javascript interface. Bottom line, the user of your library never has to write or interact with C++.

All of the following libraries are installable in a Node.js environment, but execute much of their logic in C++:

• Node Mapnik - creating map tiles
• Node OSRM - directions & routing
• sqlite - asynchronous, non-blocking SQLite3 bindings
• vtinfo - reads and returns general information about a vector tile buffer
• Node GDAL - geographic operations

Makefile

package.json

Generating local binaries

node-pre-gyp

Writing & running tests

An addon can be published to NPM just like any other Node module. Unlike a standard Node module, an addon requires binaries to execute the code. If the binaries don't exist, they need to be built. User's may not have the tools necessary to compile C++ binaries on their system, like gcc or clang, so it's considered best practice to publish binaries to a public location. This greatly improves the speed at which they can install a module and use it.

Node-pre-gyp does a lot of the heavy lifting for publishing binaries, using the node-pre-gyp publish command.

Depending on the reason why you are creating an addon library, your naming scheme should follow these general guidelines:

• If your project is a Node.js port (originally a C++ project), name it node-{project name}
• If your project is originally written in pure Node.js and you are porting to start using addons, name it {project}-cpp.

Example

Mapnik is a C++ library, named mapnik. Its Node.js interface is named node-mapnik.

1. Why do we go into C++ from Node?
2. Performance & Scale
3. Library naming schemes for github repos
4. Examples
5. @ Mapbox 1. Node OSRM 1. Node Mapnik 1. VT Shaver 1. vtinfo
6. elsewhere?
7. Building & Testing (for me)
8. Build locally and run tests that execute against binaries
9. Publishing (for them & AWS)
10. NPM, but we need binaries for the C++
11. Why do we publish binaries?
12. How do we publish binaries?
13. How are binaries retrieved by others?
14. How to publish in other Node or OS versions
15. Do it yerself
16. Node C++ Skeleton
17. node-pre-gyp
18. C++ glossary
19. Including other C++ header libs into a project
20. Versioning a Node C++ library?
21. What happens when you hit different binaries? Snafu!
22. How do we prevent this @ Mapbox?