N|Solid Overview

NodeSource has developed N|Solid to meet the needs of enterprise production environments. Built upon the experience and insights of several core contributors to the Node.js ecosystem, N|Solid provides live instrumentation of your production system's health, and stability with no changes to your application code. In addition, N|Solid offers the ability to control access for your critical applications.

Overview of the N|Solid Components

N|Solid Runtime

At times, Node.js can feel like a black box. Shifting to an asynchronous programming model changes how developers are required to handle and interpret existing data. In order to help customers gain more visibility, we provide the N|Solid Runtime. The N|Solid Runtime is a build of Node.js bundled with an advanced native C++ component, the N|Solid Agent. The N|Solid Agent runs on its own thread inside your application, with direct access to the core elements of Node.js, libuv and the V8 JavaScript engine.

The N|Solid Runtime provides access to detailed metrics, and allows you to control application behavior in production. It also includes a powerful security policy model that allows restricting access to system resources by untrusted modules.

Read more about using the Runtime from the command line »

Node Compatibility

N|Solid 4.x is delivered bundled with either a Node.js v12.x Erbium LTS, Node.js v14.x Fermium LTS and, Node.js v16.x Gallium LTS (included in N|Solid v4.6.2) runtime. All N|Solid features are additive, meaning any application that runs on Node.js will also work with N|Solid. To verify the version of Node.js you have bundled in N|Solid, use nsolid -v.

For more information about the Node.js API, consult the Node.js API Documentation.

N|Solid Console

The N|Solid Console provides centralized access to all of your applications, and an aggregated view of each application's processes. This holistic view simplifies triage at runtime, makes it easier to find outliers, and takes advantage of advanced diagnostic tools directly from your browser.

Read more about the Console »

Summary

The combination of the Runtime and Console make N|Solid an invaluable tool for gaining insight into the behavior of your live production applications.

Backed by NodeSource's 24x7 support, N|Solid adds enterprise-grade diagnostic and security features, delivered on a release cycle that is aligned with Node.js Long Term Support (LTS) releases. This provides a stable production platform for all of your Node applications.

Console Overview

The N|Solid Console provides valuable insight into clusters of N|Solid processes running in a variety of configurations. Processes are grouped by their associated application, making it simple to find outliers in memory and CPU utilization.

Navigating to an individual process shows various metrics, and provides the ability to take heap snapshots and generate CPU profiles.

Applications Overview

Once a user launched the N|Solid Console and connected their processes to the console, users will now land on the console’s overview screen.

This screen delivers aggregated application metrics, including:

• Process Count
• Number of Vulnerabilities
• Application's status
• CPU utilization average
• Event loop utilization average
• Requests average
• Throughput
• Heap Used average
• Number of Worker Threads.

This list can be filtered and sorted by name, number of processes and number of vulnerabilities.

When selecting a specific application by clicking the application's name, users are redirected to the Application Summary.

When Number of Processes is clicked, users are redirected to the scatterplot filtered by the selected application name.

Application Summary

This screen delivers key information of the selected application including metrics, the number of vulnerabilities, number of processes, number of Worker Threads, number of events, and the application's status itself.

More insightful application information is presented in this view, with a fully dedicated metrics view, modules view, assets view, and events view; all this is shown application-wide.

Application metrics summary

In the application metrics summary view, users can a short graphical summary of how the application is behaving.

A key feature of the application's metrics view is the power of filtering the average of the metrics to processes with tags, users can select and unselect filters by their necessities and the graphs will be updated with processes with the desired tags.

Users are now able to inspect any metric considered insightful per application, those metrics could be numeric or a graph.

When a metric is clicked, it is zoomed in.

Application modules summary

In the application's modules view, the vulnerabilities are filtered application-wide, users can see the vulnerabilities not just for individual processes but complete applications, that's a better approach to take real advantage of NCM power.

Application assets summary

In the application's assets view, all the CPU profiles and Heap Snapshots of the application are listed, this makes it easier for users to diagnose per application not just per process.

Application events summary

In the applications events view, users can see all the runtime events, like security events, lifecycle events, system events, performance events, and assets events.

Learn more about event profiling here.

Processes View

The Processes view provides a visual overview of N|Solid applications monitored by the Console. By default, the N|Solid Console will display all processes. Using Filtering and Saved Views, you may filter processes by a wide variety of attributes, e.g. name, tags, performance metrics, etc. Saved views can also be created to group sets of processes together.

The Scatterplot

The Scatterplot is an animated graph that provides an overview of your applications' performance across all or a subset of connected processes, when an specific process has at least one active worker thread, the process will be highlighted.

Graph Axes Configuration

By default, the Y-axis plots the memory (Heap Used), and the X-axis plots the % CPU utilized for each process. You can configure these axes to measure different parameters.

Any numeric metric may be used to plot either axis. NodeSource recommends the following metrics as being the most useful:

Additionally, the Y-axis may also plot any textual field data that is provided by the N|Solid agents.

A full list of metrics available to N|Solid can be found in the Metrics in Detail section (some of the listed metrics may not be available in the Scatterplot).

Axis Scaling

There are three options for axis scaling available:

• Linear: The default scaling option. Plotted values are evenly distributed along the graph.
• Logarithmic: Decompresses lower values along the graph.
• Exponential: Decompresses higher values along the graph.

Processes List

The Processes List on the right side of the page provides a textual representation of the processes on the scatterplot.

Click and drag inside the scatterplot to select a subset of processes. The Processes List will update to show only these processes. Click Apply as Filter to view the selected processes as a filter, which you can then save as a Saved View. Click Clear or click on the graph to clear the selection and show all processes within the view.

Sort selected processes in the Processes List using the Sort dropdown. Sorting by App Name or Hostname will group like processes together. Sorting by Vulnerabilities will group vulnerable and non-vulnerable processes together respectively. Sorting by CPU, Memory, or Uptime will sort the processes numerically by these metrics. For sorting methods that group processes (App Name, Hostname, and Vulnerabilities), clicking on the label for a group of processes will select, or narrow selection, to that group of processes.

Clicking the process ID will take you to the Process Detail view.

Hovering your mouse over a process in the Processes List will highlight that process in the graph for easy identification among other processes. Hovering on a process when processes are selected also reveals an X icon that may be clicked to remove a single process from the selection.

Hovering over a process in the Processes List also shows a target icon that enables process targeting. Process targeting causes one minute of the process's recent historic metrics to be visualized as a trail following the process around the graph. One process can be targeted at a time. Clicking the target icon a second time will disable process targeting.

Process Detail View

The Process Detail view shows detailed information relating to a specific process. At the top of the view it displays data that N|Solid receives from the process, including the pid, host, platform, and CPU configuration.

The left side of this view contains the threads (worker threads) of the process, the thread 0 is the main thread and the rest of them are worker threads, and the right side of this page also contains controls for manually taking CPU profiles and heap snapshots of this process.

The available threads can also have a name to make them easier to identify, learn how to set thread name

Comparison of Threads

You can compare threads by selecting thread IDs on the Thread List. Please note that maximum of 2 threads can be selected for comparison.

The legend of the metrics chart will be automatically updated including each thread ID based on the threads you selected for comparision.

Note: There are some metrics marked as process-wide, which are not thread-specific.

Metrics Tab

The Metrics tab contains several charts which visualize important performance metrics which N|Solid gets from your processes.

Show or Hide Metrics

The Metrics Search Bar allows you to show or hide metrics, which supports auto-completion for all of the metrics defined in the Metrics in Detail.

To show more metrics which are not on the metrics tab by default, search metrics using the auto-completion and select the metrics from the search results. You can also set multiple filters on the search bar.

Note: Please refer to Metrics in Detail to see all available metrics.

The chart of the metrics that you selected will be shown below the ELU metrics chart. If you want to hide metrics, click x icon on the metrics label on the search bar.

Modules Tab

The Modules tab lists all modules this process is loading and using at runtime. More importantly, this tab prominently features modules that include known security vulnerabilities.

Assets Tab

The Assets tab lists CPU profile and heap snapshots related to this process. Click on the asset to view it.

Quick Start Guide

This guide will help you try N|Solid on Linux, macOS or Windows 10 locally. It provides a quick, zero-config way to try NSolid or install all of the components onto your development machine on macOS.

PLEASE NOTE: THIS WAY OF RUNNING NSOLID IS MEANT FOR LOCAL TESTING ONLY! For production deployments please refer to the documentation below.

NSolid in Production

For instructions on installing the individual components for a production cluster on Linux or Windows 10, please refer to the the installation guide.

Try N|Solid

Using one simple npx command, you can try NSolid by using a self-guided demo, run and diagnose your own application or execute an NPM task you can observe locally.

Access

To try N|Solid visit accounts.nodesource.com and sign up for a free account. Setting up an account is free and requires no credit card information. Select the ‘Try NSolid’ option, and follow the prompts.

Try N|Solid - Demo

Using the 'npx nsolid-quickstart' command loads N|Solid into your local cache where it will be executed without actually having been installed. This setup method was designed for frictionless ease of use. npx is a npm package runner (x probably stands for eXecute). The typical use is to download and run a package temporarily or for trials.

It is not recommended to use this method in production. We provided separate setup instructions for on-prem here and cloud deployments here.

To run the demo simply execute:

$ npx nsolid-quickstart --demo

The command will launch a self-guided demo that runs a ready-made simulation to introduce you to NSolid’s performance monitoring and diagnostic features.

It will open a a new browser window or tab pointing to the default N|Solid Console URL: http://localhost:6753/. There you will be asked to authenticate with your accounts.nodesource.com credentials before the console drops you into the self-guided demo experience.

Because this runs a simulated application, the demo uses zero infrastructure resources and does not interfere with any of the applications you may connect to the console.

As such, the demo can also be accessed in on-prem and cloud deployments via the ‘Start Demo’ and ‘Guided Demo’ buttons.

Try N|Solid - Bring your own application

It is possible to use N|Solid Quickstart to launch your own applications. There are two options: execute a JavaScript file or execute an NPM task.

To lauch your application from a JavaScript file just execute:

  $ npx nsolid-quickstart --exec index.js

The above command is the equivalent of running node index.js. Just replace index.js with the right name for your script.

To execute your application from an NPM task execute the following command:

  $ npx nsolid-quickstart --npm dev

This is the equivalent of running npm run dev, just replace dev with the right name for your NPM task.

Quickstart Help

There are many options included, like --lts, which allow to switch to a different Node.js LTS version (by default gallium is used), to see all available options execute:

  $ npx nsolid-quickstart

Installing via the Microsoft Installer for Windows 10

To install the N|Solid Runtime and Console via the MSI bundle for Windows 10 complete the following steps:

1. Go to accounts.nodesource.com
2. Click on the Windows tab and download the package
3. Double click on the downloaded MSI file and follow the instructions to install the necessary N|Solid and Node.js components.

4. Some npm modules need to be compiled from C/C++ when installing. To ensure you can access a fully working node environment select tools such as Python and Visual Studio Build Tools need to be installed. PLEASE NOTE: N|Solid may not be working as expected if you skip this step. This will also install Chocolatey. This script will pop up in a new PowerShell after the instalation is complete.

5. Once N|Solid's major components are intalled a PowerShell will pop up automatically to install Chocolatey, and other Node.js dependencies.

6. You may be asked to restart your operating system. Please preceed to do so.

7. Once your system has rebooted the N|Solid Console should start automatically if you installed it using the MSI package.

8. Alternatively you can always search for the NSolid app in the Windows search bar and start the service by clicking the app icon:

PLEASE NOTE: You can optionally 'Remove' or 'Install' the N|Solid Console Service via the corresponding shortcuts in the Windows Start menu:

Installing via the macOS Developer Bundle

To install N|Solid via the Developer Bundle for macOS:

1. Go to accounts.nodesource.com
2. Click on the macOS button to get an installable package
3. Double click on the downloaded pkg file and follow the instructions to install the N|Solid compoments

Installing via Homebrew

To install N|Solid via Homebrew, first make sure you have Homebrew installed and that it is up to date.

$ brew update

First, add the nsolid tap:

$ brew tap nodesource/nsolid

Now you can install N|Solid:

$ brew install nsolid-console

This will install both the N|Solid Runtime and N|Solid Console.

For more information on our Homebrew packages, see our repository.

Setting Up the N|Solid Console

Once installed, using N|Solid is fairly straightforward. To launch the Console on macOS, open a terminal window and enter the command:

$ nsolid-console

Note: You do not have to be the root user to run the console, only to install it. Logs for each process can be found under /tmp/nsolid-*.

Linux users: The N|Solid Console will start automatically using your system init scripts if you installed using the Linux distribution packages

Once the Console is running, you can see the user interface by opening a new web browser window to http://localhost:6753.

You should be redirected to http://localhost:6753/welcome, where you can register your Console using your NodeSource Accounts login. Be sure to pick the correct organization as it controls who may access your Console.

Once registered, you should be immediately redirected to the N|Solid console.

Running Node Applications in N|Solid

The next step is to configure your shell to use the N|Solid Runtime.

Open a second Terminal window and run:

$ source /usr/local/nsolid/nsolid-env

This will activate the N|Solid shell.

This is only included in the macOS Developer Bundle. If you installed via Homebrew, MSI Installer for Windows 10 or Linux packages, this is not needed.

Any Node application that you start from within the N|Solid Shell will then be shown in the N|Solid Console.

For example, monitor the Node.js REPL using N|Solid by running:

$ NSOLID_APPNAME="REPL" NSOLID_COMMAND="localhost:9001" nsolid

If you have an existing Nodejs application, e.g. "index.js", you can monitor it with N|Solid by running:

$ NSOLID_APPNAME="My_Application" NSOLID_COMMAND="localhost:9001" nsolid index.js

You can customize the name of your application in the N|Solid Console by changing the NSOLID_APPNAME environmental variable. If you do not specify an app name, N|Solid will look for a 'name' property in your package.json, and if that is not found your application will be named untitled application.

You are now ready to use N|Solid for your own applications!

Migration Guide

For N|Solid 3.x users, you will notice that begining in N|Solid 3.4.0, users of the Console are asked to authenticate before they can use the Console.

• Requiring authentication is default behavior and can be disabled using the NSOLID_CONSOLE_AUTH_REQUIRE_LOGIN=false environment variable, or the corresponding configuration file setting.

• To make use of the authentication to protect access to your Console, you should add the users that require access to the NodeSource Accounts organization which was used to activate the Console.

• In order to use nsolid-cli with authentication enabled, you should set the NSOLID_CONSOLE_AUTH_ADMIN_TOKEN environment variable to a secure value and utilize this with the corresponding --auth parameter to nsolid-cli. See user authentication for more information.

If you are upgrading from N|Solid 2.x, here are the need-to-know steps for migrating to N|Solid 3.x.

• N|Solid Storage has been eliminated, and its functionality is provided by the N|Solid 3.x Console server. Attempting to connect an existing N|Solid Storage instance to a N|Solid 3.x Console will not work unless the storage instance is running in relay mode.

• Many of the settings and environment variables from N|Solid Storage are available in the 3.x N|Solid Console; substitute CONSOLE for STORAGE in the variable name. View the Networking documentation for specific settings.

• Loading a N|Solid 3.x Console server with the same configuration as a N|Solid Storage instance will load some of your existing assets and metrics history, but Heap Snapshots will need to be accessed directly from your storage device. The default location has changed from ~/.nsolid-storage to ~/.nsolid-console.

• The N|Solid 2.x Runtime is compatible with the N|Solid 3.x Console, but not all functionality will be available.

• The environment variables used to configure N|Solid Runtime are the same as with N|Solid 2.x, however you must provide a valid license during any operation, either via a connection to an N|Solid Console, or by providing a License Token directly. Read the Configuration guide for more information.

• The nsolid-cli syntax and behavior have changed from 2.x, be sure to read the Command Line Interface documentation if you use this functionality.

Frequently Asked Questions - NSolid

The following section covers a series of questions users have asked NodeSource in the past:

1. I can't find one or more features in NSolid

NSolid supports Role Based Access Control. The role assigned to your user-profile may be configured to limit your access to a sub-set of NSolid features. Contact your organization's account administrator to see which permissions you may be missing. To learn how NodeSource's RBAC works and which permission are available, visit the RBAC section in the accounts docs.

2. Access not allowed screen.

To secure the NSolid Console against unauthorized access, every console is linked to an account. The organization’s account administrator can invite team-members to the organization’s account, thus authorizing team-members to log into their organization’s NSolid Console.

You will encounter the below screen if you tried to access a console whose organization you are not part of (anymore). To regain access, we recommend you contact your org’s administrator directly.

If you deployed the NSolid Console yourself, and can’t access the console, we recommend you observe your console’s license key. If the license key belongs to an organization that does not consider your user credentials (email and password) part of its org you will not be permitted to use the NSolid Console. In that case, please contact the Console’s administrator via the contact details provided on the ‘Access Not Allowed’ Screen or contact NodeSource support for assistance.

3. Why do I get an 'Organization Inactive' screen?

Whether you are an Org Admin or have been invited to an existing organization, you may encounter the below screen when attempting to log into the NSolid Console:

You may be seeing this screen for one of two reasons:

1. The NSolid Console was assigned to an organization whose free 30-day trial has expired.
2. The NSolid Console was assigned to an organization whose license has since expired. Please note that the console will continue to collect data until the license has been renewed.

To renew/ upgrade an enterprise license, please contact sales@nodesource.com. To renew an Advanced-tier license visit accounts.nodesource.com and update your payment details.

4. I am running an application but the NSolid Console says there are no processes connected?

This is expected behavior. To monitor Node applications using N|Solid, you must set the following environment variable when running your Node.js application. This tells the NSolid runtime where to report your metrics to.

For a step-by-step guide on setting your environmental variable please refer to the following links:

• Run NSolid Locally
• Run NSolid in AWS
• Run NSolid in Azure Quick Start Guide
• Run NSolid in GCP Quick Start Guide
• Run NSolid for Cloud Run in GKE

5. Does NSolid’s CPU provide sourcemap support?

Yes. Utilizing typescript and/or transpilers with N|Solid makes interpreting CPU profiles difficult unless the user is deeply familiar with the code. The integration of Source Maps provides a translation layer that provides a reference between the compiled source code and source code.

When compiling code, a Source Map is currently being generated. The integration of Source Maps would provide a translation layer that provides a reference between the compiled source code and source code.

In an effort to address concrete customer pain-points NodeSource has introduced SourceMap Support to the N|Solid CPU profiler. This feature continues to evolve with the specific requirements of our customers.

The feature’s UX can be viewed in the docs here.

6. NSolid Console says is Offline?

This could happen if your firewall is blocking our services and API. Please make sure that services.nodesource.io and api.nodesource.com are listed in your firewall's whitelist.

Installation

This guide will help you quickly install all of the N|Solid components for a production cluster. We provide production rpm and deb packages for use in Linux distributions and a windows MSI installer for use on Microsoft Windows Server 2016 & 2019.

Installation on Linux

Our rpm and deb packages for use in Linux distributions makes it significantly easier to set up the N|Solid components by using our repositories.

We support the following rpm based 64-bit Linux distributions:

• RHEL7 / CentOS7
• RHEL8 / CentOS8
• Fedora 30
• Fedora 31

We also support the following deb based 64-bit Linux distributions:

• Debian Jessie
• Debian Stretch
• Debian Buster
• Ubuntu Trusty
• Ubuntu Xenial
• Ubuntu Bionic

If you do not wish to use these packages, manual installation instructions are provided below.

Note: The default Node.js runtime used in the instructions below is Node.js v16 LTS (Gallium). If you wish to use a version of N|Solid based on Node.js v14 LTS (Fermium) or Node.js v12 LTS (Erbium), replace any instances of gallium with fermium or erbium.

Installing N|Solid Packages

Step 1: Set Up the NodeSource N|Solid Repository

For rpm based distributions, use the following command as root:

$ curl -sL https://nsolid-rpm.nodesource.com/nsolid_setup_4.x | sudo bash -

For deb based distributions, there is a similar command run as root:

$ curl -sL https://nsolid-deb.nodesource.com/nsolid_setup_4.x | sudo bash -

In both cases, these scripts will attempt to determine the distribution you are using, set up the appropriate repository on your system, and import the NodeSource N|Solid GPG key used to sign the packages.

Step 2: Install the Packages

For rpm based systems, install all the required components with the command:

$ sudo yum -y install nsolid-gallium nsolid-console

For deb based systems, the process is similar. Install the required components:

$ sudo apt-get -y install nsolid-gallium nsolid-console

On deb based systems, the relevant daemons will start automatically on installation, but you will still turn the services on and off using the standard tools.

For both rpm and deb based systems, you can then turn on the installed components using the standard system tools. Below are examples of how to turn on the N|Solid Console component using both init and systemd style systems.

Start the component in an init style system with:

$ /etc/init.d/nsolid-console start

Newer systemd style systems would use the command:

$ systemctl start nsolid-console

Once started, navigate to the N|Solid Console at localhost:6753. Consult the Console Overview for more information.

These N|Solid components can be installed separately, and possibly on different machines if you wish. However, if you plan on installing all of the components onto a single machine, the process can be further simplified. Instead of installing the components individually by name as we've done here, you can simply install the nsolid-console-all meta package for your system.

For rpm based distributions:

$ sudo yum -y install nsolid-gallium-console-all

And for deb based distributions:

$ sudo apt-get -y install nsolid-gallium-console-all

This meta package will install all of the components and turn them on if needed.

Upgrading N|Solid Packages

Once you have set up the NodeSource N|Solid package repository appropriate to your system, you will automatically get point releases via the standard system update tools. For example, fetch and install any new versions of the packages with:

$ sudo yum -y update

Manual Installation

These instructions use the Linux platform builds, but the same steps apply for macOS (darwin) platforms.

Step 1: Get N|Solid

1. Go to downloads.nodesource.com.
2. Scroll down to the Downloads section and click the Linux Bundle for the LTS version you wish to run.
3. Extract the archive.

The extracted archive contains all N|Solid components for your architecture:

• The N|Solid Runtime (e.g. nsolid-v4.6.2-gallium-linux-x64.tar.gz)
• The N|Solid Console (e.g. nsolid-console-v4.6.2-linux-x64.tar.gz)

Step 2: Install the N|Solid Runtime

The N|Solid Runtime is a build of Node.js that is compiled to include the N|Solid Agent. The runtime should be installed on any application servers where you intend to run your Node.js application.

$ tar xf nsolid-v4.6.2-gallium-linux-x64.tar.gz
$ cd nsolid-v4.6.2-gallium-linux-x64
$ ls
bin  CHANGELOG.md  include  lib  LICENSE  LICENSE.NSOLID  README.md  share

We suggest installing into /usr/local for greatest compatibility, though most preferred installation methods used for Node.js will also work for N|Solid.

To install to /usr/local, first remove existing npm and nsolid-cli installations:

$ rm -rf /usr/local/lib/node_modules/npm /usr/local/lib/node_modules/nsolid-cli

Then, put the N|Solid components in place:

$ cp -r lib bin share /usr/local/

To verify you are running Node compiled with the N|Solid Agent and see your version of N|Solid, use:

$ node -vv
v4.2.0

This package also contains the nsolid-cli application, which can act as a command-line interface to the N|Solid Agent via the N|Solid Console, allowing scripted or manual interaction.

Step 3: Install the N|Solid Console

The N|Solid Console is a graphical interface for your entire cluster of applications running on N|Solid. It communicates with each N|Solid Agent to track metrics, collect information, or trigger events on any of your N|Solid processes.

The N|Solid Console is best experienced using one of the following browsers on a desktop/laptop system.

• Google Chrome (21 and higher)
• Mozilla Firefox (22 and higher)
• Apple Safari (9 and higher)

The download bundle includes the nsolid-console built for your platform.

$ tar xf nsolid-console-v4.6.2-linux-x64.tar.gz
$ ls nsolid-console-v4.6.2-linux-x64

Place this folder wherever you typically install your Node.js applications. For this example, we'll put it in /var/opt.

$ cp -r nsolid-console-v4.6.2 /var/opt
$ cd /var/opt/nsolid-console-v4.6.2

Running npm start from within this directory will start the N|Solid Console with the default configuration.

For more information about configuration options for the N|Solid Console, see: configuration.

You will want to run nsolid-console as any other Node.js service in your infrastructure, using your process manager of choice.

Upgrading From Previous Versions of N|Solid

To upgrade from previous versions of N|Solid, use the instructions above to install over your previous installation. At each step, verify that the latest version of the component is installed and working.

If you have the N|Solid Hub, Proxy, or Storage components installed from an earlier version of N|Solid, these components can be removed.

For rpm based systems, this can be done with:

$ sudo yum remove nsolid-proxy nsolid-storage

For deb based systems, this would be:

$ sudo apt-get remove nsolid-proxy nsolid-storage

The etcd package may be in use by your system for other purposes, but if you no longer require it, it too can be removed:

$ sudo yum remove etcd

or

$ sudo apt-get remove etcd

Older N|Solid Artifacts

N|Solid 2.x

The storage directory structure did not substantially change from N|Solid 2.x to N|Solid 3.x, and if you start your Console with the same directory, you may see some of your data and artifacts. However, compatibility between these versions is not guaranteed. It is recommended that you start with an empty storage directory and access your old files directly on your storage device.

Installation on Windows

NodeSource provides a Windows MSI installer to deploy NSolid in production enviornments.

We support the following Windows based distributions:

• Windows 10
• Microsoft Windows Server 1909 Core
• Microsoft Windows Server 2012
• Microsoft Windows Server 2008

Note: The default Node.js runtime used in the instructions below is Node.js v16 LTS (Gallium). If you wish to use a version of N|Solid based on Node.js v14 LTS (Fermium) or Node.js v12 LTS (Erbium), replace any instances of gallium with fermium or erbium.

Installing N|Solid on Windows Servers 2008 & 2016

1. Go to accounts.nodesource.com
2. Click on the Windows tab and download the package
3. Double click on the downloaded MSI file and follow the instructions to install the necessary N|Solid and Node.js components.

Foor production enviornments, you may wish to deploy the NSolid Runime and Console separately. If that is the case ensure you select/ deselect the relevant application components to be installed via in the Custom Setup dialogue.

4. Once N|Solid's major components are intalled a PowerShell will pop up automatically to install Chocolatey, and other Node.js dependencies.

5. To stop and start the N|Solid Console via the Windows Start Menu.

6. You can optionally 'Remove' or 'Install' the N|Solid Console Service via the corresponding shortcuts in the Windows Start menu:

5.2 Installing N|Solid on Windows Core 1909:

To install and the N|Solid Runtime and N|Solid Console:

1. Open PowerShell powershell

2. Install Chocolatey

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

3. Install Chrome

Invoque-Request download_URL -OutFile nsolid-v4.6.2-gallium-x64.msi

4. Download the N|Solid package

powershell

5. Start N|Solid package installation and follow the installation steps

Start-Process .\nsolid-v4.6.2-gallium-x64.msi

6. Update current $PATH

$env:PATH = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")

7. Verify that everything was installed correctly

nsolid -v
nsolid -vv

8. Open browser with requested URL, same as running nsolid-console

Start-Process "chrome.exe" "http://localhost:6753"

9. Run console from the installed command

nsolid-console

Configuration

N|Solid Runtime

Environment Variables

The N|Solid runtime can be configured via the environment variables listed below. Although each of these environment variables are optional, certain features will be missing or limited in each case. It is recommended that you connect to the console using NSOLID_COMMAND or a StatsD collector using NSOLID_STATSD, and customize your process using any of the remaining environment variables.

package.json

The N|Solid process can also be configured via an nsolid object in the package.json file for the current application. Environment variable values override the properties in the package.json file. The mapping of nsolid object properties in the package.json file to environment variables is as follows:

Example package.json:

# package.json

{
  "name": "message-service",
  "version": "1.0.0",
  "nsolid": {
    "env": "production",
    "command": "nsolid-command-host.local:9001",
    "app": "messaging",
    "tags": "aws, mq"
  }
}

N|Solid CLI

Configuration File

The N|Solid Command Line Interface (CLI) can be configured via a configuration file, .nsolid-clirc.

# .nsolid-clirc

{
  "remote": "http://localhost:6753",
  "app": "my-node-application"
}

N|Solid Console

Command Line Arguments

These are the command line arguments if you're starting the console explicitly. For most usages, you would utilize npm start.

Environment Variables

If you are using the developer bundle, you can use the following to configure the console:

Stats Retention

Historical metrics, displayed in the line-chart UI elements, are trimmed to the last 15 minutes by default.

For more information about configuration options for N|Solid Console's data processing functions, see networking.

License Keys

Registering Your Console

In order to access N|Solid you must have a valid NodeSource account. If you have not connected your Console to your account, when you try and access the console you will be redirected to the welcome screen. Using this form to log into your NodeSource account will automatically configure your Console server and allow you to associate it with your account or organization.

Once connected, you should be immediately redirected to the N|Solid Console.

Updating Your License

If you want your Console to be associated with a different account or organization, click Settings on the right side of the menu at the top of the page.

If you are an administrator for the linked organization, you will see a "Reset License" button which can be used to restore the Console to its initial login state.

If you do not have administrative access to the registered license and need to reset the Console's registration state, you can start the Console with the NSOLID_CONSOLE_AUTH_REQUIRE_LOGIN=false environment variable to disable the authentication requirement and allow access to the Reset Console button. Remember to restart without this setting if authentication is still desired.

User Authentication

Beginning with N|Solid 3.4.0, users will be asked to authenticate before they can access the console. If the user is not the owner of the account or a member of the associated organization, they will not be permitted to access the console.

To allow additional users to access your console, add them to the NodeSource Accounts organization which is associated with this Console.

If you do not wish to enforce user authentication, it can be disabled by setting the NSOLID_CONSOLE_AUTH_REQUIRE_LOGIN=false environment variable or its associated config file setting.

This authentication also impacts nsolid-cli. If you require use of the CLI, you should set NSOLID_CONSOLE_AUTH_ADMIN_TOKEN or the corresponding config file setting to a secure value and use it with the --auth argument. This grants administrative access to the console and should be used with discretion. See Command Line Interface for instructions.

Using StatsD without connecting to Console

You must have a valid license to use the N|Solid Runtime either with the Console or communicating directly to your StatsD endpoint. If you do connect to your N|Solid Console, the license will be configured automatically for you. If you prefer to use your runtime without connecting to console, you must provide your license in the form of a signed token. This can be obtained from a running licensed N|Solid Console by running nsolid-cli license-token or by querying http://console:6753/api/v3/license-token directly.

If you do not have a N|Solid Console available, you can contact NodeSource Support and they can provide you with one. The token will be valid for the duration of your license period and must be refreshed when your expiration date is extended.

Your valid license token can be specified as the NSOLID_LICENSE_TOKEN environment variable or included in the nsolid section of package.json as "licenseToken".

Offline or Air-Gapped operation

Similar to the StatsD situation, your N|Solid installation must have a valid license to operate. If your system is not able to contact the NodeSource web services, you may use the license token as described previously by pasting it into the License Key entry field in your N|Solid Console.

Make sure you are not passing your xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx license key as the token value. This is your license key and cannot be used in offline licensing.

Troubleshooting

Debug Messages

The N|Solid Runtime and CLI all can have debug messaging enabled by using the NODE_DEBUG=nsolid environment variable.

$ NODE_DEBUG=nsolid NSOLID_APP=testapp NSOLID_COMMAND=localhost ~/ns/nsolid-node/nsolid
> NSOLID 16597: starting agent (js init) name: testapp id: e24a79b2aae58b63e385a4c0ce9ed91f3d0202ee tags: undefined
NSOLID 16597: nsolid agent e24a79b2aae58b63e385a4c0ce9ed91f3d0202ee starting with config
NSOLID 16597: { command: '9001',
  data: undefined,
  bulk: undefined,
  pubkey: undefined,
  statsd: undefined,
  statsdBucket: 'nsolid.${env}.${app}.${hostname}.${shortId}',
  statsdTags: undefined,
  hostname: 'rpi3-1',
  env: 'prod',
  interval: null,
  tags: undefined,
  app: 'testapp',
  appVersion: undefined }
NSOLID 16597: { zmq_command_remote: 'tcp://localhost:9001',
  zmq_data_remote: null,
  zmq_bulk_remote: null,
  statsd_addr: null,
  storage_pubkey: '^kvy<i^qI<r{=ZDrfK4K<#NtqY+zaH:ksm/YGE6I' }
NSOLID 16597: registering default commands
NSOLID 16597: nsolid initializing
NSOLID 16597: agent spawn()
NSOLID 16597: agent spawned
NSOLID 16597: e24a79b2aae58b63e385a4c0ce9ed91f3d0202ee nsolid initialized
NSOLID 16597: { command_remote: 'tcp://localhost:9001',
  data_remote: 'tcp://localhost:9002',
  bulk_remote: 'tcp://localhost:9003',
  storage_pubkey: '^kvy<i^qI<r{=ZDrfK4K<#NtqY+zaH:ksm/YGE6I' }

Loggin in with Okta, oneLogin or PingID

To setup SAML in NSolid see here.

If you are using NSolid for the first time it is important that you observe step 1 - 8. If you are a repeat user you can jump to step 9.

Logging in with Okta

To use the NSolid Console for the first time you must sign into accounts.nodesource.com first to accept the NodeSource Terms and Conditions. As such this user flow suggests starting with accounts.nodesource.com. Once you accepted the terms you can log into the NSolid Console directly using your Okta SAML credentials directly.

1. Visit ​**https://accounts.nodesource.com/sign-in**
2. Enter your corporate email address that is associated with your SAML credentials in the ‘EMAIL and SAML Accounts’ field:

3. The system will automatically detect your email address and offer a SAML sign-in if your employer set up Federated Authentication for your NodeSource organization.
4. If you used your email address to set up a standard account and your employer set up Federated Authentication for their NodeSource organization, you will be given the option to select which account you would like to login to. Select the the SAML Account button if you wish to log into your employer’s NodeSource account:

5. Once you selected ‘SAML Account’ you will be redirected to the Okta authentication page:

6. Use your Okta credentials (provided by your employer) to sign into Okta.
7. If this is your first time signing in you will be asked to accept NodeSource’s Terms and Conditions:

8. Once you signed in and if you accepted NodeSource Terms previously you will land on the accounts.nodesource.com download page:

You can now install NSolid (see here), or open the NSolid Console (next step)

If you happen to be a member of multiple NodeSource organizations, please note that you can still select your personal and/or other org accounts from the org-selector in the top left corner, but access to said accounts will be restricted. To regain full access to said accounts, simply log out and access said organizations via their corresponding email address, SAML or SSO login.

9. If you accepted the NodeSource Terms previously you can directly navigate to the NSolid Console. The URL to do so depends on your organization’s chosen method of deployment. Please see the Getting Started Guide for details (here).

10. When logging into the Console for the first time, you must register your console with your SAML details.

11. Enter your corporate email address that is associated with your SAML credentials in the ‘EMAIL and SAML Accounts’ field:

12. If you used your email address to set up a standard account and your employer set up Federated Authentication for their NodeSource organization, you will be given the option to select which account you would like to login to. Select the the SAML Account button if you wish to log into your employer’s NodeSource account:

13. Once you selected ‘SAML Account’ you will be redirected to the Okta authentication page:

14. Use your Okta credentials (provided by your employer) to sign into Okta.
15. Once you successfully authenticated, the console is linked to your organization and your team members will be able to use it.

16. To connect a process please consult the Quick Start guide here.

17. To invite team-members (see here).

Logging in with PingID via PingOne Desktop:

There are two ways to use PingID to authenticate into NodeSource’s Accounts and NSolid Console. This covers the use of the PingOne Desktop Application view. You can also use accounts.nodesource.com to sign in directly with your PingID credentials (see here).

If you are using NSolid for the first time it is important that you observe step 1 - 9. If you are a repeat user you can jump to step 10.

1. Visit your PingID portal via the PingID URL you were provided with by your employer. It will look something like this: https://desktop.pingone.com/cd-xxxxxxx.[yourPingIDorgname
2. Sign into the PingID platform with the username and password you were provided with by your organization

3. Upon Sign In you will be redirected to the applications page.

4. There you should see an icon for:
   1. NodeSource Accounts: Our authentication and user management platform and
   2. NSolid Console: The runtime-powered observability tool to monitor your application’s performance and security posture.
5. Please select NodeSource Accounts first
6. You will be redirected to ​**https://accounts.nodesource.com/sign-in**
7. If this is your first time signing in you will be asked to accept NodeSource’s Terms and Conditions:

8. Once you signed in and if you accepted NodeSource Terms previously you will land on the accounts.nodesource.com download page:

9. You can now install NSolid (see here), or open the NSolid Console (next step)
10. Once you installed NSolid you can open the NSolid Console via the PingOne Desktop Application view:

11. Once you successfully authenticated, the console is linked to your organization and your team members will be able to use it.

12. To connect a process please consult the Quick Start guide here.

13. To invite team-members (see here).

Logging in with PingID directly:

If you are using NSolid for the first time it is important that you observe step 1 - 8. If you are a repeat user you can jump to step 9.

To use the NSolid Console for the first time you must sign into accounts.nodesource.com first to accept the NodeSource Terms and Conditions. As such this user flow suggests starting with accounts.nodesource.com. Once you accepted the terms you can log into the NSolid Console directly using your PingID SAML credentials directly.

1. Visit ​**https://accounts.nodesource.com/sign-in**
2. Enter your corporate email address that is associated with your SAML credentials in the ‘EMAIL and SAML Accounts’ field:

3. The system will automatically detect your email address and offer a SAML sign-in if your employer set up Federated Authentication for your NodeSource organization.
4. If you used your email address to set up a standard account and your employer set up Federated Authentication for their NodeSource organization, you will be given the option to select which account you would like to login to. Select the the SAML Account button if you wish to log into your employer’s NodeSource account:

5. Once you selected ‘SAML Account’ you will be redirected to the PingID authentication page:

6. Use your PingID credentials (provided by your employer) to sign into PingID.
7. If this is your first time signing in you will be asked to accept NodeSource’s Terms and Conditions:

8. Once you signed in and if you accepted NodeSource Terms previously you will land on the accounts.nodesource.com download page:

You can now install NSolid (see here), or open the NSolid Console (next step)

If you happen to be a member of multiple NodeSource organizations, please note that you can still select your personal and/or other org accounts from the org-selector in the top left corner, but access to said accounts will be restricted. To regain full access to said accounts, simply log out and access said organizations via their corresponding email address, SAML or SSO login.

9. If you accepted the NodeSource Terms previously you can directly navigate to the NSolid Console. The URL to do so depends on your organization’s chosen method of deployment. Please see the Getting Started Guide for details (here).
10. When logging into the Console for the first time, you must register your console with your SAML details.
11. Enter your corporate email address that is associated with your SAML credentials in the ‘EMAIL and SAML Accounts’ field:

12. If you used your email address to set up a standard account and your employer set up Federated Authentication for their NodeSource organization, you will be given the option to select which account you would like to login to. Select the the SAML Account button if you wish to log into your employer’s NodeSource account:

13. Once you selected ‘SAML Account’ you will be redirected to the PingID authentication page:

14. Use your PingID credentials (provided by your employer) to sign into PingID.
15. Once you successfully authenticated, the console is linked to your organization and your team members will be able to use it.

16. To connect a process please consult the Quick Start guide here.

17. To invite team-members (see here).

Logging in with oneLogin:

If you are using NSolid for the first time it is important that you observe step 1 - 8. If you are a repeat user you can jump to step 9.

To use the NSolid Console for the first time you must sign into accounts.nodesource.com first to accept the NodeSource Terms and Conditions. As such this user flow suggests starting with accounts.nodesource.com. Once you accepted the terms you can log into the NSolid Console directly using your oneLogin SAML credentials directly.

1. Visit ​**https://accounts.nodesource.com/sign-in**

2. Enter your corporate email address that is associated with your SAML credentials in the ‘EMAIL and SAML Accounts’ field:

3. The system will automatically detect your email address and offer a SAML sign-in if your employer set up Federated Authentication for your NodeSource organization.
4. If you used your email address to set up a standard account and your employer set up Federated Authentication for their NodeSource organization, you will be given the option to select which account you would like to login to. Select the the SAML Account button if you wish to log into your employer’s NodeSource account:

5. Once you selected ‘SAML Account’ you will be redirected to the oneLogin authentication page:

4. Use your oneLogin credentials (provided by your employer) to sign into oneLogin.
5. If this is our first time signing in you will be asked to accept NodeSource’s Terms and Conditions:

6. Once you signed in and if you accepted NodeSource Terms previously you will land on the accounts.nodesource.com download page:

You can now install NSolid (see here), or open the NSolid Console.

If you happen to be a member of multiple NodeSource organizations, please note that you can still select your personal and/or other org accounts from the org-selector in the top left corner, but access to said accounts will be restricted. To regain full access to said accounts, simply log out and access said organizations via their corresponding email address, SAML or SSO login.

7. If you accepted the NodeSource Terms previously you can directly navigate to the NSolid Console. The URL to do so depends on your organization’s chosen method of deployment. Please see the Getting Started Guide for details (here).

8. When logging into the Console for the first time, you must register your console with your SAML details.

9. Enter your corporate email address that is associated with your SAML credentials in the ‘EMAIL and SAML Accounts’ field:

8. If you used your email address to set up a standard account and your employer set up Federated Authentication for their NodeSource organization, you will be given the option to select which account you would like to login to. Select the the SAML Account button if you wish to log into your employer’s NodeSource account:

9. Once you selected ‘SAML Account’ you will be redirected to the oneLogin authentication page:

6. Use your oneLogin credentials (provided by your employer) to sign into oneLogin.

7. Once you authenticated successfully a browser window containing the NSolid Console will open. To connect a process please consult the Quick Start guide here.

8. Once you successfully authenticated, the console is linked to your organization and your team members will be able to use it.

9. To connect a process please consult the Quick Start guide here.

10. To invite team-members (see here).

StatsD

The N|Solid Agent is able to periodically send a suite of metrics directly to a StatsD-compatible data collector endpoint. As this functionality is built directly into the nsolid executable, you do not need to be connected to the N|Solid Console server in order to use it. Independent nsolid processes can be configured to talk to StatsD collectors using environment variables when starting your application.

Consult the Metrics in Detail section for a complete list of metrics that are sent to the StatsD collector for your N|Solid processes.

Consult the Using StatsD or AppDynamics without connecting to Console section for instructions on configuring a license token, if your N|Solid processes are not connected to the N|Solid Console server.

Supported backends

StatsD is a standardized protocol that is supported by many data collection databases and services. Supported backends include AWS CloudWatch, DataDog, ElasticSearch, Graphite, InfluxDB, MongoDB, OpenTSDB, StackDriver, Zabbix and many more. See the StatsD documentation for a more complete list.

Connection options

Supply the NSOLID_STATSD environment variable when starting an N|Solid process to have the Agent attempt to connect to an endpoint. The format of this value is "host:port". If unspecified, the default host is localhost and port is 8125. The host can be specified as a hostname string, an IPv4 or IPv6 address.

Once connected, a suite of metrics will be sent to the collector using the StatsD protocol. The NSOLID_INTERVAL environment variable can be used to adjust the default reporting interval of 3 seconds. Be aware that this will also change the reporting interval for the N|Solid Console connection if connected via the NSOLID_COMMAND environment variable.

StatsD buckets

StatsD metrics are reported using "buckets". A bucket is the full name, or key, of the entry being reported. Buckets need to be descriptive enough to clearly identify the process, host and metric type being reported. It is important that you have enough granularity to be able to inspect the data at the level you require, while also retaining the ability to group metrics for aggregate reporting in your frontend using bucket wildcard mechanisms. This will depend somewhat on your reporting frontend and may require some experimentation.

By default, N|Solid creates bucket names prefixed with "nsolid.<env>.<app>.<hostname>.<shortId>.", followed by the name of the metric. In this string:

• env is the value of the NSOLID_ENV environment variable if it exists, or the "env" property of the "nsolid" section in package.json if it exists. Defaults to "prod" if not otherwise supplied.
• app is the value of the NSOLID_APP environment variable if it exists, or the "app" property of the "nsolid" section in package.json if it exists, or the "name" property in package.json if it exists, or process.title if it has been set. Defaults to "untitled application" if not otherwise supplied.
• hostname is the value of the NSOLID_HOSTNAME environment variable if it exists, or the "hostname" property of the "nsolid" section in package.json if it exists. Otherwise, the value of os.hostname() is used.
• shortId is the first 7 characters of the random alphanumeric ID created by the N|Solid Agent at startup. This can be used to uniquely identify your processes. The full 40 character string is also available as id if required.

To override the default StatsD metric bucket strings, provide a string via the NSOLID_STATSD_BUCKET environment variable to be used as the full prefix. ${key} style variables can be used to insert any of the above values. The default bucket prefix would be specified as follows: "nsolid.${env}.${app}.${hostname}.${shortId}".

Your StatsD data, by default, will be formatted like so when sent to the collector:

nsolid.prod.myapp.nodehost.803bbd5.uptime:314.4|g
nsolid.prod.myapp.nodehost.803bbd5.rss:28401664|g
nsolid.prod.myapp.nodehost.803bbd5.heapTotal:8425472|g
nsolid.prod.myapp.nodehost.803bbd5.heapUsed:5342488|g
...

StatsD tags

Some backends, such as DataDog support "tags" for metric reporting. By default, N|Solid does not append any tags to its metrics. If required, you can supply your own tags to be appended to all reported metrics from an individual process.

Using the NSOLID_STATSD_TAGS environment variable, or the statsdTags property of the nsolid section in package.json, supply a string with the same variable substitution format as for the buckets above. In addition to env, app, hostname, shortId and id variables, you can also make use of tags to insert the N|Solid tags that have been supplied for the process.

StatsD tags should be a comma-separated list of strings that your backend can decode. Refer to your backend documentation for how these values are made use of for reporting and whether their use will be suitable for your deployment.

Networking

N|Solid Console

The 3.x N|Solid Console is the component that provides data persistence for your measurements and artifacts, such as heap snapshots and CPU profiles. It communicates directly with N|Solid Agent processes, and provides the web interface and the API interface for nsolid-cli.

Command Line Arguments

These are the supported command line arguments if N|Solid Console is started explicitly.

Configuration File

N|Solid Console can be configured via a configuration file if the --config argument is specified. This file is formatted as TOML (similar to ini file format). A sample file is available in the file config-sample.toml in the root directory of the nsolid-console server. The contents of the file are also listed below.

To run with a customized configuration, copy the file and edit it so that it contains your customized values. You can omit keys from any section which you don't want to override, and omit entire sections if no keys in that section are overridden.

Alternatively, or in addition, you can set environment variables for every configuration key and value. The names of the environment variables are listed with the configuration keys and values below.

For more information on the TOML language used here, please see: https://github.com/toml-lang/toml

# config-sample.toml

#-------------------------------------------------------------------------------
# This is a sample configuration file for nsolid-console.  It shows the
# configuration keys and default values for those keys.
#
# To run with a customized configuration, copy this file and edit it so that it
# contains your customized values.  You can omit keys from any section which
# you don't want to override, and omit entire sections if no keys in that
# section are overridden.
#
# Then invoke nsolid-console with the -c or --config command line parameter
# and the name of the customized file.
#
# For example:
#    nsolid-console --config <your-config-file.toml>
#
# Alternatively, or in addition, you can set environment variables for
# every configuration key and value.  The names of the environment variables
# are listed with the configuration keys and values below.
#-------------------------------------------------------------------------------
# For more information on N|Solid, please see:
#    https://docs.nodesource.com/docs/
#
# For more information on the TOML language used here, please see:
#    https://github.com/toml-lang/toml
#-------------------------------------------------------------------------------

#-------------------------------------------------------------------------------
# Properties that are file or directory names can be absolute or relative file
# names. When relative, the names are considered relative to the configuration
# file they were specified in.  When using relative file names as environment
# variables, they will be considered relative to the current working directory.
#-------------------------------------------------------------------------------

#-------------------------------------------------------------------------------
# set the logging level - "info" or "debug" - debug is more verbose
#-------------------------------------------------------------------------------

# environment variable NSOLID_CONSOLE_LOG_LEVEL

logLevel = "info"

#-------------------------------------------------------------------------------
# settings for licensing
#
# key = "the-license-key"
#-------------------------------------------------------------------------------

[license]

# environment variable NSOLID_CONSOLE_LICENSE_KEY

key = ""

#-------------------------------------------------------------------------------
# settings for authentication
#
# requireLogin - If the console should require users to authenticate. This
#                is only enforced with accounts-generated license keys
# adminToken   - A random token that can be used by nsolid-cli for auth
#-------------------------------------------------------------------------------

# environment variable NSOLID_CONSOLE_AUTH_REQUIRE_LOGIN
# environment variable NSOLID_CONSOLE_AUTH_ADMIN_TOKEN

[auth]

requireLogin = true
adminToken = ""

#-------------------------------------------------------------------------------
# settings for the API HTTP server
#
# proto  - "http" or "https"
# server - "<ip addr | hostname>:<port>"
#
# If the environment variable PORT is set, that value, prefixed with ':', will
# be set to the server property. Eg, using PORT=42 will set web.server to ':42'.
#
# httpProxy and httpsProxy are used to set an explicit HTTP/S proxy for HTTP/S
# requests that console makes to the public internet.  You can also set the
# environment variables HTTP_PROXY or http_proxy, and HTTPS_PROXY or https_proxy
# as the value to use.
#-------------------------------------------------------------------------------

[web]

# environment variable NSOLID_CONSOLE_WEB_PROTO
# environment variable NSOLID_CONSOLE_WEB_SERVER
# environment variable NSOLID_CONSOLE_WEB_HTTP_PROXY
# environment variable NSOLID_CONSOLE_WEB_HTTPS_PROXY

proto      = "http"
server     = "0.0.0.0:6753" # 0.0.0.0 allows access on all network interfaces
httpProxy  = ""
httpsProxy = ""

#-------------------------------------------------------------------------------
# settings for running the API HTTP server in https mode
#
# key and cert are file names, as from Node.js https.createServer() options
#
# when set to blank, the files inside the nsolid-console/etc folder will be used:
#   key:  nsolid-self-signed.key
#   cert: nsolid-self-signed.crt
#
# See note above concerning relative paths.
#
# note: generate new default credentials with ./gen-https-creds.sh
# note: default credentials files shipped with nsolid-console expire in 08/2026
#-------------------------------------------------------------------------------

[web.https]

# If set, secureOptions is a bitmask-formatted string of any crypto constants
# documented at https://nodejs.org/api/crypto.html#crypto_openssl_options, eg:
#    "SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1 | SSL_OP_NO_TLSv1_1"
# ciphers should be an OpenSSL cipher list formatted string, see
# https://www.openssl.org/docs/man1.0.2/apps/ciphers.html#CIPHER-LIST-FORMAT

# environment variable NSOLID_CONSOLE_WEB_HTTPS_KEY
# environment variable NSOLID_CONSOLE_WEB_HTTPS_CERT
# environment variable NSOLID_CONSOLE_WEB_HTTPS_SECURE_OPTIONS
# environment variable NSOLID_CONSOLE_WEB_HTTPS_CIPHERS

key           = ""
cert          = ""
secureOptions = ""
ciphers       = ""


#-------------------------------------------------------------------------------
# zmq sockets for access by N|Solid processes
#
# commandBindAddr, dataBindAddr, bulkBindAddr are the zmq port addresses
# that will be bound by the server.
#
# publicKey, privateKey are the keys used to encrypt the messages sent over
# zmq.  For more information see:
# https://github.com/msealand/zmq-zap.node/blob/master/examples/curve.js
#
# HWM, bulkHWM are used to set the high water mark for zmq's internal buffering.
# The value indicates the number of messages to buffer, and 0 indicates no limit.
#
# The N|Solid agent will use environment variables that correspond to these
# settings.  For the sockets, the following N|Solid agent environment variable
# should use an address that it can use to connect to the socket the server has
# bound.
#
#    N|Solid agent env vars    server configuration values
#    ----------------------    ---------------------------
#    NSOLID_COMMAND            sockets.commandBindAddr
#    NSOLID_DATA               sockets.dataBindAddr
#    NSOLID_BULK               sockets.bulkBindAddr
#    NSOLID_PUBKEY             sockets.publicKey
#    NSOLID_SAAS               sockets.saas
#
# If running the console server in server mode AND relay mode on the same host,
# the configuration values for the sockets.xxxBindAddr above must be different
# for the server mode and relay configurations.
#-------------------------------------------------------------------------------

[sockets]

# For commandBindAddr, dataBindAddr, and bulkBindAddr, the format should be:
#    tcp://[ipAddress]:[port]
# [ipAddress] should be an ip address - host names cannot be used; use
# "*" to addresses on any available network interface.

# environment variable NSOLID_CONSOLE_SOCKETS_COMMAND_BIND_ADDR
# environment variable NSOLID_CONSOLE_SOCKETS_DATA_BIND_ADDR
# environment variable NSOLID_CONSOLE_SOCKETS_BULK_BIND_ADDR
# environment variable NSOLID_CONSOLE_SOCKETS_PUBLIC_KEY
# environment variable NSOLID_CONSOLE_SOCKETS_PRIVATE_KEY
# environment variable NSOLID_CONSOLE_SOCKETS_HWM
# environment variable NSOLID_CONSOLE_SOCKETS_BULK_HWM
# environment variable NSOLID_CONSOLE_SOCKETS_SAAS

commandBindAddr = "tcp://*:9001"
dataBindAddr    = "tcp://*:9002"
bulkBindAddr    = "tcp://*:9003"
publicKey       = "^kvy<i^qI<r{=ZDrfK4K<#NtqY+zaH:ksm/YGE6I"
privateKey      = "2).NRO5d[JbEFli7F@hdvE1(Fv?B6iIAn>NcLLDx"
HWM             = 0
bulkHWM         = 0
saas            = ""

# The following socket address values are only relevant when running in --relay
# mode. They are analogs of their similiarly named 'xxxBindAddr' values, but
# used as the zmq sockets of the console server to connect to.  These should
# be formatted as "tcp://host:port"
#
# If you want to use a different keypair on the remote endpoint, you may
# specify a remotePublicKey matching that endpoint.  This can be used to
# perform a rolling update to keys without losing data from agents with the old
# encryption key (by using multiple relays)

# environment variable NSOLID_CONSOLE_SOCKETS_COMMAND_REMOTE_ADDR
# environment variable NSOLID_CONSOLE_SOCKETS_DATA_REMOTE_ADDR
# environment variable NSOLID_CONSOLE_SOCKETS_BULK_REMOTE_ADDR
# environment variable NSOLID_CONSOLE_SOCKETS_REMOTE_PUBLIC_KEY

commandRemoteAddr = ""
dataRemoteAddr    = ""
bulkRemoteAddr    = ""
remotePublicKey   = ""

#-------------------------------------------------------------------------------
# Relay mode settings
#
# These settings are specific to running in relay mode.
#
# maxBufferSizeMb is the max buffering "hwm" values shared between the
# "pull/push" sockets in megabytes.
#
# flushInterval is the delay in milliseconds between message sends when the
# buffer is being flushed after a reconnect.
#
# cleanupSizeMb is the amount of megabytes of buffers to be dropped when limits
# are reached.
#
# logSizeInterval is the interval in seconds that will trigger a buffering
# status to be logged. Only happens when upstream (remote) sockets are
# disconnected.
#-------------------------------------------------------------------------------

[relay]

# environment variable NSOLID_CONSOLE_RELAY_MAX_BUFFER_SIZE_MB
# environment variable NSOLID_CONSOLE_RELAY_FLUSH_INTERVAL
# environment variable NSOLID_CONSOLE_RELAY_CLEANUP_SIZE_MB
# environment variable NSOLID_CONSOLE_RELAY_LOG_SIZE_INTERVAL

maxBufferSizeMb = -1
flushInterval   = 1
cleanupSizeMb   = 100
logSizeInterval = 10

#-------------------------------------------------------------------------------
# data settings
#
# dir           - the directory where data will be persisted
# retentionDays - the number of days of metric data that will be retained
# oldDataThresholdMs - how old a record should be to be considered backfill
# backfillTimeoutMs  - after not seeing old data for this long, switch to
#                      standard mode and backfill downsampled records
# dbname        - Influxdb database name
#
# See note above concerning relative paths.
#-------------------------------------------------------------------------------

[data]

# environment variable NSOLID_CONSOLE_DATA_DIR
# environment variable NSOLID_CONSOLE_DATA_RETENTION_DAYS
# environment variable NSOLID_CONSOLE_DATA_OLD_DATA_THRESHOLD_MS
# environment variable NSOLID_CONSOLE_DATA_BACKFILL_TIMEOUT_MS
# environment variable NSOLID_CONSOLE_DATA_DBNAME

dir                = "~/.nsolid-console/data"
retentionDays      = 31
oldDataThresholdMs = 10000
backfillTimeoutMs  = 30000
dbname             = 'database'

#-------------------------------------------------------------------------------
# influxdb settings
#
# url            - base url to connect to externally managed influxdb
# user           - user for the influxdb API, could de local or external influxdb
# password       - password credentials for the Influxdb API
# org            - The influxdb organization to write on
# token          - The influxdb API token
# defaultBucket  - The default bucket (where time series are stored)
# uiDisabled     - Disable/eneble the InfluxDB user interface (disabled by default)
# retentionPeriodSeconds - retention in seconds for the default bucket
# maxConcurrency - Amount of concurrent workers writting metrics
#-------------------------------------------------------------------------------

# environment variable NSOLID_CONSOLE_INFLUXDB_URL
# environment variable NSOLID_CONSOLE_INFLUXDB_USER
# environment variable NSOLID_CONSOLE_INFLUXDB_PASSWORD
# environment variable NSOLID_CONSOLE_INFLUXDB_ORG
# environment variable NSOLID_CONSOLE_INFLUXDB_TOKEN
# environment variable NSOLID_CONSOLE_INFLUXDB_DEFAULT_BUCKET
# environment variable NSOLID_CONSOLE_INFLUXDB_UI_DISABLED
# environment variable NSOLID_CONSOLE_INFLUXDB_RETENTION_PERIOD_SECONDS
# environment variable NSOLID_CONSOLE_INFLUXDB_MAX_CONCURRENCY

[influxdb]

url = ""
user = "nsolid"
password = "L+T;95cTBC}~jPnj"
org = "nsolid"
token = "nsolid-token"
defaultBucket = "agentData"
uiDisabled = true
retentionPeriodSeconds = 3600
maxConcurrency = 2

[[influxdb.policies]]
name = "agentData"
retentionHours = 1
shardRetentionHours = 1
startupRetentionHours = 24
startupShardRetentionHours = 24
default = true

[[influxdb.policies]]
name = "appAverageData"
retentionHours = 1
shardRetentionHours = 1
startupRetentionHours = 24

[[influxdb.policies]]
name = "one_minute_daily"
retentionHours = 24
shardRetentionHours = 1
startupRetentionHours = 24

[[influxdb.policies]]
name = "one_hour_monthly"
retentionDays = 31
shardRetentionDays = 7
useSettingsDuration = true

[[influxdb.policies]]
name = "rp_events"
retentionDays = 365
shardRetentionDays = 7

#-------------------------------------------------------------------------------
# notification settings
#
# consoleURL - the base URL of the console, for use in providing links
# back to the console in notifications.
# stackFrames - the number of frames to be shown in Slack event loop blocked
# notifications
#-------------------------------------------------------------------------------
[notification]

# environment variable NSOLID_CONSOLE_NOTIFICATION_CONSOLE_URL
# environment variable NSOLID_CONSOLE_NOTIFICATION_STACK_FRAMES

consoleURL = "http://localhost:6753"
stackFrames = 10

#-------------------------------------------------------------------------------
# view settings
#
# interval - how often to process saved views
#-------------------------------------------------------------------------------
[view]

interval = 5000

#-------------------------------------------------------------------------------
# anomaly detection settings
#
# memAnomalyTimeout - timeout for anomaly
#-------------------------------------------------------------------------------

# environment variable NSOLID_CONSOLE_MEMORY_ANOMALY_TIMEOUT

[anomaly]

timeout = 60000

#-------------------------------------------------------------------------------
# log settings
#
# influx - the name of the file where the influxd log will be written
#
# See note above concerning relative paths.
#-------------------------------------------------------------------------------

[logs]

# environment variable NSOLID_CONSOLE_LOGS_INFLUX

influx = "~/.nsolid-console/influxdb.log"

#-------------------------------------------------------------------------------
# vulnerability settings
#
# refreshMinutes - how often the vulnerability database will be refreshed
#-------------------------------------------------------------------------------

[vulnerabilities]

# environment variable NSOLID_CONSOLE_VULNERABILITIES_REFRESH_MINUTES

refreshMinutes = 30

#-------------------------------------------------------------------------------
# assets settings - storage of heap snapshots, cpu profiles, etc
#
# cleanupMinutes  - how often the asset files are cleaned up; the oldest asset
#                   files are deleted to ensure constraints below are met.
# maxAssets       - maximum number of assets to retain
# maxAssetsMb     - maximum size of all assets to retain
# summaryTimeout  - Set the minutes you want the worker pool task run for
# summaryLimit    - Set to false, to let the task in the pool run indefinitely
#                   until they complete
#-------------------------------------------------------------------------------

[assets]

# environment variable NSOLID_CONSOLE_ASSETS_CLEANUP_MINUTES
# environment variable NSOLID_CONSOLE_ASSETS_MAX_ASSETS
# environment variable NSOLID_CONSOLE_ASSETS_MAX_ASSETS_MB
# environment variable NSOLID_CONSOLE_ASSETS_SUMMARY_TIMEOUT
# environment variable NSOLID_CONSOLE_ASSETS_SUMMARY_LIMIT

cleanupMinutes  = 5
maxAssets       = 500
maxAssetsMb     = 100
summaryTimeout  = 0
summaryLimit    = true

Relay Mode

N|Solid Console supports running in a relay mode in order to support proxying through firewalls, buffering agent data during downtime, and scaling with complex network topologies.

The relay sits between your Runtime instances and N|Solid Console. In this mode, nsolid-console looks like an N|Solid Console instance to agents but does not have any N|Solid Console functionality enabled.

Relay Configuration

To use relay mode, set the --relay flag and the three [sockets] variables described below. These can be provided via the configuration file or as environment variables. If omitted, the defaults will be used.

Changing the NSOLID_COMMAND, NSOLID_DATA, and NSOLID_BULK variables is recommended, as those will be the ones that the relay itself listens on while it uses the NSOLID_CONSOLE_SOCKETS-* variables in order to connect to N|Solid Console.

In the simplest case, set the NSOLID_CONSOLE_SOCKETS_* variables to the same values that the related NSOLID_* variables were set to, picking different addresses for the NSOLID_CONSOLE_SOCKETS_* variables.

Environment Variables

Configuration File

The following example shows the [sockets] section of a configuration file written to support relay mode. Note that the related Bind addresses have been changed, and an explicit public key has been specified for the remote Console server.

# config-sample.toml

[sockets]
commandBindAddr = "tcp://<console-address>:8001"
dataBindAddr    = "tcp://<console-address>:8002"
bulkBindAddr    = "tcp://<console-address>:8003"

commandRemoteAddr = "tcp://<console-address>:9001"
dataRemoteAddr = "tcp://<console-address>:9002"
bulkRemoteAddr = "tcp://<console-address>:9003"
remotePublicKey = "^kvy<i^qI<r{=ZDrfK4K<#NtqY+zaH:ksm/YGE6I"

Docker Compose File

Relay mode environment variables need to be added to the console section of the Docker configuration file.

For example, in order to match the settings in the above configuration file, add the following:

# docker-compose.yml

  console:
    environment:
      - NSOLID_CONSOLE_SOCKETS_COMMAND_BIND_ADDR=console:8001
      - NSOLID_CONSOLE_SOCKETS_DATA_BIND_ADDR=console:8002
      - NSOLID_CONSOLE_SOCKETS_BULK_BIND_ADDR=console:8003
      - NSOLID_CONSOLE_SOCKETS_COMMAND_REMOTE_ADDR=console:9001
      - NSOLID_CONSOLE_SOCKETS_DATA_REMOTE_ADDR=console:9002
      - NSOLID_CONSOLE_SOCKETS_BULK_REMOTE_ADDR=console:9003

See the Docker page for more information on setting up Docker Compose.

External InfluxDB

If you already maintain an instance or cluster of InfluxDB enterprise or want to use Influx Cloud and want N|Solid Console to use your database to store its own metrics, you may do so using the NSOLID_CONSOLE_INFLUXDB_URL, NSOLID_CONSOLE_INFLUXDB_USER and NSOLID_CONSOLE_INFLUXDB_PASSWORD environment variables or set the values at the corresponding configuration file section. This should point to the http(s) listener for your InfluxDB and include the correct protocol and auth credentials if needed.

This will look similar to

NSOLID_CONSOLE_INFLUXDB_URL=http://my-influx-host:8086
NSOLID_CONSOLE-INFLUXDB_USER=myuser
NSOLID_CONSOLE-INFLUXDB_PASSWORD=secret

If errors occur reading or writing to this instance, a notification banner should appear in your console as well as specific error messages in the N|Solid Console log.

N|Solid Console will create and manage its own database and retention policies in the InfluxDB and should be configured using the standard configuration options.

Note: The supported InfluxDB version is v2.x, the InfluxDB instance must be a fresh instance with no initial user, organization and bucket defined; otherwise, N|Solid Console won't set up its standard options or even work.

Advanced Environment Variables

The variables described below should only be necessary if a proxy is required in front of N|Solid Console.

If this is not needed, ignore this section as the variables will be configured by the N|Solid agent via the COMMAND interface.

Moving the N|Solid Console server

The data maintained in the N|Solid Console server is stored in a directory determined by the dir property of the [data] section of the configuration file. The default value is ~/.nsolid-console/data. If the contents of the directory are copied to a new machine, the N|Solid Console server can be run with its configuration pointing to the copied directory.

Docker

For people who need to use N|Solid in a Dockerized environment, NodeSource has developed the N|Solid Docker images.

These images are built with the enterprise customer in mind, developing a versatile set of independently scalable Docker images that work together to provide a unified N|Solid experience across all of your Dockerized infrastructure.

In addition to being friendly to enterprise operations teams, the N|Solid Docker Images have been designed to be accessible to developers. For those who are already using Docker, these images provide an easy way to get up and running with the N|Solid console. With the use of docker-compose, you can now get the full N|Solid Suite on your local machine with a single command.

Overview of N|Solid Docker Images

We provide 3 separate Docker Images for N|Solid, allowing each component of the platform to be scaled and deployed independently.

All of the images are built on the official ubuntu:bionic image maintained by the Docker project. The latest tag defaults to the latest Node.js v16.x Gallium LTS release. For more details please refer to the docker-nsolid github repository.

Installing the N|Solid Docker Images

Setting up Docker

First, follow the steps for installing Docker for your operating system.

Although not required, it is recommended that you use docker-compose when first getting started. It simplifies managing these Docker images when running them locally.

Docker Compose

The docker-compose file below provides an example of the N|Solid platform configuration. It is also useful as a starting point for your docker setup.

The following docker-compose file doesn't include any N|Solid applications yet. We will explain how to add your application below. Use a valid path in the volumes configuration, or remove it if persistence is not important to you.

# nsolid.yml

version: "2"
services:
  console:
    image: nodesource/nsolid-console
    container_name: nsolid.console
    ports:
      - 9001:9001
      - 9002:9002
      - 9003:9003
      - 6753:6753
    environment:
      - NSOLID_CONSOLE_LICENSE_KEY=xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    volumes:
      - /path/to/persistent/console:/var/lib/nsolid/console
     networks:
      - nsolid
networks:
  nsolid:

Note: The NSOLID_CONSOLE_LICENSE_KEY should be a valid license key not a JWT license token, you can see more information about license tokens here.

Downloading the Docker Images

To download the images using docker-compose, open a terminal in the directory where nsolid.yml is kept, and run:

$ docker-compose -f nsolid.yml pull

This will download all of the images needed to run the N|Solid console locally.

To download the images using docker, execute the following commands:

$ docker pull nodesource/nsolid-console

# Optional CLI
$ docker pull nodesource/nsolid-cli

Using the N|Solid Docker Images

Starting the N|Solid Console

With Docker Compose

To bring up the N|Solid Console, run the following command:

$ docker-compose -f nsolid.yml up

By default, this will bring up the N|Solid Console available on 127.0.0.1:6753.

With Docker

Though using docker-compose is our recommended approach for local development, the N|Solid console can be started with docker:

# start console
$ docker run -d -p 9001-9003:9001-9003 -p 6743:6753 \
  -v /path/to/persistent/console:/var/lib/nsolid/console --name console  \
  --network docker_nsolid \
  nodesource/nsolid-console

This will bring up the N|Solid console available on localhost:6753.

Make sure you have a Docker network to link your containers, you can create one with:

$ docker network create docker_nsolid

Visit the docker docs) for more information on Docker container networking.

Creating an Example Docker Application

Create a file called server.js:

// server.js

var http = require('http');
var hostname = require('os').hostname();
var port = process.env.PORT || 8888;

var server = http.createServer(function handleRequest (request, response) {
  response.end('[' + hostname + '] Serving requests from myapp. Request URL:' + request.url);
});

server.listen(port, function () {
  console.log('Server listening on port', port);
});

Create a file called Dockerfile:

# Dockerfile

FROM nodesource/nsolid

COPY server.js /server.js

EXPOSE 8888

CMD ["nsolid", "/server.js"]

Build docker image:

$ docker build -t example .

Starting Example Application with Docker Compose

Create the file docker-compose.yml in the directory along-side nsolid.yml:

# docker-compose.yaml

version: "2"
services:
  example:
    image: example
    container_name: nsolid.example
    ports:
      - 8888:8888 # Port your application exposes
    environment:
      - NSOLID_APPNAME=example
      - NSOLID_COMMAND=console:9001
      - NSOLID_DATA=console:9002
      - NSOLID_BULK=console:9003
networks:
      - nsolid
networks:
  nsolid:

For the complete documentation on defining a service with docker-compose.yml, refer to the Docker project’s documentation page: https://docs.docker.com/compose/overview/.

At this point, you are ready to bring up your application using docker-compose:

$ docker-compose -f nsolid.yml -f docker-compose.yml up

Starting Example Application with Docker

Start your service:

# start and connect your application to N|Solid
$ docker run -d --name example -e 'NSOLID_APPNAME=example' -e 'NSOLID_COMMAND=console:9001' -e 'NSOLID_DATA=console:9002' -e 'NSOLID_BULK=console:9003' --network docker_nsolid example

Adding Your Own Application with Docker Compose

If you are new to Docker, follow the steps in our blog post to get your application into a Docker container.

Your Dockerfile should begin with the following line:

#Dockerfile

FROM nodesource/nsolid

Congratulations, you are now up and running with N|Solid and Docker!

AWS Quick Start Guide

Amazon Web Services (AWS) offers reliable, scalable, and inexpensive cloud computing services. We offer a dedicated repository to help you deploy N|Solid on AWS quickly and easily. We have several CloudFormation templates already written, as well as a list of AMI IDs for every region.

License Key

If you don't already have a license key, please visit the NodeSource Accounts website to start your trial service.

Getting Started

Easily run N|Solid on AWS using our CloudFormation templates in the dedicated GitHub repository. You can find a list of templates and their descriptions in the README.md. Once you find a template that you want to use, follow these steps to use the CloudFormation templates in nsolid-aws:

1. Find the template you want to run in the /templates folder, then click the Deploy to AWS button.

2. This will open up CloudFormation in your own account. Click the Next button.

3. Fill in the required parameters and change the Stack Name if desired. Then click the Next button.

4. Adjust any CloudFormation options if desired. Click Next.

5. If the template requires IAM capabilities, you will need to check the "I acknowledge that AWS CloudFormation might create IAM resources with custom names." box. Once you are ready, click the Create button.

AMI List

You can also use our N|Solid AMIs for your own projects. See AMI-LIST.md for a full list of AMI IDs in every region.

Running Node Applications in N|Solid

ssh into the machine as the ubuntu user and set these four N|Solid environment variables:

$ ssh -i ~/.ssh/your-aws-key.pem ubuntu@<ip-address>

$ NSOLID_COMMAND=localhost:9001 \
$ NSOLID_DATA=localhost:9002 \
$ NSOLID_BULK=localhost:9003 \
$ NSOLID_APPNAME="My_Application" node app.js

You can customize the name of your application in the N|Solid Console by changing the NSOLID_APPNAME environmental variable. If you do not specify an app name, N|Solid will look for a name property in your package.json, and if that is not found, your application name will default to "untitled application".

You are now ready to use N|Solid for your own applications!

Configuring N|Solid Console

Process Management

• Nginx: /etc/init.d/nginx
• N|Solid Console: /etc/init.d/nsolid-console

Nginx

The configuration for the external proxy is located at /etc/nginx/sites-enabled/nsolid-nginx.conf.

N|Solid Console

The default configuration file is located at /etc/nsolid/console-config.toml with all artifacts stored at /var/lib/nsolid/console.

Microsoft Azure Quickstart Guide

Azure is a comprehensive set of cloud services that developers and IT professionals use to build, deploy, and manage applications through a global network of datacenters. We offer a dedicated repository to help you deploy N|Solid on Azure quickly and easily. We have several Resource Manager templates already written.

License Key

If you don't already have a license key, please visit the NodeSource Accounts website to start your trial service.

Getting Started

Easily run N|Solid on Azure using our Resource Manager templates. You can find a list of templates and their descriptions in the README. Once you find a template you want to use, follow these steps to use the Resource Manager templates in nsolid-azure:

1. Make a copy of our N|Solid VHDs into your own account for use in the templates.

$ az storage blob copy start --destination-blob nsolid-runtime-disk --destination-container <your-storage-container> --source-uri <runtime-vhd-uri> --account-name <your-storage-account-name>

$ az storage blob copy start --destination-blob nsolid-console-disk --destination-container <your-storage-container> --source-uri <console-vhd-uri> --account-name <your-storage-account-name>

2. Find the template you want to run in the /templates folder, then click the Deploy to Azure button. This will open up a Custom Deployment in Azure Resource Manager in your own account.

3. Fill in the required parameters with your VHD URI's created in step 1.

4. Select either an existing Resource Group or create a new one.

5. Set the location to "West US".

6. Agree to the Terms and Conditions.

7. Click the Purchase button.

Image List

You can also use our N|Solid Images for your own projects. See IMAGE-LIST.md for a full list of Image IDs.

Connecting the N|Solid Runtime to the N|Solid Console on Azure

To connect an N|Solid Runtime to your N|Solid Console, SSH into your N|Solid Runtime instance, and run the following commands:

$ ssh ns@<ip-address>  
# password is nodesource

$ NSOLID_COMMAND={nsolid_console_ip_address}:9001 \
$ NSOLID_DATA={nsolid_console_ip_address}:9002 \
$ NSOLID_BULK={nsolid_console_ip_address}:9003 \
$ NSOLID_APPNAME="My Application Name" node app.js

In the above code block, you will need to replace {nsolid_console_ip_address} with the IP address of your N|Solid Console. You will also want to replace "My Application Name" with the name of your application, as you want it to be displayed in the N|Solid Console.

You can customize the name of your application in the N|Solid Console by changing the NSOLID_APPNAME environmental variable. If you do not specify an app name, N|Solid will look for a name property in your package.json, and if that is not found, your application name will default to "untitled application".

You can connect multiple N|Solid Runtimes to your N|Solid Console, enabling you to monitor all your Node.js deployments from a single dashboard.

You are now ready to use N|Solid for your own applications!

Configuring N|Solid Console

Process Management

• Nginx: /etc/init.d/nginx
• N|Solid Console: /etc/init.d/nsolid-console

Nginx

The configuration for the external proxy is located at /etc/nginx/sites-enabled/nsolid-nginx.conf.

N|Solid Console

The default configuration file is located at /etc/nsolid/console-config.toml and will configure all artifacts stored at /var/lib/nsolid/console.

GCP Quick Start Guide

Google Cloud Platform (GCP) lets you build and host applications and websites, store data, and analyze data on Google's scalable infrastructure. We offer a dedicated repository to help you deploy N|Solid on GCP quickly and easily, with several Deployment Manager templates already written.

License Key

If you don't already have a license key, please visit the NodeSource Accounts website to start your trial service.

Getting Started

Easily run N|Solid on GCP using our Deployment Manager templates. You can find a list of templates and their descriptions in the README.md. Once you find a template that you want to use, follow these steps to use the Deployment Manager templates in nsolid-gcp:

1. Clone the repository to your computer:

$ git clone https://github.com/nodesource/nsolid-gcp

2. Find the template you want to run in the /templates folder.

3. Execute the gcloud Deployment Manager command to create the N|Solid Deployment:

$ gcloud deployment-manager deployments create nsolid --config templates/nsolid-quick-start/nsolid.yaml

Image List

You can also use our N|Solid Images for your own projects. See IMAGE-LIST.md for a full list of Image IDs.

Running Node Applications with N|Solid

ssh into the machine as ubuntu user and set these four N|Solid env variables:

$ ssh -i ~/.ssh/your-gcp-key.pem ubuntu@<ip-address>

$ NSOLID_COMMAND=<NSOLID_CONSOLE_IP>:9001 \
$ NSOLID_DATA=<NSOLID_CONSOLE_IP>:9002 \
$ NSOLID_BULK=<NSOLID_CONSOLE_IP>:9003 \
$ NSOLID_APPNAME="My_Application" node app.js

You can customize the name of your application in the N|Solid Console by changing the NSOLID_APPNAME environmental variable. If you do not specify an app name, N|Solid will look for a name property in your package.json, and if that is not found, your application name will default to "untitled application".

You are now ready to use N|Solid for your own applications!

Configuring N|Solid Console

Process Management

• Nginx: /etc/init.d/nginx
• N|Solid Console: /etc/init.d/nsolid-console

Nginx

The configuration for the external proxy is located at /etc/nginx/sites-enabled/nsolid-nginx.conf.

N|Solid Console

The default configuration file is located at /etc/nsolid/console-config.toml, with all artifacts stored at /var/lib/nsolid/console.

N|Solid for Cloud Run in GKE

Cloud Run on GKE is a GCP service that provides a simpler developer experience for deploying stateless services to your GKE cluster. To complement convenience with control, the N|Solid base-image for Cloud Run on GKE provides developers with a drop-and-replace Node.js runtime that delivers sophisticated performance insights out of the box and in production with zero code-modification.

What is Google Cloud Run

Cloud Run abstracts away Kubernetes concepts while providing automatic scaling based on HTTP requests; scaling to zero pods; automatic networking; and integration with Stackdriver. Running in your cluster enables access to custom machine types, PC/Compute Engine networks, and the ability to run side-by-side with other workloads deployed into your cluster. Cloud Run on GKE, based on Knative, provides a consistent experience which enables you to run your serverless workloads anywhere: fully managed on Google Cloud, on GKE, or on your own Kubernetes cluster.

Getting Started

Deploying an application with N|Solid is as easy as:

• Signing up for a free account at accounts.nodesource.com
• Setting up your N|Solid Console to GCP
• Deploying your application on CloudRun.

Other Prerequisites

Please ensure you have the following prerequisites covered:

• Make sure Docker is installed
• Create a Google Cloud account. (You can sign up for a free trial!)
• Create a Project in Google Cloud Dashboard (see here).
• Ensure you have the the Google Cloud SDK installed.

Signing up

You can follow these easy steps to get set up with a NodeSource account: here.

Setting up your N|Solid Console

The N|Solid Console can be set up via the Google Cloud Deployment Manager, which allows you to specify all the resources needed for your application in a declarative format using yaml.

NodeSource provides an easy-to-use deployment template.

Follow these steps to use the Deployment Manager templates:

Visit the GCP Deployment manager at https://console.cloud.google.com/dm

Note down your Project ID

Setup gcloud defaults

  gcloud auth login
  gcloud config set project <PROJECT_ID>
  gcloud config set compute/zone <ZONE>

Your PROJECT_ID can be found in your dashboard

A Cluster is deployed to a single zone. You can find more about zones here. Or you can select one from the list generated from gcloud compute zones list

In a terminal, execute the following template

  gcloud deployment-manager deployments create nsolid --config https://raw.githubusercontent.com/nodesource/nsolid-gcp/master/templates/nsolid-console-only/nsolid-console.yaml

Navigating back to the Deployment Manager, you should now be able to see the setup being in progress.

Once deployed you can select nsolid in the Deployment Manager console and see the resources nsolid to run successfull

Select the nsolid console click on Manage Resources in the top right corner

In the resource view locate the external IP and copy it into a browser window

You should be able to access the nsolid console now

Congratulations, you have successfully deployed the nsolid console to GCP.

Sign in to the nsolid console using your NodeSource account credentials:

This will allow you to select whether you wish to log in with your personal org organization account.

Can`t see your organization on the screen? Visit accounts.nodesource.com to create an organization or contact your console administrator to receive an invitation.

Finally, you will be shown the nsolid console landing view.

The next step is to deploy your Node.js application to CloudRun and connect the runtime to the console so that you can view your application`s process performance and metrics.

Deploying your application on Cloud Run

Deploying your application with N|Solid and hooking it up to your console can be accomplished in a few easy steps.

Add the following docker file to your application (name it Dockerfile - no extensions)

FROM nodesource/nsolid:dubnium-latest

RUN mkdir -p /usr/src/app

WORKDIR /usr/src/app

COPY package.json /usr/src/app/package.json
RUN npm install --production
COPY . /usr/src/app/

ENTRYPOINT ["nsolid", "app.js"]

Now build your docker image

$ docker build -t gcr.io/[project-ID]/walkthrough:latest .

[project-ID] is the ID you captured in 'step Note down your project ID'.

Push your Docker image to the Google Container Registry

$ docker push gcr.io/[project-ID]/walkthrough:latest

[project-ID] is the ID you captured in 'step Note down your project ID'.

Note: Should you receive a warning that you don't have the needed permissions to perform this operation you can review GCP`s advanced authentication methods here.

To check whether you image has been successfully pushed go to https://console.cloud.google.com/gcr/images to see your image listed here:

Set up your Kubernetes Cluster on GKE

Visit console.cloud.google.com/kubernetes and create a new cluster:

Complete the form and select the More node pool options button

Select the Allow full access to all Could APIs option:

Select Save and Create to create your cluster.

Set up you cloud service with Cloud Run

Visit console.cloud.google.com/run and select CREATE SERVICE

Under container image URL select the image you pushed in step Push your Docker image to the Google Container Registry.

Next select your GKE Cluster you created in step 'Set up you Kubernetes Cluster on GKE'.

In the show optional settings section, add the following environmental variables:

The NSOLID_COMMAND environmental variable comprises of two components.

[External IP]:[Port]

[External IP]: This is the external IP of your console deployment see here

[Port]: This is the port number displayed on the N|Solid console once you successfully authenticated.

Create the service.

Patch your Kubernetes cluster to enable egress for a Cloud Run service

As a last step you will be required to patch your kubernetes cluster.

In a terminal type:

$ gcloud container clusters describe [CLUSTER_NAME] | grep -e clusterIpv4Cidr -e servicesIpv4Cidr

Where [CLUSTER_NAME] is replaced with our cloud run cluster that was set up Set up you Kubernetes Cluster on GKE Step.

This will produce the following output:

clusterIpv4Cidr: 10.XX.X.X/XX
servicesIpv4Cidr: 10.XX.XXX.X/XX

Finally run:

$ kubectl patch ksvc [SERVICE_NAME] --type merge -p '{"spec":{"runLatest":{"configuration":{"revisionTemplate":{"metadata":{"annotations":{"traffic.sidecar.istio.io/includeOutboundIPRanges": "[IP_ADDRESS_RANGES]"}}}}}}}'

Where [SERVICE_NAME] is replaced with the name of your Cloud Run name you chose in Set up you cloud service with Cloud Run.

Replace [IP_ADDRESS_RANGES] with the output of the previous command, comma separated.

In this case 10.XX.X.X/XX, 10.XX.XXX.X/XX.

This will only have to be done once per new service.

When navigating back to the console you should now see your application's processes in NSolid's application overview

Congratulations. You successfully deployed the nsolid console and an application to Cloud Run.

CPU Profiling

CPU Profiling allows you to understand where opportunities exist to improve the speed and load capacity of your Node processes.

Analyzing the Function Call Stack

Generated CPU profiles can be used to indicate all of the functions on the function call stack. For instance, if a function foo() calls a function bar() during its execution, and a sample was taken while bar() was running, the function call stack will show that foo() called bar(). Because multiple samples may be taken while bar() is executing, there will be an approximate start and stop time recorded for bar(), which is an indication of how long it took to run. In addition, further samples before and after the ones that captured bar() will capture foo(), and likewise to the bottom of the function call stack.

This data can then be analyzed to show, for the function foo(), how much time was actually spent in foo() and not in bar(). Every function has two time values, a self time and a total time. For the foo() and bar() case, if foo() only calls the bar() function, then the self time for foo() plus the total time for bar() will equal the total time for foo():

function foo() {
        … processing that takes a lot of time but calls no other functions ...
    bar()
        … processing that takes a lot of time but calls no other functions …
}

function bar() {
    …
}

foo() total time = foo() self time + bar() total time

Note: Total time shows you which functions are fastest and slowest, from start to finish, but does not definitively tell you if the time was spent in that function or other functions.

Named Functions

Named functions are easier to spot in CPU profiles. The stack frame entries available in a CPU profile include the name of a function, and source code location information for detail views. For anonymous functions, the name will often be displayed as "(anonymous)". In some cases, the V8 JavaScript engine can calculate an "inferred name" for a function. When possible, name functions so that they can be easily spotted in a CPU profile.

For instance, a function busyFunction(), which you like to easily track the callers for, is being called from an anonymous function:

setInterval(function(){busyFunction()}, 1000)

In a CPU profile, you'll see that busyFunction() is being called by (anonymous).

To make this easier to spot in a CPU profile, you can simply use:

setInterval(function busyFunctionCaller(){busyFunction()}, 1000)

In the CPU profile, you'll now see that busyFunction() is called by busyFunctionCaller().

For additional code cleanliness and clearer profiling, consider moving the entire function out into the same scope as the function usage:

setInterval(busyFunctionCaller, 1000)
//...
function busyFunctionCaller() {
  busyFunction()
}

Because JavaScript functions are "hoisted" to the top level of the scope they're defined in, you can reference busyFunctionCaller before it is actually defined.

Analyzing Profiles Using the N|Solid Console

The N|Solid Console allows you to generate and analyze profiles in one action. This is particularly useful if you need to take a series of profiles, as it avoids the overhead of switching between environments. The console also saves a profile history, simplifying navigation between profiles when troubleshooting.

1. Launch the console and select the process of interest.
2. On the process details page, click the New CPU Profile button.
3. Select your profile window (5 to 60 seconds), the thread you want to profile (0 is the main thread, the rest of them are worker threads) and desired visualization style and run profile.
4. N|Solid Console will pop up a green notification bar across the top of the screen when the profile is complete. Click it to view your new CPU profile.

For convenience, profiles are saved in your console session so you can easily flip between profiles. If your application ends before the profile is complete, N|Solid will try to save the profile data generated so far. If it is able to do so, your profile will be viewable under the Assets tab of the process detail page.

Visualizations Explained

Visualizations

All of the visualizations available for profile data show the unique set of stack traces captured during the profile, with the "area" of the stack indicating the proportional time spent in a function compared to that of its parent.

The Flame Graph visualization shows the time along the x-axis. The y-axis is used to show the function calls that make up a particular stack trace.

The Sunburst visualization is like the Flame Graph, where the x-axis is curved into a circle. Stack traces grow from the center to the outer parts of the graph.

The Treemap visualization shows time by area. The larger the rectangle, the more time a function took. The stack traces grow from the outer boxes into the inner boxes.

You can use the visualizations to find long-running functions. These can be identified by:

• wide rectangles in the Flame Graph
• long radial sections in the Sunburst
• large areas in the Treemap

The stack trace height - how many functions deep a particular stack trace went - does not necessarily indicate a time issue. Focus on interpreting the time values as discussed above.

As you hover over the visualization itself, the function represented by the visualization will be shown. Clicking will show a stack trace to the right of the visualization.

When analyzing CPU profiles with the N|Solid Console visualizations, pay careful attention to stack trace entries that take up the most area. This is displayed as a factor of width in the Flame Graph, circumference in the Sunburst, and area in the Treemap.

Code Highlighting

Code highlighting shows the origin of each function. This makes it easier to determine the source of performance issues by selectively highlighting only one of the following categories of functions at a time:

• None: don't highlight anything.
• User Code: your application code. Selected by default.
• Core Code: any module compiled into the Node.js/N|Solid core, e.g., fs, http, net.
• 3rd Party Code: any module installed into node_modules/, including those authored by you and/or the organization you belong to.

Search

The search field allows you to search for nodes with matching function names and paths across the entire profile. It is case-sensitive, and only the first 500 matching results will be displayed.

Analyzing Profiles Using N|Solid and the Chrome Profiler

You can also view the generated CPU profile in Chrome Dev Tools:

1. Click the Download Profile button on the left side of the screen
2. Open the Chrome web browser
3. Open Chrome Dev Tools from the overflow menu (the three dots in the right-hand side of the browser), hover More tools option, and click on Developer tools (or press ctrl + shift + i, cmd + alt +i on macOS)
4. You will see the Performance tab come up. Click the Load profile button (up arrow logo) that appears in the left-hand pane under this tab.
5. Find the .cpuprofile file that was downloaded, and click the Open button
6. A CPU profile chart will appear in the Performance tab
7. Below it you will be able to see the Chart, Heavy (Bottom Up), and Tree (Top Down) views by selecting the respective entry in the dropdown menu located immediately above the CPU profile chart.

In addition to function execution times, the CPU profile chart also shows what functions were on the stack at the time the process was sampled.

Profiling Using the Command Line Interface (CLI)

N|Solid's Command Line Interface (CLI) is a great way to quickly pull profiles from remote processes for local examination. You can learn more about the profile command in the N|Solid Command Line Interface (CLI) reference.

Profiling Using the N|Solid Node API

The N|Solid Node API provides an efficient way to trigger a CPU Profile programmatically from within your application and have the resulting profile saved to your N|Solid console. The profile duration can be up to 600000 milliseconds, which is the default if none is specified.

const nsolid = require('nsolid')
nsolid.profile(durationMilliseconds, err => {
  if (err) {
    // The profile could not be started!
  }
})

To complete your profile before the duration expires, call profileEnd:

nsolid.profileEnd(err => {
  if (err) {
    // Error stopping profile (was one running?)
  }
})

Sourcemap Support

Utilizing typescript and/or transpilers with N|Solid makes interpreting CPU profiles difficult unless the user is deeply familiar with the code. The integration of Source Maps provides a translation layer that provides a reference between the compiled source code and source code.

When compiling code, a Source Map is currently being generated. The integration of Source Maps provides a translation layer that provides a reference between the compiled source code and source code.

In an effort to address concrete customer pain-points, NodeSource has introduced SourceMap Support to the N|Solid CPU profiler. This feature continues to evolve with the specific requirements of our customers. NodeSource therefore envisages a phased rollout.

Currently the feature can be accessed as follows:

1. The NSolid Console backend assumes that the sourcemap is right next to the file, i.e. foo.js.map, which it subsequently reads and sends that to the client.
2. When opening the Flamegraph view, ensure the Sourcemap-Box is checked.
3. This makes the client traverse the CPU profile and request a sourcemap for all function locations from the Console backend
4. The Client then uses the updated CPU profile to generate a flamegraph.
5. If the Sourcemap box is unchecked, the original CPU profile is used again.
6. Stored asset of the Profile includes a ourcemap setting and applies it when the asset is loaded again.

Flamegraph Export

Users can now export NSolid Falamegraphs at the click of a button when viewing a CPU profile. In addition to downloading the CPU profile to visualize it in external tools such as Chrome Dev Tools, users can export the Flamegraph visualization to an SVG format in a few easy steps.

Step 1: Once a CPU profile has been taken, view it in the NSolid Console’s the flame graph visualization.

Step2: In the view, direct your attention to your browser’s search bar. Above on the right hand side is a ‘download’ button.

Filtering and Tagging

Tags

You can use the NSOLID_TAGS environment variable to add tags that will be applied to a process. This can aid in identification and filtering, particularly when creating Saved Views in N|Solid Console. Tags can also be retrieved using the info command in the N|Solid CLI.

Setting Tags

$ NSOLID_APPNAME="leaderboard" NSOLID_TAGS="DC, us-east-1" nsolid app

Retrieving Tags

$ nsolid-cli info --q app=leaderboard

Example JSON Response

{
  "id": "57234c38b3f08dd8a3e9a567c82887a552c27b01",
  "app": "leaderboard",
  "tags": [ "DC", "us-east-1" ],
  "pid": 2440,
  ...
}

Tag Constraints

• Tags can contain any whitespace characters
• Tags are trimmed, so only whitespace in the middle of a tag is kept intact
• Tags cannot contain commas and commas cannot be escaped
• Tags are transmitted exactly the way they are entered and therefore are case sensitive, i.e. world is different than wOrld
• The byte length of a tag must be between 2 and 140
• There is no minimum number of tags per process
• There is a maximum of 20 tags per process allowed
• Tags cannot be changed at runtime

Filtering

As many applications are comprised of tens to thousands of processes, it is often necessary to be able to filter a subset based on some criteria. The N|Solid Console offers a filtering mechanism with auto-complete to help you navigate your complex applications.

Filters

Typing just the filter name will list all of the items for that filter. Subsequent characters are then used to find partial matches within that filter.

Here are the currently supported filters in the N|Solid Console:

You may enter multiple filters and search terms in the filter box. Multiple terms will be logically AND'ed together when they are applied to the set of processes being filtered.

Heap Snapshots

When faced with a memory leak or performance issue, taking heap snapshots is a great way to help identify the underlying problem. N|Solid provides several ways to capture snapshots: the N|Solid Console, the N|Solid CLI, and the N|Solid Node API.

Taking Heap Snapshots with the N|Solid Console

Using the N|Solid Console, take a series of snapshots and quickly switch between them to better identify problematic areas:

1. Launch the console and locate the Processes list on the right.
2. Click the process ID of the process of interest, and click New Heap Snapshot from the Process Detail view.
3. Select the thread ID of the process of interest, and click Get Heap Snapshot button.
4. Once the snapshot completes, a green alert will pop up on the page. Click View Snapshot to see the results, and navigate through the various objects indexed during the snapshot process. To navigate a large number of indexed objects, you can use the Search Constructors search box to narrow the results.
5. Snapshots taken in the N|Solid Console can be downloaded to your local drive for exploration via a 3rd party tool (i.e. Chrome Developer Tools). If you wish to download the snapshot for analysis through other tools, simply click Download on the right.
6. If you want a particular snapshot to be saved and prevented from being deleted automatically, click Save to mark it for retention.

Note: You can trigger heap snapshots automatically by creating a customized saved view on a memory-related metric (such as Heap Used > 128MB) that triggers a heap snapshot action when when processes exceed the metric threshold. This is particularly useful for generating heap snapshots on applications that have rare or intermittent memory leaks.

Analyzing Heap Snapshots

Heap Snapshots capture a record of all live JavaScript objects in your Node application available at the time the snapshot is captured. The objects are grouped by their constructor name, with aggregated counts of the total number of objects, their shallow size, and their retained size as columns in the Constructors view.

Shallow Size indicates the amount of space the object uses for its own properties. For instance, an object with no properties will have a very small shallow size, where an object with a number of properties will have a larger shallow size.

Retained Size indicates the amount of space the object is referencing, and keeping from being garbage collected.

From the Constructors view of the snapshot, you can sort by constructor name, number of objects, shallow size, and retained size. The list can also be filtered by constructor name.

You can use the number of objects value to look for the objects that are leaking, and you can use the retained size value to look for objects referencing those leaking objects.

In cases where you are leaking objects - and thus consuming more memory than you should be - sorting by number of objects and by retained size can provide a good indication of where to look for the cause of the leak. A large number of objects of a particular type may be an indication that these objects are still being referenced by other objects, when they could be garbage collected. A large retained size may be an indication that these objects are referencing more objects than they should be, keeping them from being garbage collected.

Comparing Snapshots

The heap snapshot comparison feature makes debugging memory leaks a snap. To begin, you'll need two heap snapshots: a baseline snapshot, taken before the leak is evident, as well as a comparison snapshot, taken when the process or application has exhibited the leaking behavior.

1. Navigate to the heap snapshots list, by way of the Assets tab.
2. Hover over a heap snaphot to use as the baseline for the comparison, then click on the snapshot comparison button:
3. Filter through the list of comparison snapshots, selecting one taken from the same application after the leak has ballooned the total heap size:
4. Both snapshots will be presented side-by-side:

In the area above the tabular data section, summary information for each snapshot is presented. Hovering over a particular row will display calculated differences between the two snapshots in the middle header area (the count, shallow size and retained size deltas.) To view the total snapshot-wide count and shallow size deltas, simply move the mouse pointer so that no rows are hovered.

Detailed differences between the two snapshots are shown in the section below the headers. Cells on the left-hand snapshot will be colored, reflecting the percentage difference within the row.

The redder the cell, the greater the value has increased over the corresponding value in the other snapshot. Greener cells indicate the reverse. When the percentage difference is below 5% of the other snapshot's value, no arrow will be displayed. (NB: this percentage difference is different from the percentage displayed inside the cell.) A double arrow is displayed when the percentage difference is above 20% of the other snapshot's value.

5. Sort on Shallow Size, in descending order, to find the constructor responsible for the largest heap allocation. The constructor of interest is likely to be near the top of this list.

Taking Heap Snapshots from the Command Line Interface (CLI)

N|Solid's Command Line Interface (CLI) is a great way to quickly take heap snapshots from remote processes for local examination. You can learn more about the snapshot command in the N|Solid Command Line Interface (CLI) reference.

Heap Snapshots Using the N|Solid Node API

Use the N|Solid Node API to programmatically create a heap snapshot from within your application. The resulting snapshot(s) can be saved to your N|Solid Console. Heap Snapshots can be taken asynchronously and synchronously.

The following code snippet takes a heap snapshot asynchronously:

const nsolid = require('nsolid')
nsolid.snapshot(err => {
  if (err) {
    // The snapshot could not be created!
  }
})

The following code snippet takes a heap snapshot synchronously:

const nsolid = require('nsolid')
try {
  nsolid.snapshot()
} catch (err) {
  // The snapshot could not be created!
}

Best Practices

Using constructors will show class names in heap snapshots. Since the heap snapshot groups objects by constructor name, if at all possible you should use named constructors for objects you would like to track, as opposed to using literal objects. Literal objects will be aggregated under the Object constructor name.

For instance, the following code snippet creates objects which will be categorized under the Object constructor:

trackableObject = {x: "some value", y: "another value"}

To be able to track trackableObject in a snapshot, make it an instance of a specifically named class:

trackableObject = new TrackableObject()
trackableObject.x = "some value"
trackableObject.y = "another value"
// ...
function TrackableObject() {}

You'll then see these objects grouped by themselves in the heap snapshot.

For more of an "inline" feel, you can enhance your constructor to take initialization values:

trackableObject = new TrackableObject({x: "some value", y: "another value"})
// ...
function TrackableObject(initialValue) {
  this.x = initialValue.x
  this.y = initialValue.y

With EcmaScript 6 parameter destructuring, you can make it a little shorter:

trackableObject = new TrackableObject({x: "some value", y: "another value"})
// ...
function TrackableObject({x, y}) {
  this.x = x
  this.y = y
}

You can also make the constructor open-ended regarding the properties:

trackableObject = new TrackableObject({x: "some value", y: "another value"})
// ...
function TrackableObject(initialValue) {
  for (var key in initialValue) {
    this[key] = initialValue[key]
  }
}

Saved Views

A Saved View is a graph defined by a search query. Saved views allow you to view processes that fulfill certain parameters and to instruct N|Solid to take certain actions (such as taking heap snapshots, CPU profiles, or sending webhooks) when one or more processes cross over performance thresholds or match query parameters that you set.

Performance problems often occur in production code non-deterministically, and often the only diagnostics available are an error log or possibly a stack trace. With N|Solid's saved views, you can automatically capture critical information when problems arise in production.

Saved View Presets

Several preset saved views are included to provide out-of-the-box insight into the running state of your Node.js application:

Memory Clustering. This preset compares Heap Used and Resident Set Size, and helps capture processes' total memory space.

Garbage Collection Clustering. In the GC Clustering preset view, GC Count and GC Duration 99th Percentile are directly compared, providing insight into how garbage collection duration affects processes' memory usage.

Garbage Collection Anomalies. By comparing Garbage Collections Count and Process Uptime, this saved view provides insights into GC Count outliers.

Active Resource Usage. With this preset, the number of Active Handles and the Resident Set Size are compared, providing insight into longer term resource usage.

Garbage Collection Efficiency. This preset compares Major Garbage Collections Count and GC Median Duration to provide insights into garbage collection efficiency.

Creating a Saved View with N|Solid Console

1. Click Processes in the Nav Bar to get to the Processes view. The default view shows all processes connected to N|Solid Console.

2. To create a saved view, enter some query parameters and values into the filter search bar. Clicking on the bar will display a partial list of query terms. Type to begin searching for the parameter you wish to use. Click on one of the terms to select an operator and value.

3. Click Set Filter to save this search term to your active filter. Repeat this process until your filter is set up as desired with all of your required search terms.

4. When the filter has been named and saved, it will become available in the View dropdown selector.

Actions

Actions are a way to trigger some "actions" (as the features name describe) when a saved view filter is attended.

Configuring actions

1. Navigate to the Processes view and select a saved view from the dropdown menu. If you have never created a saved view before, create a saved view as described in the session below. You might see the default All Processes view Actions and notifications but the Actions can't be added to the All Processes view, so create a new saved view.

2. Click on the gear icon to the right of the saved view name. 

3. Select one or more actions or notification integrations, if desired. These will run on each process that matches the saved view filter.

The saved views can be Workers Processes exclusive, if the Worker threads option is enabled, this view will be triggered only on Apps with active worker threads.

Note: Be careful about setting a very high value on your saved view's filter expression, such as 99% Heap, and then selecting an action that uses the same resource, such as Take Heap Snapshot. Taking intensive action at a very high ceiling is likely to negatively impact your application.

Heap snapshot

This one is the easiest Action to set up. Just select the Heap Snapshot option in the Actions drop-down menu. Learn more about Heap snapshot here.

Tracing

After selecting the Trace or CPU Profiling option in the Actions drop-down menu, a modal will open, there you can inform how much time you want the trace runs when the saved view has been triggered. Learn more about tracing here.

This action will activate and deactivate the trace option for a few seconds on the Process configuration

CPU Profile

After selecting the CPU Profiling option in the Actions drop-down menu, a modal will open, there you can inform how much time you want the trace runs when the saved view has been triggered. Learn more about CPU Profiling here.

After that, just click on the save action.

Action results

CPU Profiling and Heap Snapshot

After the saved view has been trigged, you could check the results in the Application assets summary or the assets tab.

Tracing

After the saved view has been triggered, you could check the results filtering by date on Distributed Tracing page.

Integrations (Teams, Slack, Webhooks, [...])

You can enable Saved Views, Vulnerabilities and global event notifications as Microsoft Teams, Slack messages, invoked webhooks, or emails.

Slack Integration

To configure notifications to use Slack incoming messages, you will need to create a Slack Incoming Webhook. From the Slack Incoming Webhooks New Configuration page, select a channel the messages should be posted to, and then click the Add Incoming Webhooks Integration button. This will take you to a page where the Webhook URL is displayed, and you can further configure the message to be sent.

Copy the webhook URL displayed on that page. In the N|Solid Console, click Settings and choose Integrations from the left side menu. Paste the Slack webhook URL into the form and name the new integration. Once you have saved this integration, you will be able to attach it to global events, vulnerabilities, and saved views as a notification.

Microsoft Teams Integrations

Similar to how teams configure NSolid’s Slack integration, organizations using Microsoft Teams can now configure notifications to use Microsoft Teams incoming messages feature.

To set up Microsoft Teams in NSolid, users need to create a Microsoft Teams Incoming Webhook (see MS Teams docs here) and use it to configure the Microsoft Teams integration in Settings>Integrations.

On the New Teams Webhook page, users can enter the channel name they wish to post messages to in the Webhook Name field, and paste the webhook URL into the Webhook Url and save the webhook. Once saved the integration is active and users can select Microsoft Teams as the notification destination of their set alerts.

Now the Microsoft Teams integration is available.

Once configured users can use the Microsoft Teams integration in both Threshold Event Notifications based on Saved Views and Global Event Notifications .

To have NSolid send a Global Notification to your configured Microsoft Teams channel visit Settings>GlobalNotifications and select the configured Microsoft Teams Notification from the drop down.

Once selected users can observe the message format and content:

Users can also use the Microsoft Teams integration when their processes enter a saved view:

PagerDuty Integrations

Organizations using PagerDuty can now configure notifications to use PagerDuty incidents feature.

To have NSolid send a Global Notification to your configured PagerDuty incidents visit Settings>GlobalNotifications and select the configured PagerDuty Notification from the drop down.

On the New PagerDuty integration page, users can add a nickname for the identity of the integration and the provided

Now the PagerDuty integration is available.

Once configured users can use the PagerDuty integration in both Threshold Event Notifications based on Saved Views and Global Event Notifications.

To have NSolid send a Global Notification to your configured PagerDuty integration Settings>GlobalNotifications and select the configured PagerDuty Notification from the drop down.

Examples

Below are examples of typical saved view and vulnerability notifications sent as incoming Slack messages.

Process Event Loop Blocked

Saved View Entered

Vulnerability Detected

In case you need support to set up the PagerDuty integration, please visit our support portal: https://support.nodesource.com/

Webhook Integration

You can also have notifications sent as HTTP POST messages to a webserver you manage. As the messages are sent to your server, you can run your own customized code to handle the notification.

Notification data is sent in the HTTP request as a JSON object in the body of the request. The HTTP request will be sent with the Content-Type header set to application/json.

In the N|Solid Console, click Settings and choose Integrations from the left side menu. Paste your webhook URL into the form and name the new integration. Once you have saved this integration, you will be able to attach it to global events, vulnerabilities, and saved views as a notification.

Events

The data the webhook receives has an event property. The value of this property will change depending on the event that prompts the notification. The following events are supported.

Common Data

All notifications will include the following properties in the notification object and additional properties that may be specific to individual events.

The property named agent will be an object with the following properties:

In addition to the data specified above, some events may contain additional information.

Agent Exit

Agent exit events will include the exit code of the process. If there was a stack trace associated with an abnormal exit, that stack trace will also be included. Additionally, if you catch an exception with an uncaughtException handler and exit synchronously, as Node.js documentation recommends, the exception will also be included.

const nsolid = require('nsolid');
process.on('uncaughtException', err => {
  console.error(err);
  proces.exit(1);
});

If you must exit asynchronously from an uncaughtException handler it is still possible to report the exception by passing it to nsolid.saveFatalError() prior to shutting down.

const nsolid = require('nsolid');
process.on('uncaughtException', err => {
  nsolid.saveFatalError(err);
  shutdownApp(() => {
    proces.exit(1);
  });
});

Event Loop Blocked Data

The value of the event property in the notification object will be nsolid-process-blocked.

The notification object will include an additional property named stacktrace, whose value is a string version of the stack trace being executed when the event loop was blocked.

Vulnerability Event Data

The value of the event property in the notification object will be nsolid-vulnerability-detected.

The notification object will include an additional property named vulnerability, whose value is a list of vulnerabilities for this process. The new vulernability whose discovery triggered this event will have a new property with the value true.

Email Notifications

In N|Solid Console, click Settings and choose Global Notifications from the left side menu. Select a global event, such as vulnerabilities found, a new CPU profile, or a new HeapSnapshot, etc, that is meant to be notified via email. Click on New Notification dropdown and select Email. Enter one or more email addresses; once you have saved this notification, the previously provided emails will receive the respective notifications.

Security Vulnerabilities

The N|Solid Console can be configured to perform periodic verification of all packages loaded by all N|Solid processes. All loaded packages are verified against a list of known vulnerabilities.

When new vulnerabilities are found, information about each vulnerability will be reported in the Console. Notification options can also be configured to streamline reporting.

Detecting Security Vulnerabilities

The Security View

If there are any vulnerabilities found in your applications, the Security link in the Nav Bar will be annotated with a numbered badge indicating the number of vulnerabilities found across all of your applications.

Clicking on Security in the Nav Bar will display the Security view. This view displays a list of all the vulnerabilities found across all applications.

All of the vulnerabilities found in all applications will be listed on the left. The numbered badge in that list indicates the number of applications which are affected by the vulnerability. Clicking on a vulnerability in the list will display more details about that vulnerability. The Affected Processes tab will display information about the processes which are affected.

The Affected Processes subview contains an entry for every application affected by the vulnerability. By clicking on the disclosure triangle next to the application name, a list of the module dependencies for the vulnerable package is displayed.

You can use the Hide/Show toggle on the right to have the vulnerability ignored when determining the number of vulnerabilities across all the applications.

Vulnerable Processes in the Processes View

From the Processes view, you can sort the Processes list on the right by Vulnerabilities, which will sort the currently filtered processes into two sets - Vulnerable and Secure.

The vulnerable processes will be shown with a bright red dot in the scatterplot, and secure processes will be shown with a light colored dot in the scatterplot. Clicking on the process in the Processes list will display the Process Details view, which contains additional vulnerability information.

Vulnerability Information in the Process Details View

The Process Details view contains a Modules subview, which contains information about the vulnerabilities found in the process. Clicking on the vulnerability title will display the Security view for the vulnerable module.

Configuring Notifications for Security Vulnerabilities

You can configure the N|Solid Console to notify you when new vulnerabilities are found in your applications. To configure these notifications, click the Global Notifications link on the left side of the Settings view.

Scroll down to the section Vulnerability Notifications. Here you can add Integrations to be invoked when a new vulnerability is found.

NodeSource Certified Modules v2 (NCM 2)

Table of Contents

• Using NCM2 with the Console
• N|Solid strict mode

Using NCM 2 with the Console.

The Console has a dedicated section for NodeSource Certified Modules v2.

1. Open the Process Detail View of the process.

2. Click the modules tab and look at the NCM column.

3. To see details in the NCM 2 report, just click the magnifying glass icon of the package that you want to review (see ncm details).

Learn more about NCM 2 reports.

N|Solid strict mode

The N|Solid strict mode can be used with the prompt nsolid-strict instead of the well known nsolid, the key difference between strict and regular mode is that the strict mode will stop any application with encountered vulnerabilities identified by the NodeSource Certified Modules v2.

Example running vulnerable apps and secure apps respectively:

$ nsolid-strict vulnerable-node-app.js
nsolid STRICT MODE verifying...
Unsecure server running normally :)
nsolid STRICT MODE access denied due to policy violation:
{
  "package": "unsecure-pkg",
  "version": "1.19.4",
  "group": "risk",
  "name": "has-install-scripts",
  "pass": false,
  "severity": "CRITICAL",
  "title": "This package version has install scripts: postinstall."
}

$ nsolid vulnerable-node-app.js
Unsecure server running normally :)

$ nsolid-strict secure-node-app.js
nsolid STRICT MODE verifying...
Secure server running normally :)

Import/Export Settings

Users can now export configuration settings using either NSolid Console, the result is a JSON format file which can be used to import settings across different setups.

In the N|Solid Console, go to Settings, in the left tab list you’ll see the new Import/ Export Settings section.

PLEASE NOTE: If you cannot see this new section, please go to the application permissions on accounts.nodesource.com and verify that the settings -> import / export settings is checked for your role.

Export

To Export settings, choose which ones you want to export, you are able to export Saved Views, Integrations, and Global Notifications individually or all of them at the same time.

The console provides you with a file to download with the extension ".nsconfig", which contains the configuration you selected to export in json format.

Example JSON Result

{
  "_metadata": {
    "_timestamp": "",
  },
  "integrations": {},
  "notifications": {},
  "savedViews": {}
}

Also, inside the “.nsolid-console” folder there’s a new folder structure where you are going to have a backup of every export file generated in case you need it in the future.

Import

Clicking import will open a modal, you can select what settings you want to import and whether you want to add to the current settings or if you want to clean up and only use imported ones. In the case of integrations, it can not be cleaned, because these can be used by notifications or saved views.

You should choose the backup file previously downloaded, remember that it must have the extension .nsconfig. If you lost the export file, remember you can always check inside the “.nsolid-console” folder for a backup. Once the file is successfully imported, you will see the new configuration applied and ready to run.

Event profiler

N|Solid event profiler provides a rich set of events, covering security events, lifecycle events, system events, performance events, user defined machine learning events and assets events.

PLEASE NOTE: If you cannot see this new section, please go to the application permissions on accounts.nodesource.com and verify that the view events historic is checked for your role.

Table of Contents N|Solid event profiler

• Using event-profiler with the console
• event-profiler with nsolid-cli
• Events in detail

Using event profiler with the console

The console has a dedicated section for event profiling.

• Click on Events tab on the navigation bar

  

• The event profiler section has a summary section:

  

Manage events records with event profiler UI.

The events report UI scales no matter how many events you have records of.

• Filtering events with event profiler.

  Events can be filtered by severity, type, application name, hostname, and agent id; to do this, simply click on the filter box:

  

• Sorting events with event profiler.

  Events can be sorted by time, application name, hostname, agent id, and event name itself, click on the property you want to sort:

  

• Limit events with event profiler by dates.

  Events report can be limited by dates, click on the dates picker and select your desired date range:

  

N|Solid events

HTTP Tracing

NodeSource has introduced Tracing support, a facility to gather information throughout the lifecycle of an HTTP/DNS/Other request. The collected information can be used for debugging latency issues, service monitoring and more. This is a valuable addition to users for those who are interested in debugging a request latency. Tracing traces user requests through a Node application, collecting data that can help find the cause of latency issues, errors, and other problems.

Enabling Tracing via N|Solid

To enable the Tracing using N|Solid, set the env var NSOLID_TRACING_ENABLED=1. By enabling this feature, you can troubleshoot HTTP, DNS and other network request problems that you might encounter using the N|Solid Console.

Enabling Tracing via N|S Console UI

1. Enter in your process
2. Click in Process configuration
3. Activate the toogle for Tracing

Using Distributed Tracing

Distributed tracing in the N|Solid Console is basically an extension of the HTTP tracing in N|Solid. This documentation will use the following examples to cover how it works on the Console side under the following conditions:

1. An application handling the authentication process:

  // console.js
  const http = require('http')
  const PORT = process.env.PORT || 3000
  http.createServer((req, res) => {
    if (req.url === '/auth') {
      console.log('AUTH handler')
      const authURL = 'http://localhost:4000'
      const gReq = http.request(authURL)
      gReq.end()
      gReq.on('close', () => res.writeHead(200).end('Google made it'))
    } else if (req.url === '/auth-2fa') {
      const authURL = 'http://localhost:4000/2fa'
      const twoFactor = http.request(authURL)
      twoFactor.end()
      twoFactor.on('close', () => res.writeHead(200).end('Auth made it with sms'))
    } else {
      res.end('ok')
    }
  }).listen(PORT)

2. Another application doing the two-factor authentication process (google-auth):

  // google-auth.js
  const http = require('http')
  const PORT = process.env.PORT || 4000
  http.createServer((req, res) => {
    if (req.url === '/2fa') {
      const smsServiceURL = 'http://localhost:5000'
      const smsService = http.request(smsServiceURL)
      res.writeHead(200).end('Auth made it')
      smsService.end()
    } else {
      console.log('Auth root handler')
      res.writeHead(200).end('ok')
    }
  }).listen(PORT)

3. The authentication requires a two-factor authentication
4. The two-factor authentication service uses an SMS service to send a message (twilio):

  // twilio.js
  const http = require('http')
  const PORT = process.env.PORT || 5000
  http.createServer((req, res) => {
    res.writeHead(200).end('ok')
  }).listen(PORT)

5. The N|Solid Console receives traces from all services involved in the distributed system.

Distributed Tracing UI

In the N|Solid Console, go to the the applications dashboard and click on the TRACING button of the application to see the traces.

The view will be shown as below:

Tracing is consists of three key components below:

• Timeline Graph: a timeline graph of Tracing data showing the density of the number of tracing spans.
• Filter: a filter input area to filter the results by attributes of the span.
• Spans(Results): a span is the building block of a trace and is a named, timed operation that represents a piece of the workflow in the distributed system. multiple spans are pieced together to create a trace.

Timeline Graph

A timeline graph displays the density of the number of tracing spans. Below is the description of the color of a slot on the timeline graph:

Assume that a simple request was made to the “console” service to monitor traces:

As a result, the Console displays the whole “span” information.

Span

A span is the building block of a trace and is a named, timed operation that represents a piece of the workflow in the distributed system. Multiple spans are pieced together to create a trace.

Traces are often viewed as a tree of spans that reflects the time that each span started and completed. It also shows you the relationship between spans. A trace starts with a root span where the request starts. This root span can have one or more child spans, and each one of those child spans can have child spans.

Inspecting Span Details

To inspect the span details of a span, click on the title Service & Operation:

Below are the attributes of the span:

Let's authenticate using the console service, which is going to perform a request to the google-auth-service to authenticate the user.

The graph below represents a “path”, from the console service to the authentication service, N|Solid will monitor the HTTP traces in this distributed system.

As such, graph is showing the whole “path”, starting from the console and finishing with the sms service. This is how distributed tracing works using N|Solid managed systems.

The collected information can be used for debugging latency issues, service monitoring, and more. This is a valuable addition to users for those who are interested in debugging a request latency. Tracing traces of user requests through multiple Node applications, and collecting data that can help find the cause of latency issues, errors, and other problems in a distributed system.

Changing Update Settings

You can configure the update settings for distributed tracing on the top of the Distributed Tracing view:

Basically, you can turn on or off Real Time Update and change the update interval. The default update interval is 10 seconds. Available options are 10, 15, 30 or 60 seconds.

Changing Time Range

To change the time range of tracing, click the calendar icon below the timeline graph:

It will show a calendar for selecting a time range. The timeline graph range is updated every 1 minute, with an interval of 1 minute for the date range to move.

Filtering Results

Filtering the results by attributes of the span is made easy for users. Just tab the Filter input area and select attribute(s) you want to filter as below:

OpenTelemetry Support

N|Solid has added support for some OpenTelemetry features:

• It provides an implementation of the OpenTelemetry TraceAPI which allows the user to instrument their own code using the de-facto standard API.
• It also supports the use lots of instrumentation modules available in the OpenTelemetry echosystem.
• It support exporting traces using the OpenTelemetry Protocol(OTLP) over HTTP.

OpenTelemetry JS Tracing API

Using the OpenTelemetry JS API @opentelemetry/api to instrument your own code is very easy. N|Solid provides a nsolid.otel.register() API which allows to use the N|Solid implementation of the OpenTelemetry TraceAPI. See a very basic example in the following code. Notice that for the traces to be generated the enviroment variable NSOLID_TRACING_ENABLED should be set.

// Run this code with `NSOLID_TRACING_ENABLED=1` so traces are generated.
const nsolid = require('nsolid');
const api = require('@opentelemetry/api');

if (!nsolid.otel.register(api)) {
  throw new Error('Error registering api');
}

const tracer = api.trace.getTracer('Test tracer');
const span = tracer.startSpan('initial', { attributes: { a: 1, b: 2 }});
span.updateName('my name');
span.setAttributes({c: 3, d: 4 });
span.setAttribute('e', 5);
span.addEvent('my_event 1', Date.now());
span.addEvent('my_event 2', { attr1: 'val1', attr2: 'val2'}, Date.now());
span.end();

Integration with OpenTelemetry-compatible instrumentation modules

N|Solid also provides a nsolid.otel.registerInstrumentations() API to register instrumentation modules that use the OpenTelemetry TraceAPI that are available in the OpenTelemetry echosystem. The following code shows an example using the @opentelemetry/instrumentation-fs module:

// Run this code with `NSOLID_TRACING_ENABLED=1` so traces are generated.
const nsolid = require('nsolid');
const api = require('@opentelemetry/api');
const os = require('os');
const path = require('path');

const { FsInstrumentation } = require('@opentelemetry/instrumentation-fs');

nsolid.start({
  tracingEnabled: true
});

if (!nsolid.otel.register(api)) {
  throw new Error('Error registering api');
}

nsolid.otel.registerInstrumentations([
  new FsInstrumentation({
  })
]);

const fs = require('fs');
fs.mkdtemp(path.join(os.tmpdir(), 'foo-'), (err, directory) => {
  if (err) throw err;
  console.log(directory);
});

Exporting Traces via OTLP over HTTP

It's possible now to export traces with N|Solid to endpoints supporting the OpenTelemetry Protocol(OTLP) over HTTP. On top of that we make very easy to send traces to specific vendors endpoints such as Datadog, DynaTrace and NewRelic. And not only that, for these vendors we're also able to export the metrics N|Solid generates, so this info can also be displayed in their solutions with no need to use their agents which have the performance issues explained in this article.

To configure the OTLP endpoint there are two configuration options we need to set either via NSOLID_OTLP and NSOLID_OTLP_CONFIG the environment variables or the other ways N|Solid provides to set them.

NSOLID_OTLP defines the type of endpoint we're exporting the traces to. Allowed values at the moment are:

• datadog
• dynatrace
• newrelic
• otlp

NSOLID_OTLP_CONFIG defines the configuration for the type of endpoint selected in NSOLID_OTLP. This configuration is a string containing a JS object serialized using JSON. The format of this JS object differs depending on the type of endpoint.

{ zone: 'us' | 'eu',  key: 'your_datadog_key', url: 'otlp_endpoint_url' }

{ site: 'you_dynatrace_size',  token: 'your_dynatrace' }

{ zone: 'us' | 'eu',  key: 'your_datadog_key' }

{ url: 'otlp_endpoint_url' }

Here is an example of how to configure N|Solid to export data to Dynatrace. Notice you need N|Solid to be licensed.

$ NSOLID_OTLP=dynatrace NSOLID_OTLP_CONFIG='{"site":"my_site","token":"my_token"}' NSOLID_LICENSE_TOKEN=my_nsolid_license nsolid my_process.js

CPU Anomaly Detection

In order to correctly identify an anomaly it is important that the detection method be accurate. CPU is no longer enough of a measurement to scale applications. Other factors such as garbage collection, crypto, and other tasks placed in libuv's thread pool can increase the CPU usage in a way that is not indicative of the application's overall health. Even applications that don't use Worker threads are susceptible to this issue.

In addition, there is no cross-platform way of measuring the CPU usage per thread, which doesn't mean that CPU is useless. CPU and event loop utilization (or ELU) is crucial to see if an application is reaching hardware limitations. But not being able to gather metrics on a per-thread basis drastically limits our ability to determine when the application is reaching its threshold.

Note: ELU(Event loop utilization) is the ratio of time the event loop is not idling in the event provider to the total time the event loop is running, and is equal to the loop processing time divided by the loop duration.

With that being said, N|Solid Console provides ELU-based Scatterplot, which utilizes the most reliable metric to use as a baseline for comparison.

The ELU Scatterplot

The Scatterplot is an animated graph that provides an overview of your applications' performance across all or a subset of connected processes, when an specific process has at least one active worker thread, the process will be highlighted.

Using ELU as the axis to compare metrics across multiple processes is a reliable way to identify anomalies without false positives. With this information anomalous processes can be automated to take CPU profiles, heap snapshots and etc.

In the N|Solid Console, go to the the applications dashboard and click CPU ANOMALY DETECTION.

The blue dots are the raw data. Red line is the regression line (estimated average), yellow and green are the error from the regression.

The default y-axis value is delay, which equals to (providerDelay + processingDelay) / 1e6 in microseconds.

• The blue dots: The blue dots are the raw data from all the applications. All the application raw data are the same color. It only highlights the points from the same application when a single point is hovered with the mouse.

• The red line: The red line is the moving average of all the raw data (blue dots). There is no application specific information to show when those points are hovered.

• The yellow and green line: The yellow and green lines are the error margin for the moving average (red dots).

Anomalies List

At the left side, there's a list of anomalies which can be filtered by agent ID. To see the details of an anomaly, click the title of an item to expand it and read the description.

Note: If you are redirected from Events tab, the corresponding anomalies will be shown.

Memory Anomaly Detection

Understanding memory management reduces the possibility of wasting your application’s resources, and the unexpected effects on performance. In many cases, there is no clear understanding as to why the memory grows endlessly, however, a check for correlation between set of metrics who intereact with each other can give insights about memory usage and response time degradation.

Memory Anomalies in the N|Solid Console provides a way to detect early cases of memory miss behavior or upcoming Out of Memory situations before it happens.

Inspecting Anomalies

In the N|Solid Console, go to the the Applications Dashboard and click MEMORY ANOMALY DETECTION.

If there are no anomalies registered, you'll get an empty list and 0 values in the charts and placeholders.

If any of the processes or worker threads under an application name have reported anomaly events, you'll get a list at the left side with a short title, the process id and if it came from them main or a worker thread. You can filter this list by processID with the search box at the top.

To see the details and load the metrics of an anomaly, click the title of an item to expand it, and extended description will appear with the timestamp and the charts at the right side will load a snapshot of the metrics one minute before and after the timestamp registered in the event.

Note: If you are redirected from Events tab, the corresponding anomalies will be shown.

Machine Learning

NodeSource has introduced Machine Learning support, this feature allows for the training of models that will later detect similar patterns in your application data and fire custom events.

Machine Learning UI

In the N|Solid Console, the Machine Learning feature can be accessed from the app summary or process detail views, each of these handle different data sets and will have a different effect in the model you train.

Training ML Models

The Machine Learning models can be trained using two kinds of data sets. The models trained in the app summary view will use the aggregated data of all the processes running inside the app. On the other hand, the models trained in the process detail view will use the process specific data.

Train a model in the app summary view

When a process/app is first connected it will take a certain amount of data in order to be successfully trained, you will find a progress loader under process configuration:

To train a model in a app summary page click on Train ML Model button.

Train a model in a process detail view

To train a model in a process detail page click on Train ML Model button.

Modal creation and training

After clicking on the Train ML Model button a modal will open, here you can create, filter and train models, this modal is the same for both pages.

To create a model click on CREATE NEW MODEL

Name and give a short description for the model, then save.

Select the created modal an click on TRAIN

When the trained model finds a data pattern similar to the one it was trained with, it will fire an event and show a banner on top of the navbar.

Click on View Event to be redirected to the events tab, here you will find the most recent machine learning event.

The events will also appear in the application status section, clicking on VIEW ANOMALIES will redirect to the events tab.

Manage the default and custom models

Machine Learning models can be administered in the settings tab where you will find a set of default models and the user trained models, here the frequency of events being fired can be modified and the custom user models can be deactivated, deleted or edited. For a full reset of the created models click on RESET MODELS.

Custom user models have an edit and delete icon, these models are found beneath the default models.

PLEASE NOTE: Only the name and description of the user created model can be edited, if you want to change the model data please retrain the model in app summary or in the process detail pages. Default models are activated by default, these can only be activated or deactivated.

Command Line Interface (CLI)

Table of Contents

• Options
• asset
• assets
• custom
• events
• events-historic
• generate-keypair
• info
• list
• metrics
• metrics-historic
• packages
• profile
• snapshot
• startup-times
• vulnerabilities
• query
• summary
• export-settings
• import-settings
• events-historic

Options

N|Solid comes with a set of predefined endpoints for interaction and introspection that can be accessed at the command-line using the N|Solid Command Line Interface, nsolid-cli.

The N|Solid CLI tool mirrors the Console's API schema. Many options are documented below. For commands not documented here, consult your Console's API Docs at http://localhost:6753/api/v3/api-docs (replace localhost:6753 with your Console server's address.)

To see a list of help information for nsolid-cli, use the -h option:

$ nsolid-cli -h

The output of the nsolid-cli commands, unless otherwise noted, is line-delimited JSON (each line is separated by a newline delimiter \n). The output shown in the examples below is expanded for readability.

--auth

If your Console is running with authentication enabled, you will need to configure an administrative access token to allow nsolid-cli to be permitted to access it. This can be set with the NSOLID_CONSOLE_AUTH_ADMIN_TOKEN environment variable or the corresponding config file setting to a secure value value. Once set, you can pass this value to nsolid-cli using the --auth argument. To disable this authentication, see user authentication.

--attach

For the specific case of importing settings from a configuration file .nsconfig, you can use this option to read the contents from the file in the filesystem.

--start and --end options

For commands that take --start and --end options, you can pass the following formats for the date/time value:

--q option

Some commands take a --q parameter followed by a query filter expression. Only results matching all query filter terms will be returned.

A filter term is a string with no spaces separating the values.

"field+operator+value"

The field may be any field from info or metrics, or vulns, vuln, or package. See the output of those commands to see what options exist for the field portion of the query.

The operators for each field type are described below.

The value can be a single value or a list of values separated by ,. The term will evaluate to true if the field and operator match any of the specified values. Values may escape any character by preceding it with the % character. In practice, the only characters that need to be escaped are %, (space), and ,.

A query filter expression is a list of filter terms, separated by space characters.

Here is an example showing the list command being passed the --q flag with a query filter expression that has two filter terms:

$ nsolid-cli list --q "vulns>=2 package=express"

String Operators

Note that multiple string values may be specified in the expression, separated by commas.

Number Operators

Package Operators

A value for packages fields is either package name or {package name}@{simple-semver-range}, where a simple semver range is one of:

Semver Ranges

Using the < and > operators will do a semver range check comparing the versions.

asset

Download an asset.

Usage

$ nsolid-cli asset --id 217040c0-02d1-4956-8648-7fb84b78c65e > my.heapsnapshot

Asset IDs are available via the assets command described below. The asset file itself will be written to stdout. The N|Solid CLI tool will automatically manage decompression if the asset is compressed.

assets

Lists the assets (CPU profiles and heap snapshots) that are currently available for download.

Usage

$ nsolid-cli assets --app my-app-name --type snapshot

Returns a JSON stream including the following properties:

Example JSON Result

{
  "time": "2017-11-29T17:08:17.364Z",
  "asset": "3011f777-b8e0-4696-ae6c-50358bce298a",
  "type": "snapshot",
  "id": "272470293ef95e530b1d9d072e6ed87e0c980173",
  "app": "my-app-name",
  "hostname": "my-computer.local",
  "tags": [
    "region:north",
    "zone:A"
  ],
  "size": 4288158,
  "compressed": true,
  "pid": 5940,
  "title": "my-app-name",
  "info": { ... },
  "metrics": { ... },
  "starred": false
}

custom

Invoke a custom command. For more information on custom commands, see Custom Commands.

Usage

$ nsolid-cli custom --id=[agent id] --name=verbose --data=off

Returns a JSON object with the following properties:

Example JSON Result

{
  "time": "2017-12-04T00:56:28.566Z",
  "id": "81535293aea1fe8c1e2f3f7518d8db3f96cf7b39",
  "app": "nsolid2",
  "hostname": "computer.local",
  "tags": [
    "localdev"
  ],
  "result": {
    "verbose": false
  }
}

events

Subscribe to the event stream, which emits a wide array of event types and metadata.

Usage

$ nsolid-cli events

There are many types of events, and more are added all the time. Some of the primary types are described below.

Example JSON Result

{"time":"2017-12-04T01:47:16.386Z","event":"license-updated","args":{"licensed":true}}
{"time":"2017-12-04T01:47:17.905Z","event":"field-range-changed","args":{"domain":"metrics","name":"loopsPerSecond","range":[0,22]}}
{"time":"2017-12-04T01:47:48.613Z","event":"agent-packages-added","agent":"21fdaca1fd8533465392697e3d305e1991808836","args":{"app":"my-app","hostname":"x1c","tags":[],"pid":27646}}
{"time":"2017-12-04T01:47:48.613Z","event":"agent-found","agent":"21fdaca1fd8533465392697e3d305e1991808836","args":{"app":"my-app","hostname":"x1c","tags":[],"pid":27646}}
{"time":"2017-12-04T01:47:53.087Z","event":"agent-exit","agent":"21fdaca1fd8533465392697e3d305e1991808836","args":{"app":"my-app","hostname":"x1c","tags":[],"pid":27646,"exitCode":0}}

events-historic

Extract the events from a range of time in the past.

Usage

$ nsolid-cli events-historic --start "-2h" --end "-1h"

generate-keypair

Returns a randomly generated keypair suitable for use in the N|Solid Console socket configuration. If your N|Solid Console instance is running on an untrusted network, it is recommended that you generate and use new keys.

Usage

$ nsolid-cli generate-keypair

Returns a JSON object with the following properties:

Example JSON Result

{
  "public": "[t&m}{EZH7=HR(IW:+Ttk:=r.Y$:CP+-Q&5L?2N!",
  "private": "4QZof={^Pman?I?mB0o!]%z/{Jlu6:mJfl[Ms@[^"
}

info

Returns objects which contain static information about processes and the hosts they are running on.

Usage

$ nsolid-cli info

Returns a JSON stream including the following properties:

Example JSON Result

{
  "id": "5dd6f7a940bfc3633cc3ffc82332640d51ce5112",
  "app": "my-app-name",
  "appVersion": "1.0.0",
  "tags": [
    "region:north",
    "zone:A"
  ],
  "pid": 14880,
  "processStart": 1512335595061,
  "nodeEnv": "dev",
  "execPath": "/usr/bin/nsolid",
  "main": "/var/my-app/app.js",
  "arch": "x64",
  "platform": "linux",
  "hostname": "my-computer.local",
  "totalMem": 8244523008,
  "versions": {
    "http_parser": "2.7.0",
    "node": "8.10.0",
    "nsolid": "3.1.0",
    "v8": "6.2.414.50",
    "uv": "1.19.1",
    "zlib": "1.2.11",
    "ares": "1.10.1-DEV",
    "modules": "57",
    "nghttp2": "1.25.0",
    "openssl": "1.0.2n",
    "icu": "60.1",
    "unicode": "10.0",
    "cldr": "32.0",
    "tz": "2017c",
    "nsolid_lib": {
      "v8_profiler": "nsolid-v5.7.0-fix1",
      "sodium": "nsolid-2.1.0",
      "cjson": "nsolid-3.0.0",
      "function_origin": "nsolid-v1.2.1",
      "nan": "v2.5.1",
      "cli": "v3.0.0",
      "agent": "v8.0.3",
      "zmq-bindings": "nsolid-2.15.4-fix1",
      "zmq": "nsolid-v4.2.0-fix4",
      "persistents_with_classid": "v1.1.1"
    }
  },
  "cpuCores": 4,
  "cpuModel": "Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz",
  "time": "2017-12-03T21:13:15.061Z"
}

list

Returns an array of all available matching N|Solid processes, along with their most recent info and metrics data. The command ls is an alias for this command.

Usage

$ nsolid-cli list --q id=5dd6

Returns newline delimited JSON objects where each row includes the following properties:

Example JSON Result

{
  "time": "2017-12-04T01:17:31.299Z",
  "info": { ... },
  "metrics": { ... }
}

metrics

Subscribes to the metrics for a set of agents.

Usage

$ nsolid-cli metrics

Consult the Metrics in Detail section for complete details on the metrics available.

Example JSON Result

{
  "time": "2017-12-04T01:23:16.163Z",
  "id": "5dd6f7a940bfc3633cc3ffc82332640d51ce5112",
  "app": "my-app-name",
  "hostname": "my-computer.local",
  "tags": [
    "region:north",
    "zone:A"
  ],
  "activeHandles": 750,
  "activeRequests": 0,
  "blockInputOpCount": 0,
  "blockOutputOpCount": 19424,
  "cpuPercent": 0.2666424917029181,
  "cpuSpeed": 2640,
  "cpuSystemPercent": 0.13332124585145905,
  "cpuUserPercent": 0.13332124585145905,
  "ctxSwitchInvoluntaryCount": 9988,
  "ctxSwitchVoluntaryCount": 1795924,
  "dns99Ptile": 0,
  "dnsCount": 0,
  "dnsMedian": 0,
  "externalMem": 734532,
  "freeMem": 251301888,
  "gcCount": 446,
  "gcCpuPercent": 0,
  "gcDurUs99Ptile": 409,
  "gcDurUsMedian": 527,
  "gcForcedCount": 0,
  "gcFullCount": 0,
  "gcMajorCount": 54,
  "heapSizeLimit": 1501560832,
  "heapTotal": 73269248,
  "heapUsed": 60332056,
  "httpClient99Ptile": 0,
  "httpClientAbortCount": 0,
  "httpClientCount": 0,
  "httpClientMedian": 0,
  "httpServer99Ptile": 0,
  "httpServerAbortCount": 0,
  "httpServerCount": 0,
  "httpServerMedian": 0,
  "ipcReceivedCount": 0,
  "ipcSentCount": 0,
  "load15m": 0.2333984375,
  "load1m": 0.2265625,
  "load5m": 0.28564453125,
  "loopAvgTasks": 0,
  "loopEstimatedLag": 0,
  "loopIdlePercent": 100,
  "loopTotalCount": 713,
  "loopsPerSecond": 0,
  "pageFaultHardCount": 0,
  "pageFaultSoftCount": 132206,
  "rss": 138301440,
  "signalCount": 0,
  "swapCount": 0,
  "systemUptime": 23006,
  "title": "node",
  "totalAvailableSize": 1437491784,
  "totalHeapSizeExecutable": 5242880,
  "totalPhysicalSize": 72644376,
  "uptime": 15000.85,
  "user": "appuser",
  "vulns": 1
}

metrics-historic

Retrieve metrics records over a historical time range. Records match the metrics command output.

Usage

$ nsolid-cli metrics-historic --start=-5m --end=-1m

packages

Returns a list of packages and modules available in the specified process.

Usage

$ nsolid-cli packages --id=[agent id]

Returns a JSON object with the following properties:

Example JSON Result

{
  "id": "a40827afbc3620e40887d6774249c321848d54f6",
  "time": "2017-12-04T01:26:12.514Z",
  "packages": [
    {
      "name": "my-app-name",
      "version": "1.0.0",
      "path": "/var/my-app",
      "main": "app.js",
      "dependencies": [
        "node_modules/debug",
        "node_modules/minimist",
        "node_modules/split"
      ],
      "dependents": [],
      "vulns": []
    },
    {
      "name": "debug",
      "version": "2.2.0",
      "path": "/var/my-app/node_modules/debug",
      "main": "./node.js",
      "dependencies": [
        "../ms"
      ],
      "dependents": [
        "../.."
      ],
      "vulns": [
        "npm:debug:20170905"
      ]
    },
    {
      "name": "minimist",
      "version": "1.2.0",
      "path": "/var/my-app/node_modules/minimist",
      "main": "index.js",
      "dependencies": [],
      "dependents": [
        "../.."
      ],
      "vulns": []
    },
    {
      "name": "ms",
      "version": "0.7.1",
      "path": "/var/my-app/node_modules/ms",
      "main": "./index",
      "dependencies": [],
      "dependents": [
        "../debug"
      ],
      "vulns": [
        "npm:ms:20170412"
      ]
    },
    {
      "name": "split",
      "version": "1.0.0",
      "path": "/var/my-app/node_modules/split",
      "dependencies": [
        "../through"
      ],
      "dependents": [
        "../.."
      ],
      "vulns": []
    },
    {
      "name": "through",
      "version": "2.3.8",
      "path": "/var/my-app/node_modules/through",
      "main": "index.js",
      "dependencies": [],
      "dependents": [
        "../split"
      ],
      "vulns": []
    }
  ],
  "vulnerabilities": [
    {
      "package": "debug",
      "title": "Regular Expression Denial of Service (ReDoS)",
      "published": "2017-09-26T03:55:05.106Z",
      "credit": [
        "Cristian-Alexandru Staicu"
      ],
      "id": "npm:debug:20170905",
      "ids": {
        "NSP": 534,
        "CWE": [
          "CWE-400"
        ],
        "CVE": [],
        "ALTERNATIVE": [
          "SNYK-JS-DEBUG-10762"
        ]
      },
      "vulnerable": "<2.6.9 || >=3.0.0 <3.1.0",
      "severity": "low",
      "description": " ... ",
      "nsolidMetaData": {
        "hidden": false
      },
      "packages": [
        "/var/my-app/node_modules/debug"
      ],
      "depsChains": [
        [
          "nsolid-dev-demo@1.0.0",
          "debug@2.2.0"
        ]
      ],
      "topLevel": 1
    },
    {
      "package": "ms",
      "title": "Regular Expression Denial of Service (ReDoS)",
      "published": "2017-05-15T06:02:45.497Z",
      "credit": [
        "Snyk Security Research Team"
      ],
      "id": "npm:ms:20170412",
      "ids": {
        "CWE": [
          "CWE-400"
        ],
        "CVE": [],
        "ALTERNATIVE": [
          "SNYK-JS-MS-10509"
        ]
      },
      "vulnerable": "<2.0.0",
      "severity": "low",
      "description": " ... ",
      "nsolidMetaData": {
        "hidden": false
      },
      "packages": [
        "/var/my-app/node_modules/ms"
      ],
      "depsChains": [
        [
          "nsolid-dev-demo@1.0.0",
          "debug@2.2.0",
          "ms@0.7.1"
        ]
      ],
      "topLevel": 1
    }
  ]
}

profile

Generates a V8 CPU profile of the specified process.

Usage

$ nsolid-cli profile --id=[agent id] > my.cpuprofile

Once the profile file has been created, it can be opened using Chrome’s Development Tool’s CPU Profile Debugging Tool.

Note: To load the file, Chrome requires that the generated file have the extension .cpuprofile.

snapshot

Generates a V8 heap snapshot of the specified process.

Usage

$ nsolid-cli snapshot --id=[agent id] > my.heapsnapshot

Once the snapshot file has been created, it can be opened using Chrome’s Development Tool’s heap snapshot browser

Note: To load the file, Chrome requires that the generated file have the extension .heapsnapshot.

startup-times

Lists the time to reach certain process lifecycle startup phases from initial process execution.

Usage

$ nsolid-cli startup-times

Returns a JSON stream including the following properties:

Example JSON Result

{
  "loaded_environment": [
    0,
    322526338
  ],
  "initialized_node": [
    0,
    120919
  ],
  "initialized_v8": [
    0,
    240910
  ],
  "id": "5dd6f7a940bfc3633cc3ffc82332640d51ce5112",
  "time": "2017-12-04T01:32:30.042Z",
  "tags": [
    "region:north",
    "zone:A"
  ],
  "app": "my-app-name",
  "hostname": "my-computer.local"
}

Additional timers can be added to your application with custom lifecycle events.

vulnerabilities

Returns known security vulnerabilities for all processes.

Usage

$ nsolid-cli vulnerabilities

Returns a JSON object representing all current known vulnerabilities.

Example JSON Result

{
  "time": "2017-12-04T01:34:54.805Z",
  "vulnerabilities": [
    {
      "report": {
        "package": "ms",
        "title": "Regular Expression Denial of Service (DoS)",
        "published": "2015-11-06T02:09:36.187Z",
        "credit": [
          "Adam Baldwin"
        ],
        "id": "npm:ms:20151024",
        "ids": {
          "CWE": [
            "CWE-400"
          ],
          "CVE": [
            "CVE-2015-8315"
          ],
          "NSP": 46,
          "ALTERNATIVE": [
            "SNYK-JS-MS-10064"
          ]
        },
        "vulnerable": "<=0.7.0",
        "severity": "medium",
        "description": " ... ",
        "nsolidMetaData": {
          "hidden": false
        }
      },
      "processes": [
        {
          "id": "e1c17bc36d7a9cc76ead259ace0307d4b9705646",
          "app": "my-app-name",
          "tags": [
            "region:north",
            "zone:A"
          ],
          "hostname": "my-computer.local",
          "topLevel": 1,
          "depChains": [
            [
              "ms@0.7.0"
            ]
          ]
        }
        ...
      ]
    },
  ...
 ]
}

query

Subscribe to matching agent data on an interval.

Usage

$ nsolid-cli query --q id=5

Returns newline delimited JSON objects with framing objects.

Example JSON Result

{"time":"2017-12-04T01:42:29.502Z","type":"start"}
{"time":"2017-12-04T01:42:29.502Z","type":"interval-start"}
{"time":"2017-12-04T01:42:29.502Z","type":"agent-enter","id":"5dd6f7a940bfc3633cc3ffc82332640d51ce5112","info":{ ... },"metrics":{ ... }
{"time":"2017-12-04T01:42:29.502Z","type":"summary","totalAgents":9}
{"time":"2017-12-04T01:42:29.502Z","type":"interval-end"}

summary

Pull a summary of all connected N|Solid processes. Reports counts of matched properties or resources.

Usage

$ nsolid-cli summary

Example JSON Result

{
  "time": "2017-12-04T02:12:02.506Z",
  "processes": 9,
  "apps": {
    "my-app-name": 5,
    "api-server": 1,
    "web": 1,
    "batch-service": 2
  },
  "tags": {
    "region:north": 4,
    "zone:A": 5,
    "region:south": 5,
    "zone:B": 2,
    "zone:C": 2
  },
  "rss": 693153792,
  "cpu": 3.4667812728278924,
  "node": {
    "8.9.1": 9
  },
  "nsolid": {
    "3.0.0": 9
  },
  "packages": 552,
  "vulnerabilities": {
    "npm:ms:20170412": 4,
    "npm:qs:20140806": 2,
  },
  "hiddenVulnerabilities": {}
}

export-settings

Pull a JSON object with one or many settings configuration to backup or import later. The only settings available to export are: integrations, savedViews and notifications.

Usage

$ nsolid-cli export-settings --item integrations,savedViews,notifications

Example JSON Result

{
  "_metadata": {
    "_timestamp": "",
  },
  "integrations": {},
  "notifications": {},
  "savedViews": {}
}

import-settings

Apply a previously backup/exported settings. The only settings available to import are: integrations, savedViews and notifications.

Usage

$ nsolid-cli import-settings --item integrations --action clean --attach backup.nsconfig

N|Solid API

Table of Contents N|Solid API

• nsolid.start(config)
• nsolid.appName
• nsolid.appVersion
• nsolid.app
• nsolid.clearFatalError()
• nsolid.config
• nsolid.getThreadName()
• nsolid.id
• nsolid.info()
• nsolid.metrics()
• nsolid.metricsPaused
• nsolid.onLicense(OnLicenseHandler)
• nsolid.packages()
• nsolid.pauseMetrics()
• nsolid.processStart
• nsolid.profile([duration])
• nsolid.profileEnd()
• nsolid.resumeMetrics()
• nsolid.saveFatalError(error)
• nsolid.setThreadName(name)
• nsolid.snapshot()
• nsolid.startupTimes()
• nsolid.statsd
• nsolid.tags
• nsolid.traceStats
• nsolid.zmq

nsolid.start()

Another option, which can be useful if you don't have access to static configuration data, is to use the N|Solid API from within your application. For example, if you have a configuration management platform or database that knows the hostname and port to your Console, you may obtain it and then invoke nsolid.start() to connect and begin reporting metrics. This can be done at any time, as long as your process is not already connected to an N|Solid Console. The keys are the same as the package.json keys documented above, with the following difference:

• version is specified as appVersion

Example:

const nsolid = require('nsolid')
nsolid.start({
  command: 'nsolid-command-host.local:9001',
  tags: ['nsolid-awesome', 'Auth service'],
  app: 'nsolid-awesome',
  appVersion: '1.0.0'
})

nsolid.appName

Get the N|Solid app name (equal to nsolid.app getter).

const nsolid = require('nsolid')
nsolid.appName
'nsolid-awesome'

nsolid.appVersion

Get the N|Solid app version (taken from the package.json file).

const nsolid = require('nsolid')
nsolid.appVersion
'1.0.0'

nsolid.app

Get the N|Solid app name.

const nsolid = require('nsolid')
nsolid.app
'nsolid-awesome'

nsolid.clearFatalError()

const nsolid = require('nsolid')
process.on('uncaughtException', err => {
  nsolid.clearFatalError(err)
  proces.exit(1)
})

nsolid.config

Get the N|Solid application config.

const nsolid = require('nsolid')
nsolid.config
{
  app: 'nsolid-awesome',
  appVersion: '1.0.0',
  blockedLoopThreshold: 200,
  env: 'prod',
  hostname: 'nsolid-host',
  interval: 3000,
  pauseMetrics: false,
  pubkey: '<3',
  statsdBucket: 'nsolid.${env}.${app}.${hostname}.${shortId}',
  tags: ['nsolid-awesome', 'Auth service']
}

nsolid.getThreadName()

Get an N|Solid worker thread name.

const { Worker, isMainThread } = require('worker_threads');
const { getThreadName, setThreadName } = require('nsolid')

if (!isMainThread) {
  console.log(getThreadName()) // ''
  setThreadName('worker-parser')
  console.log(getThreadName()) // worker-parser
  return setTimeout(() => {}, 1000)
}
const worker = new Worker(__filename)

nsolid.id

Get the N|Solid agent id.

const nsolid = require('nsolid')
nsolid.id
'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

nsolid.info()

It returns relevant information about the running process and platform is running on.

const nsolid = require('nsolid')
nsolid.info()
{
  id: '14e708a4bdb1fc5763fa1f29a9567229ab1b1ac4',
  app: 'beautiful-nsolid-app',
  appVersion: '1.0.0',
  tags: undefined,
  pid: 855556,
  processStart: 1600731441576,
  nodeEnv: undefined,
  execPath: 'your-beautiful-app-directory/beautiful-nsolid-app',
  main: 'index.js',
  arch: 'x64',
  platform: 'linux',
  hostname: 'NodeSource',
  totalMem: 16302604288,
  versions: {
    node: '12.18.4',
    nsolid: '4.0.0',
    v8: '7.8.279.23-node.39',
    uv: '1.38.0',
    zlib: '1.2.11',
    brotli: '1.0.7',
    ares: '1.16.0',
    modules: '72',
    nghttp2: '1.41.0',
    napi: '6',
    llhttp: '2.1.2',
    http_parser: '2.9.3',
    openssl: '1.1.1g',
    cldr: '37.0',
    icu: '67.1',
    tz: '2019c',
    unicode: '13.0'
  },
  cpuCores: 4,
  cpuModel: 'Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz'
}

The nsolid.info() method can be called asynchronously too:

const nsolid = require('nsolid')
nsolid.info((err, info) => {
  if (!err) {
    // Yay!
  }
})

nsolid.metrics()

It retrieves a list of enviroment and process metrics.

const nsolid = require('nsolid')
nsolid.metrics()
{
  threadId: 0,
  timestamp: 0,
  activeHandles: 0,
  activeRequests: 0,
  heapTotal: 4464640,
  totalHeapSizeExecutable: 524288,
  totalPhysicalSize: 3685528,
  totalAvailableSize: 2194638976,
  heapUsed: 2666008,
  heapSizeLimit: 2197815296,
  mallocedMemory: 8192,
  externalMem: 833517,
  peakMallocedMemory: 91744,
  numberOfNativeContexts: 1,
  numberOfDetachedContexts: 0,
  gcCount: 1,
  gcForcedCount: 0,
  gcFullCount: 0,
  gcMajorCount: 0,
  gcDurUs99Ptile: 812,
  gcDurUsMedian: 812,
  dnsCount: 0,
  httpClientAbortCount: 0,
  httpClientCount: 0,
  httpServerAbortCount: 0,
  httpServerCount: 0,
  dns99Ptile: 0,
  dnsMedian: 0,
  httpClient99Ptile: 0,
  httpClientMedian: 0,
  httpServer99Ptile: 0,
  httpServerMedian: 0,
  loopIdleTime: 0,
  loopIterations: 0,
  loopIterWithEvents: 0,
  eventsProcessed: 0,
  eventsWaiting: 0,
  providerDelay: 0,
  processingDelay: 0,
  loopUtilization: 1,
  res5s: 0,
  res1m: 0,
  res5m: 0,
  res15m: 0,
  loopTotalCount: 0,
  loopAvgTasks: 0,
  loopEstimatedLag: 40.416415,
  loopIdlePercent: 0,
  title: 'nsolid',
  user: 'NodeSource',
  uptime: 0,
  systemUptime: 96314,
  freeMem: 2586660864,
  blockInputOpCount: 0,
  blockOutputOpCount: 0,
  ctxSwitchInvoluntaryCount: 14,
  ctxSwitchVoluntaryCount: 21,
  ipcReceivedCount: 0,
  ipcSentCount: 0,
  pageFaultHardCount: 0,
  pageFaultSoftCount: 1929,
  signalCount: 0,
  swapCount: 0,
  rss: 31719424,
  load1m: 2.79,
  load5m: 2.52,
  load15m: 2.47,
  cpuUserPercent: 86.061918,
  cpuSystemPercent: 15.533329,
  cpuPercent: 101.595247,
  cpu: 101.595247
}

If callback is passed, it returns the metrics asynchronously:

const nsolid = require('nsolid')
nsolid.metrics((err, metrics) => {
  if (!err) {
    // Yay!! All the metrics are in store in metrics param
  }
})

See Metrics in detail for more information about each metric.

nsolid.metricsPaused

Whether the agent is currently retrieving metrics or not.

const nsolid = require('nsolid')
nsolid.metricsPaused
false

See nsolid.pauseMetrics()

nsolid.onLicense(OnLicenseHandler)

Registers a listener to be called anytime a new license is set in the NSolid runtime.

• OnLicenseHandler is the handler of the actual listener.

const nsolid = require('nsolid')
nsolid.onLicense(({ licensed, expired }) => {
  if (expired) {
    Logger.error('Invalid license!')
    return
  }
  doSomethingAwesome()
})

nsolid.packages()

It retrieves the list of packages listed in the package.json file and its dependencies, it returns a list of all packages that can be required by walking through all directories that could contain a module.

const nsolid = require('nsolid')
nsolid.packages()
[
 {
    path: 'nsolid-service/node_modules/js-tokens',
    name: 'js-tokens',
    version: '4.0.0',
    main: 'index.js',
    dependencies: [],
    required: false
 },
 {
    path: 'nsolid-service/node_modules/important-package',
    name: 'important-package',
    version: '3.0.0',
    main: 'main.js',
    dependencies: ['../express', '../object-assign'],
    required: false
 },
 ...
]

The nsolid.packages() method has async implementation too:

const nsolid = require('nsolid')
nsolid.packages((err, packages) => {
  if (!err) {
    console.log(packages)
  }
})
[
 {
    path: 'your-beautiful-app-directory/beautiful-nsolid-app',
    name: 'beautiful-nsolid-app',
    version: '1.5.2',
    main: 'index.js',
    dependencies: [],
    required: false
 }
]

nsolid.pauseMetrics()

It pauses the process metrics collection, and do nothing if metrics were already paused (this method must be called on the main thread).

const nsolid = require('nsolid')
nsolid.pauseMetrics()

See nsolid.metricsPaused.

nsolid.processStart

It gets the start date of the process in milliseconds.

const nsolid = require('nsolid')
nsolid.processStart
1600730465456

nsolid.profile([duration])

This is the way to trigger a CPU profile programmatically from within your application and have the resulting profile saved on your N|Solid console. The profile duration 600000 milliseconds, which is the default if none is specified, and profiler can be stoped before the duration expires.

• duration is the duration in milliseconds of the Cpu profile (600000 milliseconds is the default duration).

const nsolid = require('nsolid')
nsolid.profile(600000 /* durationMilliseconds */, err => {
  if (err) {
    // The profile could not be started!
  }
})

The CPU profiler can be triggered in a synchronous way:

const nsolid = require('nsolid')
try {
  nsolid.profile(600000 /* durationMilliseconds */)
} catch (err) {
  // The profile could not be started!
}

Learn more about Cpu profiling.

nsolid.profileEnd()

To complete the profile before the duration expires, call profileEnd method:

const nsolid = require('nsolid')
nsolid.profileEnd(err => {
  if (err) {
    // Error stopping profiler (may it was not running)
  }
})

Learn more about Cpu profiling.

nsolid.resumeMetrics()

It resumes the process metrics collection, and does nothing if metrics were already resumed (this method must be called on the main thread).

const nsolid = require('nsolid')
nsolid.resumeMetrics()

nsolid.saveFatalError(error)

If you must exit asynchronously from an uncaughtException handler it is still possible to report the exception by passing it to nsolid.saveFatalError() prior to shutting down.

const nsolid = require('nsolid')
process.on('uncaughtException', err => {
  nsolid.saveFatalError(err)
  proces.exit(1)
})

nsolid.setThreadName(name)

If it is preferred to handle workers threads with names, this could be achieved by using the nsolid.setThreadName(name) method.

const { Worker, isMainThread } = require('worker_threads');
const { getThreadName, setThreadName } = require('nsolid')

if (!isMainThread) {
  console.log(getThreadName()) // ''
  setThreadName('worker-renamed')
  console.log(getThreadName()) // worker-renamed
  return setTimeout(() => {}, 1000)
} else {
  const worker = new Worker(__filename)
}

The main thread can also have a name

const { isMainThread } = require('worker_threads');
const { setThreadName } = require('nsolid')

if (isMainThread) setThreadName('app')

A worker (or the main thread) can be named only with strings, otherwise, an exception will be thrown.

const { Worker, isMainThread } = require('worker_threads');
const { getThreadName, setThreadName } = require('nsolid')

if (!isMainThread) {
  try {
    setThreadName(null)
  } catch (err) { /** TypeError [ERR_INVALID_ARG_TYPE]: The "name" argument must be of type string. Received null */ }
} else {
  const worker = new Worker(__filename)
}

nsolid.snapshot()

This is the way to take a heap snapshot programmatically from within your application and have the resulting snapshot(s) saved on your N|Solid console.

const nsolid = require('nsolid')
nsolid.snapshot(err => {
  if (err) {
    // The snapshot could not be created!
  }
})

Snapshots can also be taken on a synchronous way:

const nsolid = require('nsolid')
try {
  nsolid.snapshot()
} catch (err) {
  // The snapshot could not be created!
}

Learn more about Heap snapshots.

nsolid.startupTimes()

It retrieves the startup times of the process in a high resolution array of [seconds, nanoseconds].

const nsolid = require('nsolid')
nsolid.startupTimes()
{
  initialized_node: [ 0, 971137 ],
  initialized_v8: [ 0, 181632 ],
  loaded_environment: [ 0, 3874746 ]
}

The nsolid.startupTimes() method can be called asynchronously:

const nsolid = require('nsolid')
nsolid.startupTimes((err, startupTimes)) {
  if (!err) {
    console.log(startupTimes)
  }
}
{
  initialized_node: [ 0, 971137 ],
  initialized_v8: [ 0, 181632 ],
  loaded_environment: [ 0, 3874746 ]
}

nsolid.statsd

The nsolid.statsd object:

The statsd statuses are:

• unconfigured
• initializing
• connecting
• ready

Usage example:

const nsolid = require('nsolid')
nsolid.statsd.status()
'unconfigured'

The nsolid.statsd.format object:

Usage example:

const nsolid = require('nsolid')
nsolid.statsd.format.bucket()
''

nsolid.tags

Get the N|Solid app tags (taken from the package.json file or NSOLID_TAGS environment variable).

const nsolid = require('nsolid')
nsolid.tags
['nsolid-awesome', 'Auth service']

nsolid.traceStats

The nsolid.traceStats object:

nsolid.zmq

The nsolid.zmq object:

The zmq agent statuses are:

• buffering
• connecting
• initializing
• ready
• unconfigured

Usage example:

const nsolid = require('nsolid')
nsolid.zmq.status()
'ready'

Metrics in Detail

N|Solid makes available a rich set of metrics covering the Node.js process's host system, the process itself, the internal behavior of Node.js, and the internal behavior of the V8 JavaScript engine. Combining these data points can give you sophisticated insight into your Node.js deployment.

Many of the key metrics are displayed directly on the N|Solid Console. To make numeric metrics available to your StatsD collector directly from the N|Solid Process, configure using the NSOLID_STATSD environment variable (see the StatsD section for more detail). For the full set of metrics, use the nsolid-cli utility.

Process Information

System Information

Note: The host system may be actual hardware, a virtual machine, or a container.

System Data

General Process Stats

JavaScript Memory Stats

Note: The memory in "heap" is a subset of the resident set size.

Node.js Event Loop Stats

Note: To learn more about event loop utilization visit The Event loop utilization blogpost.

JavaScript Garbage Collection Stats

HTTP and DNS Stats

Metrics Metadata

pauseMetrics() and resumeMetrics() to dynamically stop and resume metrics collection

This feature allows users to dynamically toggle metrics collection off.

pauseMetrics() will cause the agent to no longer send metrics via zmq or StatsD.

Internally metrics are collected and recalculated every duration of NSOLID_INTERVAL (default: 3 seconds).

When sending metrics is paused the metrics will still be collected and recalculated internally every NSOLID_INTERVAL. This way when the metrics are again sent they aren't skewed for the duration they were paused.

Custom Commands

Custom commands can be triggered via the N|Solid Command Line Interface (CLI). Custom commands allow you to interact with your application's processes in ways specific to your business needs.

Create A Custom Command

To implement a custom command, create a function to handle the command, and register that function with N|Solid.

The custom command function should be defined to take a single parameter, request:

function customCommandHandler(request) { ... }

The request parameter is an object with the following properties/functions:

Your function you must call either request.return() or request.throw() to signal completion of the command.

To get access to N|Solid's built-in nsolid module, call require("nsolid"). A custom command handler is registered using the nsolid.on() function:

The nsolid.on() function takes the following parameters:

nsolid.on(commandName, handler)

Example Custom Command Registration

const nsolid = require("nsolid")
...
nsolid.on("foo", fooCommand)
...
function fooCommand(request) { ... }

Sample Case: Log Level Custom Command

Below is an example of how you can use custom commands to dynamically change the configuration state of the application, specifically the log level. The example assumes that a global boolean variable Verbose is used to indicate whether to log verbosely or not.

//------------------------------------------------------------------------------
// This program is a simple "server" which does nothing, but does implement
// an N|Solid custom command, named `verbose`.  Once you've started this program
// with the N|Solid agent enabled, you can send the `verbose` command as in:
//
// nsolid-cli --id $NSOLID_AGENTID custom --name verbose
// nsolid-cli --id $NSOLID_AGENTID custom --name verbose --data on
// nsolid-cli --id $NSOLID_AGENTID custom --name verbose --data off
//
// All these forms get or set the "verbose" level of logging in the program.
//
// The server logs a message every second when "verbose" is off, and logs
// an additional message after that one when "verbose" is on.  The default
// setting of "verbose" is false.
//------------------------------------------------------------------------------

"use strict"

// get access to N|Solid's built-in module `nsolid`
const nsolid = require("nsolid")

// the current "verbose" level
let Verbose = false

// register the `verbose` command for nsolid-cli
nsolid.on("verbose", verboseCommand)

// your server which doesn't do much
setInterval(onInterval, 2000)

console.log("N|Solid custom command demo - log-level - started")
console.log("")
console.log("to use the verbose command with `nsolid-cli`, run:")
console.log("   nsolid-cli --id $NSOLID_AGENTID custom --name verbose")
console.log("   nsolid-cli --id $NSOLID_AGENTID custom --name verbose --data on ")
console.log("   nsolid-cli --id $NSOLID_AGENTID custom --name verbose --data off ")

//------------------------------------------------------------------------------
function onInterval() {
  log("interval event!")
  logVerbose("some extra logging here")
}

//------------------------------------------------------------------------------
// implements the `verbose` command  for nsolid-cli
//------------------------------------------------------------------------------
function verboseCommand(request) {

  // if "on" or "off" passed in with --data, set Verbose appropriately
  if (request.value == "on") {
    Verbose = true
  }
  else if (request.value == "off") {
    Verbose = false
  }
  else if (request.value) {
    return request.throw("expecting data of `on` or `off`, got " + request.value)
  }

  // return current value of Verbose
  return request.return({verbose: Verbose})
}

//------------------------------------------------------------------------------
function log(message) {
  console.log(message)
}

//------------------------------------------------------------------------------
function logVerbose(message) {
  if (!Verbose) return
  log(message)
}

When running your application with the N|Solid agent active, you can use the following command to return the current value of the Verbose setting:

$ nsolid-cli --id $NSOLID_AGENTID custom --name verbose

To set Verbose on or off, use one of the following commands:

$ NSOLID_AGENTID="69d916ad395061f80589e20bef9af3cb50ece9cb" # This will change
$ nsolid-cli --id $NSOLID_AGENTID custom --name verbose --data on
$ nsolid-cli --id $NSOLID_AGENTID custom --name verbose --data off

The output of these three CLI commands looks like the following:

{"verbose":false,"time":1472496720972,"timeNS":"1472496720972042976","id":"69d916ad395061f80589e20bef9af3cb50ece9cb","app":"my-verbose-app","hostname":"titania"}

Custom Lifecycle Events

Implementing Custom Lifecycle Events

In addition to the built-in lifecycle events, you can add your own using the process.recordStartupTime(label) function. The label will then be used in the JSON output of the startup-times command.

You can use this to record the times at various stages of your application's startup. For instance:

• when a database connection is requested
• when the database connection is returned
• when an http server is ready for incoming events

Using the CLI

To obtain the startup timing values, you can use the nsolid-cli startup-times command.

N|Solid provides three startup times by default via startup-times:

Usage

$ nsolid-cli startup-times

Example JSON Result

{
  "id": "bf24f4ed072b3bb4b220aa81fa3a73fde8038409",
  "app": "MyApp",
  "hostname": "myApp.example.com",
  "initialized_node":   [ 0, 130404 ],
  "initialized_v8":     [ 0, 482651 ],
  "loaded_environment": [ 0, 620207709 ]
}

This indicates that Node was initialized in 130,404 nanoseconds (which is 130 microseconds, 0.130 milliseconds, or 0.000130 seconds).

Note: The timing information is provided in hrtime format, which is a two element array of [seconds, nanoseconds]. A nanosecond is one billionth (1,000,000,000th) of a second. The time values are the elapsed time since the process was started.

Example: Adding A Custom Timer

Below is an example web server instrumented to provide the time it takes for the web server to start listening to connections:

# http_sample.js

const http = require("http")

const server = http.createServer(onRequest)

server.listen(3000, onListen)

function onListen() {
  console.log("server listening on http://localhost:3000")
  process.recordStartupTime("http_listen")
}

function onRequest(request, response) {
  response.end("Hello, world!")
}

To start this program with the N|Solid agent listening on port 5000, use the following command:

$ NSOLID_APPNAME=http-sample nsolid http_sample.js

To obtain the startup times, including the custom timer, use the following command:

$ nsolid-cli startup-times --app http-sample

Example JSON Result

{
  "id": "bf24f4ed072b3bb4b220aa81fa3a73fde8038409",
  "app": "http-sample",
  "hostname": "http-sample.example.com",
  "initialized_node":   [ 0, 129554 ],
  "initialized_v8":     [ 0, 460521 ],
  "loaded_environment": [ 0, 95201339 ],
  "http_listen":        [ 0, 94902772 ]
}

Glossary

Below are some important terms and phrases related to N|Solid and its ecosystem that appear throughout the documentation.