N|Solid Documentation

Versions

Command Line Interface (CLI)

N|Solid comes with a set of predefined endpoints for interaction and introspection that can be accessed at the command-line using the N|Solid Command Line Interface, nsolid-cli.

To see a list of all commands and options available with nsolid-cli, use the -h option:

nsolid-cli -h

The output of the nsolid-cli commands, unless otherwise noted, is line-delimited JSON (each line is separated by a newline delimiter \n). The output shown in the examples below is shown expanded for readability.

--start and --end options

For commands that take --start and --end options, you can pass the following formats for the date/time value:

format description
milliseconds since epoch value returned from Date.now()
yyyy-mm-ddThh:mm:ss ISO date
-Ns N seconds since now
-Nm N minutes since now
-Nh N hours since now
-Nd N days since now
0 now, when used with the --end option

The --end option can only be specified if the --start option is also specified. If the --end option is not specified, the command will continue returning JSON objects till terminated.

application

Returns objects which contains summary information about all the processes in the specified application. The current object will be returned on the specified interval.

Options Description
--app Application name (required)
--interval Number of seconds before returning next current object (default: 1)

Usage

nsolid-cli application --app=my-app-name

Example JSON Result

{
  "time": 1474520062141,
  "app": "my-app-name",
  "hostCount": 1,
  "processCount": 1,
  "processes": [
    {
      "id": "7e5c0f931033a6b1cd8e037b8ea099f9cf78af65",
      "app": "my-app-name",
      "hostname": "my-computer.local",
      "tags": [
        "region:north",
        "zone:A"
      ],
      "cpu": 0.007,
      "heapUsed": 19178512,
      "title": "my-app-name",
      "uptime": 4697.88,
      "pid": 23888,
      "processStart": 1474515364290,
      "packageCount": 80,
      "vulnerabilityCount": 0,
      "thresholds": {}
    }
  ],
  "vulnerabilities": [],
  "vulnerabilityCount": 0
}

Returns a stream of the following objects:

Property Description
time Millisecond since epoch time message was sent
app The NSOLID_APPNAME value
hostCount The number of hosts this application is running on
processCount The number of processes running this application
processes Array of process objects, described below
vulnerabilityCount The number of vulnerabilities found in all processes
vulnerabilities An array of vulnerabilities found

Process objects:

Property Description
id The agent id
app The NSOLID_APPNAME value
hostname The host the process is running on
tags The NSOLID_TAGS values
cpu % cpu being used by the process
heapUsed The size of the JavaScript heap used (bytes)
title The process title
uptime The time in seconds this process has been running
pid The operating system process id
processStart The time the process started
packageCount The number of packages loaded
vulnerabilityCount The number of vulnerabilities found
thresholds Threshold data

applications

Return objects which contains summary information about all the applications available. The current object will be returned on the specified interval.

Options Description
--interval Number of seconds before returning next current object (default: 1)

Usage

nsolid-cli applications

Example JSON Result

{
  "time": 1474520766732,
  "applications": [
    {
      "processCount": 1,
      "vulnerabilityCount": 0,
      "name": "my-app-name",
      "hostCount": 1
    }
  ]
}

Returns a stream of the following objects:

Property Description
time Millisecond since epoch time message was sent
applications Array of application objects, described below

Application objects:

Property Description
name The NSOLID_APPNAME value
processCount The number of processes running this application
vulnerabilityCount The number of vulnerabilities found
hostCount The number of hosts this application is running on

asset

Download an asset.

Options Description
--asset The asset id (required)

Usage

nsolid-cli asset --asset 217040c0-02d1-4956-8648-7fb84b78c65e > my.heapsnapshot

Asset id's are available from the assets command. The bytes of the asset will be written to stdout.

assets

Lists the assets - CPU profiles and heap snapshots - that are currently available for download.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)
--type either 'snapshot' or 'profile', to limit the type of asset returned
--start Return assets that were created after the specified time
--end Return assets that were created before the specified time

Usage

nsolid-cli assets --app my-app-name --type snapshot

Example JSON Result

{
  "time": 1476464230018,
  "asset": "217040c0-02d1-4956-8648-7fb84b78c65e",
  "type": "snapshot",
  "id": "a6d4acf3e713043a293b3fdbd5df9bac885c727a",
  "app": "my-app-name",
  "hostname": "my-computer.local",
  "tags": [
    "region:north",
    "zone:A"
  ],
  "size": 12206681,
  "pid": 77465,
  "title": "my-app-name"
}

Returns a stream of the following objects:

Property Description
time Millisecond since epoch time message was sent
asset an asset id to use the the asset command
type 'snapshot' for heap snapshots, 'profile' for CPU profiles
id The agent id
app The NSOLID_APPNAME value
hostname The host the process is running on
tags The NSOLID_TAGS values
size The size of the asset in bytes
pid The operating system process id
title The process title

async-activity

N|Solid can list any pending asynchronous operations and where the JavaScript callbacks will resume. This can be extremely helpful for determining how many concurrent operations a given N|Solid process is handling and what they are. It can also be useful to determine where an application is hanging, if there are any left-behind callbacks or a misue of resources (e.g. reading the same file repetitively)

The output will list the in-flight asynchronous activity, including the function location and line of source code to reference where the callback will resume.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)

Usage

nsolid-cli async-activity

Example JSON Result

{
  "handles": [
    {
      "name": "onrequest",
      "location": {
        "file": "nsolid-user/server-http.js",
        "line": 3,
        "column": 19,
        "inferredName": ""
      },
      "anonymous": false,
      "code": "function onrequest(req, res) { res.end('ok\n') }",
      "type": {
        "group": "handle",
        "flag": 4,
        "name": "TCP server connection"
      },
      "details": {
        "fd": 10,
        "transport": "TCP",
        "connectionKey": "6::::0",
        "connections": 0,
        "unrefd": false
      },
      "metadata": {
        "async_wrap_provider": "TCPWRAP",
        "async_wrap_providerKey": "15"
      }
    },
    {
      "type": {
        "group": "handle",
        "flag": 64,
        "name": "pipe socket"
      },
      "details": {
        "fd": 1,
        "transport": "Pipe",
        "bytesDispatched": 0,
        "hadError": false,
        "host": null,
        "pendingData": "<Buffer>"
      },
      "metadata": {
        "async_wrap_provider": "PIPEWRAP",
        "async_wrap_providerKey": "8"
      }
    },
    {
      "type": {
        "group": "handle",
        "flag": 64,
        "name": "pipe socket"
      },
      "details": {
        "fd": 2,
        "transport": "Pipe",
        "bytesDispatched": 57985,
        "hadError": false,
        "host": null,
        "pendingData": "<Buffer>"
      },
      "metadata": {
        "async_wrap_provider": "PIPEWRAP",
        "async_wrap_providerKey": "8"
      }
    }
  ],
  "requests": [
    {
      "name": "onreadFile",
      "location": {
        "file": "nsolid-user/server-http.js",
        "line": 14,
        "column": 20,
        "inferredName": ""
      },
      "anonymous": false,
      "code": "function onreadFile(err, src) {\n  if (err) return console.error(err)\n  readAgain()\n}",
      "type": {
        "group": "request",
        "flag": 4,
        "name": "readFile",
        "api": "fs.readFile(filename[, options], callback)"
      },
      "details": {
        "path": "nsolid-user/server-http.js",
        "flag": "r",
        "isUserFd": false,
        "buffers": [
          "<Buffer>"
        ],
        "buffer": "<Buffer>",
        "pos": 0,
        "encoding": null,
        "err": null
      },
      "metadata": {
        "async_wrap_provider": "FSREQWRAP",
        "async_wrap_providerKey": "3"
      }
    }
  ],
  "pending": [],
  "time": 1474407457414,
  "id": "8ed04c57f248624c6068945b68a5f023497498cd",
  "app": "runtime/async-activity/03-server-http_app",
  "hostname": "ebc192f5fae2"
}

Returns a stream of the following objects:

Property Description
handles tend to be longer-lived larger-scale asynchronous operations, such as open sockets or timers
requests tend to be shorter-lived smaller-scale operations such as writing to file handles and other file-related operations
pending represent async activity that is lower level and a lot of times are created by cod inside core, like DNS lookups, signal handlers and TCP connections

custom

Invoke a custom command. For more information on custom commands, see Custom Commands.

Options Description
--id The agent id (required)
--name The name of the custom command (required)
--data Data to be sent with the command

Usage

nsolid-cli custom --id=[agent id] --name=verbose --data=off

Example JSON Result

{
  "time": 1476754051460,
  "id": "81535293aea1fe8c1e2f3f7518d8db3f96cf7b39",
  "app": "nsolid2",
  "hostname": "pmuellr-MacBook-Pro.local",
  "tags": [
    "foobar"
  ],
  "result": {
    "verbose": false
  }
}

Returns a JSON object with the following properties

Property Description
time Millisecond since epoch time message was sent
id The agent id
app The NSOLID_APPNAME value
hostname The host the process is running on
tags The NSOLID_TAGS values
result The result of the custom command

info

Returns objects which contain static information about processes and the hosts they are running on.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)
--field Output field (may be specified multiple times)
--start Return objects for processes that started after the specified time
--end Return objects for processes that started before the specified time

Usage

nsolid-cli info

Example JSON Result

{
  "id": "7e5c0f931033a6b1cd8e037b8ea099f9cf78af65",
  "tags": [
    "region:north",
    "zone:A"
  ],
  "pid": 23888,
  "processStart": 1474515364290,
  "execPath": "/Users/me/nsolid",
  "main": "/Users/me/Projects/my-app-name.js",
  "arch": "x64",
  "platform": "darwin",
  "totalMem": 17179869184,
  "versions": {
    "http_parser": "2.7.0",
    "node": "6.5.1",
    "nsolid": "2.0.0",
    "v8": "5.1.281.81",
    "uv": "1.9.1",
    "zlib": "1.2.8",
    "ares": "1.10.1-DEV",
    "icu": "57.1",
    "modules": "48",
    "openssl": "1.0.2h",
    "nsolid_lib": {
      "v8_profiler": "nsolid",
      "sodium": "nsolid",
      "cjson": "nsolid",
      "function_origin": "nsolid",
      "nan": "v2.3.5",
      "cli": "master",
      "agent": "master",
      "zmq-bindings": "nsolid",
      "zmq": "nsolid",
      "persistents_with_classid": "v1.1.0"
    }
  },
  "cpuCores": 8,
  "cpuModel": "Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz",
  "time": 1474520894162,
  "app": "my-app-name",
  "hostname": "my-computer.local"
}

Returns a stream of the following objects:

Property Description
time Millisecond since epoch time message was sent
id The agent id
app The NSOLID_APPNAME value
hostname The host the process is running on
tags The NSOLID_TAGS values
pid Operating system process id
processStart The time the process started
execPath Path of the executable running the application
main The main module used when the application started up
arch The CPU architecture
platform Name of the N|Solid platform
totalMem Total available memory in the system
cpuCores The number of CPU cores
cpuModel The CPU model
versions Object describing the versions of components used in the runtime

info with the --force option

Requests current information from each running agent, or specified agents if using a filter.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)
--field Output field (may be specified multiple times)

list

Returns an array of all available matching N|Solid processes, along with their most recent info and metrics data.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)

Usage

nsolid-cli list

Example JSON Result

{
  "id": "7e5c0f931033a6b1cd8e037b8ea099f9cf78af65",
  "app": "my-app-name",
  "hostname": "my-computer.local",
  "tags": [
    "region:north",
    "zone:A"
  ],
  "info": { ... output from the info command ... },
  "metrics": { ... output from the metrics command ... },
}

Returns an array of the following object shapes:

Property Description
info The object returned from the info command
metrics The object returned from the metrics command

metrics

Returns a stream of metrics records matching the query string parameters.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)
--field Output field (may be specified multiple times)
--start Return metrics created after the specified time
--end Return metrics created before the specified time

Usage

nsolid-cli metrics

Example JSON Result

{
  "time": 1474521264390,
  "uptime": 5900.431,
  "rss": 88784896,
  "heapTotal": 26275840,
  "heapUsed": 22151064,
  "activeRequests": 0,
  "activeHandles": 7,
  "title": "my-app-name",
  "user": "me",
  "cpu": 0,
  "freeMem": 190222336,
  "systemUptime": 1705341,
  "load1m": 2.20068359375,
  "load5m": 1.9150390625,
  "id": "7e5c0f931033a6b1cd8e037b8ea099f9cf78af65",
  "load15m": 1.81689453125,
  "cpuSpeed": 2500,
  "app": "my-app-name",
  "hostname": "my-computer.local",
  "percentCpuThreshold": -1,
  "percentHeapThreshold": -1,
  "percentHostMemoryUsedByProcess": 0.0052,
  "percentHostMemoryUsed": 0.9889,
  "tags": [
    "region:north",
    "zone:A"
  ]
}

Returns a stream of the following objects:

Property Description
time Millisecond since epoch time message was sent
id The agent id
app The NSOLID_APPNAME value
hostname The host the process is running on
tags The NSOLID_TAGS values
uptime The time in seconds this process has been running
rss The total Resident Set Size (total memory used) by this process (bytes)
heapTotal The total allocated size of the JavaScript heap (bytes). A subset of rss
heapUsed The amount of heapTotal being used by JavaScript (bytes)Returns live metrics reflecting the health and resource utilization of the process.
activeRequests The number of active requests the event loop will process. For more information see the async-activity command.
activeHandles The number of active handles the event loop will process. For more information, see the async-activity command.
title The process title
user The user the process is running as
cpu % cpu being used by the process
freeMem The amount of free (unused) memory in bytes
systemUptime The uptime of the system, in seconds
load1m The one-minute load average*
load5m The five-minute load average*
load15m The fifteen-minute load average*
cpuSpeed The current speed of the CPU (averaged across all cores) in MHz
percentCpuThreshold Percent of CPU threshold used (or -1 if no threshold set)
percentHeapThreshold Percent of heap threshold used (or -1 if no threshold set)
percentHostMemoryUsedByProcess Percent of memory used by process
percentHostMemoryUsed Percent of memory used by all processes

metrics with the --force option

Requests current metrics from each running agent, or specified agents if using a filter.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)
--field Output field (may be specified multiple times)

packages

Return the list of packages and modules available in the specified process.

Options Description
--id The agent id (required)

Usage

nsolid-cli packages --id=[agent id]

Example JSON Result

{
  "packages": [
    {
      "path": "/Users/me/my-app-name",
      "name": "my-app-name",
      "main": "my-app-name",
      "version": "1.0.0",
      "dependencies": [
        "node_modules/async",
        "node_modules/bole",
        "node_modules/client-request",
        "node_modules/lodash",
        "node_modules/marked",
        "node_modules/minimist",
        "node_modules/mkdirp",
        "node_modules/semver",
        "node_modules/stringcase",
        "node_modules/toml",
        "node_modules/user-home",
        "node_modules/uuid"
      ],
      "modules": [
        {
          "path": "index.js",
          "shasum": "b76d1c3b6af5515dca0b36ed0ef87b248d9acc94"
        }
      ]
    }
  ]
}

Returns an object with the following shape:

Property Description
time Millisecond since epoch time message was sent
id The agent id
app The NSOLID_APPNAME value
hostname The host the process is running on
tags The NSOLID_TAGS values
packages A packages object, described below

Packages objects:

Property Description
path The path the package was loaded from
name The name of the package
version The version of the package
main The main module for the package
dependencies An array of relative package names this package depended on
modules An array of module objects, described below

Modules objects:

Property Description
path The package-relative path name of the module loaded
shasum The SHASUM hash of the module contents

ping

Have all matching processes send a simple response back.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)

Usage

nsolid-cli ping

Example JSON Result

{
  "pong": true,
  "id": "7e5c0f931033a6b1cd8e037b8ea099f9cf78af65",
  "time": 1474521577972
}

Returns a stream of the following objects:

Property Description
time Millisecond since epoch time message was sent
id The agent id
pong Will always have the value true

profile

Have a CPU profile taken in in the specified process.

Options Description
--id The agent id (required)
--duration Duration of profile in seconds. Default is 5 seconds

Usage

nsolid-cli profile --id=[agent id] > my.cpuprofile

Returns a V8 CPU profile.

Once the profile file has been created, it can be opened using Chrome’s Development Tool’s CPU Profile Debugging Tool. It’s important to note that Chrome requires the generated file to have the extension .cpuprofile to load the file.

snapshot

Have a heap snapshot taken in in the specified process.

Options Description
--id The agent id (required)

Usage

nsolid-cli snapshot --id=[agent id] > my.heapsnapshot

Returns a V8 heap snapshot.

Once the snapshot file has been created, it can be opened using Chrome’s Development Tool’s heap snapshot browser It’s important to note that Chrome requires the generated file to have the extension .heapsnapshot to load the file.

startup-times

Lists the time from initial process execution to reach certain process lifecycle startup phases.

Options Description
--id The agent id
--app The NSOLID_APPNAME value
--hostname The host the process is running on
--tag An NSOLID_TAGS value (may be specified multiple times)

Usage

nsolid-cli startup-times

Example JSON Result

{
  "time": 1476755309783,
  "id": "7e5c0f931033a6b1cd8e037b8ea099f9cf78af65",
  "app": "my-app-name",
  "hostname": "my-computer.local",
  "initialized_node": [
    0,
    166736
  ],
  "initialized_v8": [
    0,
    366153
  ],
  "loaded_environment": [
    0,
    80486563
  ]
}

Returns a stream of the following objects:

Property Description
time Millisecond since epoch time message was sent
id The agent id
app The NSOLID_APPNAME value
hostname The host the process is running on
initialized_node The time it took to initialize the Node internals
initialized_v8 The time it took to initialize the V8 engine
loaded_environment The time it took to complete all initialization, which includes running some of node's internal JavaScript code, and your main module's top-level code

The times are provided in hrtime syntax: an array of 2 integers representing the time since the initial process execution in [seconds, nanoseconds].

Additional timers can be added to your application with custom lifecycle events.

vulnerabilities

Returns the known security vulnerabilities of all processes.

Options Description
--interval Number of seconds before returning next current object (default: 1)

Usage

nsolid-cli vulnerabilities

Example JSON Result

{
  "time": 1476755941704,
  "vulnerabilities": [
    {
      "package": "moment",
      "title": "Regular Expression Denial of Service",
      "published": "2016-02-01T19:00:03.862Z",
      "credit": [
        "Adam Baldwin"
      ],
      "ids": {
        "CWE": [
          "CWE-400"
        ],
        "CVE": [],
        "NSP": 55,
        "ALTERNATIVE": [
          "SNYK-JS-MOMENT-10084"
        ]
      },
      "vulnerable": "<=2.11.1",
      "description": "markdown descript of vulnerability",
      "apps": [
        {
          "appname": "my-app-name",
          "processes": [
            {
              "id": "7e5c0f931033a6b1cd8e037b8ea099f9cf78af65",
              "hostname": "my-computer.local",
              "pid": 23888,
              "loaded": [
                {
                  "version": "2.11.0",
                  "path": "/Users/me/Projects/my-app-name/node_modules/moment"
                }
              ]
            }
          ]
        }
      ],
      "patches": [
        "... url to patches ..."
      ]
    }
  ]
}

Returns a stream of the following objects:

Property Description
time Millisecond since epoch time message was sent
vulnerabilities An array of vulnerability objects, described below

Vulnerability objects:

Property Description
package the package with the vulnerability
title the title of the vulnerability
published date the vulnerability was published
credit names of the people credited with finding the vulnerability
ids identifiers for the vulnerability; CVE, CWE, etc
vulnerable semver expression describing what versions of the package are vulnerable
description description of the vulnerability, in markdown
apps an array of application objects, describing which applications and processes are running the vulnerable module, along with the version loaded and path to the package
patches link to patches available for the vulnerability