Use OpenTelemetry

Enable tracing in the N|Solid Runtime and use the Tracing view in the N|Solid Console to follow requests through your services and find I/O bottlenecks, latency issues, and errors.

About tracing

Tracing collects information throughout the lifecycle of an HTTP, DNS, or other request as it moves through a Node.js application. N|Solid creates OpenTelemetry spans for HTTP clients and servers and for DNS lookups automatically, and you can add spans from other libraries with OpenTelemetry instrumentation. Use the collected data to debug latency, monitor services, and find the causes of errors.

Enable tracing

With environment variables

Set NSOLID_TRACING_ENABLED=1 when you start your application:

NSOLID_SAAS='<your-saas-token>' NSOLID_TRACING_ENABLED=1 NSOLID_APPNAME=my-app node index.js

You can also set tracingEnabled: true in the nsolid section of your package.json. These related settings are optional:

Environment variable	Description
NSOLID_TRACE_SAMPLE_RATE	Share of traces to record, from 0 to 1. The default is 1 (all traces).
NSOLID_TRACING_MODULES_BLACKLIST	Comma-separated list of built-in modules not to trace: http, dns.

See Runtime configuration for all options.

In the N|Solid Console

You can turn tracing on or off for a running process without restarting it:

1. Open the process.
2. Click the Process Configuration icon.
3. In the Dynamic Configuration dialog, turn on Tracing. You can also exclude the http or dns modules under Modules Blacklist.

You can also use the Tracing Enabled icon to open the Tracing dialog.

Open the Tracing view

In the N|Solid Console, open the navigation menu (☰) and select Tracing.

The Tracing view looks like this:

It has three main parts:

• Timeline graph: shows the density of spans over time.
• Filter: narrows the results by span attributes.
• Spans (results): the spans that match. A span is a named, timed operation that represents one piece of the workflow in a distributed system. Spans are pieced together to form a trace.

Timeline graph

The timeline graph shows the density of spans over time. The color of each slot indicates its status:

Color	Meaning
Green	Everything is OK.
Yellow	Worth a look.
Red	Needs attention.

For example, after a request to the console service in the example below, the timeline shows its spans:

Spans and traces

A span is the building block of a trace: a named, timed operation that represents a piece of the workflow in a distributed system.

A trace is usually shown as a tree of spans that reflects when each span started and finished, and how the spans relate to each other. A trace starts with a root span, where the request begins. The root span can have one or more child spans, and each child span can have its own children.

Inspect span details

Open a trace, then click a span in the Service & Operation column to see its details:

The details heading links to the application and shows the request URL. The Attributes list shows the span attributes, including:

Attribute	Description
span_attributes_http_method	HTTP method of the request.
span_attributes_http_status_code	HTTP status code of the response.
span_attributes_http_status_text	HTTP status text of the response.
span_attributes_http_url	URL of the request.
span_duration	Duration of the span.
span_end	End time of the span.
span_id	ID of the span.
span_kind	Kind of span, such as client or server.
span_name	Name of the span.
span_parentId	ID of the parent span.
span_start	Start time of the span.
span_status_code	Status code of the span.
span_threadId	ID of the thread that created the span.
span_traceId	ID of the trace the span belongs to.
span_type	Type of span, such as HTTP or DNS.

Change update settings

Configure updates at the top of the Tracing view:

Turn Real Time Update on or off, and choose the update interval: 10 seconds (the default), 15 seconds, 30 seconds, or 1 minute.

Change the time range

To change the time range, click the calendar icon below the timeline graph:

Select a time range in the calendar. The timeline graph range updates every minute and moves in one-minute steps.

Filter results

Use the global filter bar at the top of the page to filter spans by their attributes. The Tracing view supports filters such as Application Name, Hostname, Tag, Span Name, Span Type, Span Kind, Span Status Code, Span HTTP Method, Span HTTP Status Code, Span HTTP URL, and Span DNS Hostname.

Example: trace a distributed system

Distributed tracing extends HTTP tracing across several services. This example uses three services:

1. console.js handles authentication. For /auth, it calls the google-auth service. For /auth-2fa, it calls the google-auth service's /2fa endpoint.

   // console.js
   const http = require('http')
   const PORT = process.env.PORT || 3000
   http.createServer((req, res) => {
     if (req.url === '/auth') {
       console.log('AUTH handler')
       const authURL = 'http://localhost:4000'
       const gReq = http.request(authURL)
       gReq.end()
       gReq.on('close', () => res.writeHead(200).end('Google made it'))
     } else if (req.url === '/auth-2fa') {
       const authURL = 'http://localhost:4000/2fa'
       const twoFactor = http.request(authURL)
       twoFactor.end()
       twoFactor.on('close', () => res.writeHead(200).end('Auth made it with sms'))
     } else {
       res.end('ok')
     }
   }).listen(PORT)

2. google-auth.js handles authentication. Its /2fa endpoint performs two-factor authentication by calling an SMS service.

   // google-auth.js
   const http = require('http')
   const PORT = process.env.PORT || 4000
   http.createServer((req, res) => {
     if (req.url === '/2fa') {
       const smsServiceURL = 'http://localhost:5000'
       const smsService = http.request(smsServiceURL)
       res.writeHead(200).end('Auth made it')
       smsService.end()
     } else {
       console.log('Auth root handler')
       res.writeHead(200).end('ok')
     }
   }).listen(PORT)

3. twilio.js is the SMS service.

   // twilio.js
   const http = require('http')
   const PORT = process.env.PORT || 5000
   http.createServer((req, res) => {
     res.writeHead(200).end('ok')
   }).listen(PORT)

Start the three services with tracing enabled, then send a request:

NSOLID_SAAS='<your-saas-token>' NSOLID_TRACING_ENABLED=1 NSOLID_APPNAME=console node console.js &
NSOLID_SAAS='<your-saas-token>' NSOLID_TRACING_ENABLED=1 NSOLID_APPNAME=google-auth node google-auth.js &
NSOLID_SAAS='<your-saas-token>' NSOLID_TRACING_ENABLED=1 NSOLID_APPNAME=twilio node twilio.js &
curl http://localhost:3000/auth-2fa

The N|Solid Console receives spans from every service involved. The request to /auth-2fa produces a trace that starts in the console service, continues to the google-auth service, and ends at the SMS (twilio) service:

A request to /auth produces a shorter trace, from the console service to the google-auth service only.