Serverless

N|Solid can monitor AWS Lambda functions: it collects telemetry during each invocation to help you debug latency and other issues in serverless applications. This page describes how to set up serverless monitoring and how to use the Functions view in the N|Solid Console.

AWS setup

To use AWS resources, configure the AWS SDK credentials in your environment. See the AWS documentation.

Make sure these AWS credential environment variables are set on your machine:

Env
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION
AWS_ACCESS_KEY_ID

Configuration

Use nsolid-serverless to create the necessary infrastructure and configure your serverless application to send telemetry data to the N|Solid Console.

Install

npm i -g @nodesource/nsolid-serverless

Creating the infrastructure to collect data

First, create the SQS queue that receives the telemetry data:

nsolid-serverless infra --install

Collecting telemetry data

Next, add the environment variables and layers your Lambda functions need to send data to the SQS queue you created:

nsolid-serverless functions --install

Help

nsolid-serverless --help

Adding the integration

After your functions are set up, configure the N|Solid Console. Open Settings (gear icon, top right).

In the left sidebar, expand Integrations and select N|Solid Integrations.

Scroll down to AWS SQS Integration and click New AWS SQS Integration. The following form is displayed:

Fill in the form:

• Queue Name: The name of the integration.
• Queue URL: https://sqs.<region>.amazonaws.com/<account-id>/nsolid-serverless-metrics

Click Save Integration.

See Settings for the other integrations.

Monitoring serverless functions in the N|Solid Console

On the Dashboard, select the Functions tab at the top. This tab appears when both applications and functions report to the Console.

The dashboard for the connected function is displayed:

If you have multiple functions, it looks like this:

Function detail view

Click a function's name in the list to see its details:

The detail view shows the function name (which links to the AWS console) and the following information:

• Account ID: The AWS account ID of the function.
• Runtime Version: The version of the Node.js runtime used by the function.
• Function Version: The version of the function.
• N|Solid Layer Version: The version of the N|Solid layer in your AWS Lambda function.
• REGIONS: The AWS region of the function.
• Architecture: The architecture of the function.
• Estimated Cost: The estimated cost of the function.

Metrics

The Metrics tab shows the function's metrics in nine graphs. Click a graph's metric name to choose a different metric.

The following metrics are available.

Telemetry (real-time) API metrics

These metrics come from the Lambda Telemetry API. They are real-time and associated with specific functions.

• Duration: The total time taken for the function to execute from start to finish, measured in milliseconds.
• initDuration: The time taken for the function to initialize, including any set up or warm-up processes, measured in milliseconds.
• billedDuration: The time for which AWS bills the function's execution, measured in milliseconds. This may differ from actual execution time due to rounding or billing increments.
• Invocations: The total number of times the function is called or triggered.
• Max Memory Used: The peak memory usage by the function during its execution.
• Memory Size: The allocated memory size for the function.
• Errors: The total number of errors encountered during the function's execution.
• Error Rate: The ratio of errors to the total number of invocations within the measured time interval.
• Response Duration: The time taken for the function to produce a response, measured in milliseconds.
• Response Latency: The time taken from the invocation of the function until the response is received, measured in milliseconds.
• Produced Bytes: The total amount of data produced by the function, measured in bytes.
• Timeouts: The number of times the function execution exceeded the allowed time limit and was terminated.
• Estimated Cost: The estimated cost of executing the function based on its usage and AWS pricing.
• OOM (Out of Memory): The number of times the function execution failed due to insufficient memory.
• Counters Uptime: The total time the function has been running since it was last started.
• Counters User: The CPU time spent in user mode during the function execution.
• Counters System: The CPU time spent in system mode during the function execution.
• Counters Duration: The total duration of the function execution in milliseconds.
• Counters Idle: The percentage of time the function was idle, waiting for tasks or resources.
• Counters Minor Page Faults: The number of minor page faults (page reclaims) triggered by the function.
• Counters Major Page Faults: The number of major page faults (requiring disk access) triggered by the function.
• Counters Swapped Out: The number of times the function's data was swapped out of memory to disk.
• Counters FS Read: The amount of data read from the file system by the function.
• Counters FS Write: The amount of data written to the file system by the function.
• Counters IPC Sent: The number of inter-process communication messages sent by the function.
• Counters IPC Received: The number of inter-process communication messages received by the function.
• Counters Signals Count: The number of signals received by the function.
• Counters Voluntary Context Switches: The number of times the function voluntarily yielded the CPU.
• Counters Involuntary Context Switches: The number of times the function was forcibly switched out by the CPU scheduler.
• Billed Duration: The total billed duration of the function execution, measured in milliseconds.
• Post Runtime Duration: The duration of the post-execution phase of the function, measured in milliseconds.
• Init Counters Bootstrap Complete: The time at which the function's bootstrap process completed.
• Init Counters Environment: The time at which the Node.js environment was initialized.
• Init Counters Loop Start: The time at which the function's main execution loop started.
• Init Counters Node Start: The time at which the function's node process started.
• Init Counters Start Time: The exact time the function started executing.
• Init Counters V8 Start: The start time of the V8 engine for the function.
• Post Runtime Execution Duration: The duration of the post-execution phase, specifically the execution time, measured in milliseconds.

CloudWatch metrics

These metrics are read from the CloudWatch API by the Lambda Metrics Forwarder. Unlike the Telemetry API metrics, they aren't real-time.

• Invocations: The total number of times the function is invoked.
• Errors: The total number of errors encountered during the function's execution.
• Dead Letter Errors: The number of errors when the function attempts to send a message to a dead letter queue.
• Destination Delivery Failures: The number of times the function failed to deliver messages to a destination.
• Throttles: The number of times the function execution was throttled due to exceeding concurrency limits.
• Provisioned Concurrency Invocations: The number of times the function was invoked with provisioned concurrency.
• Provisioned Concurrency Spillover Invocations: The number of times invocations spilled over beyond the provisioned concurrency limit.
• Maximum Concurrent Executions: The peak number of concurrent executions of the function.
• Maximum Provisioned Concurrent Executions: The peak number of concurrent executions using provisioned concurrency.
• Maximum Unreserved Concurrent Executions: The peak number of concurrent executions without reserved concurrency.
• Maximum Duration: The longest execution time of the function, measured in milliseconds.
• Minimum Duration: The shortest execution time of the function, measured in milliseconds.
• Duration P50: The median (50th percentile) execution time of the function, measured in milliseconds.
• Duration P99: The 99th percentile execution time of the function, measured in milliseconds.
• Maximum Post Runtime Extensions Duration: The longest duration of the post-execution phase, measured in milliseconds.
• Minimum Post Runtime Extensions Duration: The shortest duration of the post-execution phase, measured in milliseconds.
• Post Runtime Extensions Duration P50: The median (50th percentile) duration of the post-execution phase, measured in milliseconds.
• Post Runtime Extensions Duration P99: The 99th percentile duration of the post-execution phase, measured in milliseconds.
• Average Iterator Age: The average age of the iterator, indicating the time taken to process data streams.
• Maximum Iterator Age: The longest time taken to process a data stream item.
• Provisioned Concurrency Utilization: The percentage of provisioned concurrency utilized by the function.
• Duration: The total duration of the function execution, measured in milliseconds.
• Post Runtime Extensions Duration: The duration of the post-execution phase, measured in milliseconds.
• Iterator Age: The age of the iterator, indicating the time taken to process data streams.

Loading OpenTelemetry instrumentation modules

Use the NSOLID_INSTRUMENTATION environment variable in your Lambda configuration to load the OpenTelemetry instrumentation modules you want to use in your application:

• For HTTP requests using the http module, set the NSOLID_INSTRUMENTATION environment variable to http.

• If you're also performing PostgreSQL queries using the pg module, include it in the NSOLID_INSTRUMENTATION environment variable like this: http,pg.

• List every instrumentation module your application needs. This enables tracing for those modules.

Tracing

To enable tracing, set the NSOLID_TRACING_ENABLED=1 environment variable on your Lambda function.

With tracing enabled, you can use the N|Solid Console to troubleshoot HTTP, DNS, and other network request problems. For a description of the Tracing view, see Tracing.

The view looks like this:

The Tracing view has three key components:

• Latency timeline: A timeline of tracing data showing the density of tracing spans.

• Filter: Filters the results by span attributes.

• Results: The traces that match, and their spans. See Span.

note

By default, only traces related to the Lambda invocation are generated. To trace additional operations, set the NSOLID_INSTRUMENTATION environment variable.

These modules are available for tracing:

• aws: AwsInstrumentation
• dns: DnsInstrumentation
• graphql: GraphQLInstrumentation
• grpc: GrpcInstrumentation
• http: HttpInstrumentation
• ioredis: IORedisInstrumentation
• mongodb: MongoDBInstrumentation
• mysql: MySQLInstrumentation
• net: NetInstrumentation
• pg: PgInstrumentation
• redis: RedisInstrumentation

Enable the modules for every operation you want to trace.

Latency timeline

The Latency timeline displays the density of tracing spans. The color of each slot means:

Color	Description
green	Everything is OK.
yellow	Worth a look.
red	Needs investigation.

For example, after a request to the "console" service, the timeline looks like this:

The Console then displays the span information.

Span

A span is the building block of a trace and is a named, timed operation that represents a piece of the workflow in the distributed system. Multiple spans are pieced together to create a trace.

Traces are often viewed as a tree of spans that reflects the time that each span started and completed. It also shows you the relationship between spans. A trace starts with a root span where the request starts. This root span can have one or more child spans, and each one of those child spans can have child spans.

Inspecting span details

To inspect a span's details, click it under Service & Operation:

A span has these attributes:

Attribute	Description
id	the id of the application
app	the name of application
hostname	the name of the host machine
tags	the tags of the application
span_attributes_http_method	the http method of span attributes
duration	the duration of the span
span_attributes_http_status_code	the http status code of the span attributes
span_attributes_http_status_text	the http status text of the span attributes
span_attributes_http_url	the http url of the span attributes
span_end	the end time of the span
span_id	the id of the span
span_name	the name of the span
span_parentId	the parent ID of the span
span_start	the start time of the span
span_status_code	the status code of the span
span_threadId	the thread ID of the span
span_traceId	the trace ID of the span
span_type	the type of the span
resourceSpans	an array of resource spans
attributes	an array of attributes
key	the key of the attribute
value	the value of the attribute
stringValue	the string value of the attribute
telemetry.sdk.language	the language of the telemetry SDK
telemetry.sdk.name	the name of the telemetry SDK
telemetry.sdk.version	the version of the telemetry SDK
cloud.provider	the cloud provider
cloud.platform	the cloud platform
cloud.region	the cloud region
faas.name	the name of the function as a service
faas.version	the version of the function as a service
process.pid	the process ID
process.executable.name	the name of the executable
process.command	the command
process.command_line	the command line
process.runtime.version	the version of the runtime
process.runtime.name	the name of the runtime
process.runtime.description	the description of the runtime
droppedAttributesCount	the number of dropped attributes
MessageAttributes	the message attributes
eventType	the event type
StringValue	the string value
DataType	the data type
instanceId	the instance ID
functionArn	the Amazon Resource Name (ARN) of the function

SBOM

Click SBOM and select JSON or PDF to download the Software Bill of Materials (SBOM) of the function. The SBOM includes:

• Name: The name of the package.
• Version: The version of the package.
• Risk: The level of risk associated with using this package.
• License: The license under which the package is distributed.
• Author: The author of the package.
• Number of CVEs / CWEs: The number of known Common Vulnerabilities and Exposures (CVEs) or Common Weakness Enumerations (CWEs) associated with the package.
• Path: The path to the package within the application.

Changing the time range

To change the time range, use the global date range picker in the navigation bar, including its Quick Presets. See Filter Bar.

Uninstall

To uninstall the resources created by nsolid-serverless, such as the SQS queue, Lambda layer, and Lambda environment variables, run:

nsolid-serverless infra --uninstall