Metrics
Metrics are essential for observability and performance monitoring in ADS. They help you:
- Track performance trends over time.
- Identify potential issues in request processing.
- Ensure SLAs (Service Level Agreements) are met.
- Optimize system efficiency by analyzing request patterns.
ADS collects key metrics, following the RED methodology (Rate, Errors, Duration), which are widely used in monitoring distributed systems.
More specifically, ADS collects the following key metrics:
Rate - the number of access requests ADS is serving
Errors - the number of failed access requests
Duration - distribution of the amount of time each access request takes
Rate of successful evaluations - the rate of successful access requests that evaluated to Permit, Deny, Indeterminate, or NotApplicable, respectively
By default, all metrics (including Spring Boot system metrics and ADS-specific metrics) are disabled. You have to explicitly enable your desired metrics in the deployment.yaml file as described the Enable metrics section of the Basic configuration.
Pull or push
ADS supports two models for sending metrics to a backend:
| Model | Description | Example backend |
|---|---|---|
| Pull | The backend requests (scrapes) metrics from ADS at a fixed interval. | Prometheus |
| Push | ADS actively sends (pushes) metrics to a backend at set intervals. | InfluxDB |
Pull
ADS exposes an HTTP endpoint for Prometheus to pull data from the application every T time units (usually the time unit is "seconds"). This action is also referred to as polling or scraping. The operator configures scrape intervals to balance data accuracy and performance.
Push
In this model, the metrics backend receives metrics data at regular intervals, as configured by the application. ADS is set to push metrics data to the backend every T time units (usually measured in "seconds").
The push interval is set in ADS' metrics configuration. As with the pull model, to ensure optimal monitoring, operators should configure an interval that provides enough data for meaningful analysis while not negatively affecting the performance of the application. This may require some tuning and testing.
Metrics data
ADS collects application-specific metrics to enhance observability and access request tracking:
- Rate of successful evaluations (Permit, Deny, Indeterminate, NotApplicable)
- Domain sequence tracking
- ADS instance-specific annotations
While Spring Boot Actuator provides some built-in system metrics, ADS metrics are tailored for access request evaluation and domain-level monitoring.
ADS metric tags
ADS automatically tags each metric with important metadata that enables filtering and detailed analysis:
ADS instance identity
A unique identifier assigned to each ADS instance (instance-specific filtering) as described in Configuring ADS instance identity.
Namespace and domain name:
Provides hierarchical context for domain-level metrics.
Authorization domain identity
Identifies the domain associated with the authorization request as described in The Identity section.
Domain sequence:
A counter that represents how many domain changes the current instance of ADS has gone through since its startup.
The domain tag (namespace and domain name) is only available when ADS is configured to retrieve the authorization domain from ASM/ADM using the RetrieveByName endpoint.
Both Prometheus and InfluxDB support filtering using metric tags.
Metrics data examples
The following examples show how ADS presents its custom metrics data in Prometheus output.
To display only ADS-specific metrics, disable Spring's default metrics in the deployment.yaml file by setting the sub-properties under metrics.enable to false.
Rate: Number of successful access requests
ADS measures the number of successful access requests served. This metric helps monitor the system's load.
# HELP successful_requests_total
# TYPE successful_requests_total counter
successful_requests_total{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0",} 8.0
Errors: Number of failed access requests
This metric tracks failed access requests. It helps detect authorization failures or policy evaluation issues.
# HELP error_requests_total
# TYPE error_requests_total counter
error_requests_total{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0",} 1.0
Duration: Time taken for authorization requests
This metric measures the request processing time to detect performance bottlenecks.
# HELP duration_info_seconds_max
# TYPE duration_info_seconds_max gauge
duration_info_seconds_max 0.014
# HELP duration_info_seconds
# TYPE duration_info_seconds summary
duration_info_seconds{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", quantile="0.5",} 0.005210112
duration_info_seconds{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", quantile="0.75",} 0.014123008
duration_info_seconds{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", quantile="0.9",} 0.014123008
duration_info_seconds{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", quantile="0.99",} 0.014123008
duration_info_seconds{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", quantile="0.999",} 0.014123008
duration_info_seconds{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", quantile="1.0",} 0.014123008
duration_info_seconds_count 9.0
duration_info_seconds_sum 0.244
duration_info_seconds_bucket{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", le="0.001",} 182.0
duration_info_seconds_bucket{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", le="0.001048576",} 182.0
...
duration_info_seconds_bucket{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", le="30.0",} 1200.0
duration_info_seconds_bucket{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", le="+Inf",} 1200.0
Rate of successful evaluations
Tracks how many access requests resulted in a final authorization decision (Permit, Deny, Indeterminate, NotApplicable).
# HELP decisions_total
# TYPE decisions_total counter
decisions_total{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="0", type="Indeterminate",} 5.0
decisions_total{ads_id="default-1603e8a2",domain="namespace0:domain1", domain_id="80bbc5fa-0647-4f22-804f-949056787c6b",domain_sequence="1", type="Deny",} 1.0
Only requests that result in single decisions are included in the data for decision values.
Enable metrics
By default, all metrics are disabled, as shown in the deployment.yaml sample below, to minimize unnecessary data collection and system overhead. To enable specific metrics, set their respective values to true.
management:
metrics:
enable:
## Spring Boot system metrics ##
jvm: false
logback: false
process: false
system: false
spring: false
tomcat: false
http: false
executor: false
## ADS-specific metrics ##
decisions: false
duration.info: false
error.requests: false
successful.requests: false
distribution:
expiry:
duration.info: 5m # Keeps 'duration.info' metric data for 5 minutes
buffer-length:
duration.info: 3 # Stores 3 samples for 'duration.info' metric
Metrics' properties
The following tables list the metrics' configuration properties.
Spring Boot Actuator properties
Sub-properties under management.metrics.enable. Default values are false.
| Property | Description |
|---|---|
jvm | Enables/disables JVM-related metrics. |
logback | Enables/disables logback logging metrics. |
process | Enables/disables process-level metrics (CPU, memory, uptime). |
system | Enables/disables system resource monitoring (for example, CPU). |
spring | Enables/disables Spring Boot framework-specific metrics. |
tomcat | Enables/disables Tomcat-related metrics (if applicable). |
http | Enables/disables HTTP request metrics. |
executor | Enables/disables thread pool execution metrics. |
ADS-specific metrics' properties
Sub-properties under management.metrics.enable. Default values are false.
| Property | Description |
|---|---|
decisions | Enables/disables tracking of the total number of access request decisions (Permit, Deny, Indeterminate, NotApplicable). |
duration.info | Enables/disables measurement of the time taken to process each access request. |
error.requests | Enables/disables tracking of failed access requests due to policy evaluation errors. |
successful.requests | Enables/disables tracking of the total number of successfully evaluated requests (Permit or Deny). |
Metrics distribution properties
Sub-properties under management.metrics.distribution.
| Sub-property | Description |
|---|---|
expiry | A map where each key is a metric name (meter ID) and the value is the retention duration before expiration. |
expiry.<metric-name> | Sets buffer size for a specific metric. Default value is 5m. |
buffer-length | A map where each key is a metric name (meter ID) and the value is the number of samples stored for distribution statistics. |
buffer-length.<metric-name> | Sets buffer size for a specific metric. For example, duration.info: 3. |
To prevent undersampling, Axiomatics strongly recommends using a minimum expiry value that is at least twice the highest step value used across all enabled metrics backends in your configuration. This applies regardless of using a single or multiple backends. Additionally, it is recommended to determine the value for the step sub-property first and then adjust the expiry value accordingly.
Metrics backend configuration
ADS supports Prometheus (pull-based) and InfluxDB (push-based) as external metrics backends for storing and displaying metrics. To use a metrics backend, you must first enable metrics in the deployment.yaml file.
Follow the instructions below to configure your preferred metrics backend for ADS.
Sub-properties are optional and leaving them empty or using null values will render the entry invalid and prevent system initialization. If a sub-property is not included in the configuration, the default value will be applied.
- Prometheus
- InfluxDB
To enable Prometheus metrics in ADS, ensure your deployment.yaml file includes:
management:
endpoints:
web:
exposure:
include: prometheus, health, info ## This enables the prometheus endpoint
prometheus:
metrics:
export: ## Configuration properties for prometheus
enabled: true ## Required to enable metrics
descriptions: off
step: 30s
The nested properties featured in the sample code above are listed in the following table:
| Property | Sub-property | Description |
|---|---|---|
management.endpoints.web.exposure | include | Specifies which metrics endpoint to expose. In our example, Prometheus. |
management.prometheus.metrics.export | enabled | Enables Prometheus metrics collection. Default value is false. |
descriptions | Enables/disables detailed descriptions in Prometheus output. Default: off. | |
step | Defines how often Prometheus scrapes ADS metrics. Default value is 30s. |
Metrics configuration properties and nested properties
The expiry and buffer-length properties are maps, meaning that each metric (meter ID) can have its own expiration time and buffer length.
Prometheus endpoint
This is the administration endpoint at which ADS exposes the current values of the metrics for Prometheus to pull (or scrape). The endpoint is only available when there is a valid Prometheus configuration enabled.
GET /prometheus
To enable InfluxDB as a metrics backend, add the following configuration to deployment.yaml:
management:
influx:
metrics:
export:
enabled: true ## Required
step: 30s
bucket: my-bucket ## Required
org: my-organization ## Required
token: <String that represents the authentication token> ## Required
uri: http://localhost:8086 ## Required
InfluxDB configuration
The sub-properties bucket, org, and token are created during the set up of your InfluxDB account and their values can then be copied into the configuration file. The token value is automatically generated.
The following table lists the nested properties of the influx configuration, which define how ADS integrates with InfluxDB as a metrics backend:
| Property | Description |
|---|---|
enabled | Enables InfluxDB metrics collection when set to true. |
org | Specifies the InfluxDB organization where metrics are stored. Accepts both Org ID or Name. |
bucket | Defines the destination bucket for storing ADS metrics. Accepts both Bucket ID or Name. |
token | The authentication token required to push data to InfluxDB. Generated automatically in InfluxDB. |
step | Defines the push interval (how often ADS sends metrics). Must be a positive integer + time unit (e.g., 30s, 1m).Default: 1 minute.Important: It is strongly recommended to set step to half the value of decay. Determine step first and adjust decay accordingly. |
uri | Defines the InfluxDB instance URL where metrics are sent. Default: http://localhost:8086. |
InfluxDB configuration nested properties
ADS does not push metrics immediately after collection. Instead, metrics are batched and pushed at regular intervals based on the step setting.
Example: step = 1 minute
If ADS receives 1 request per second, it stores 60 requests before pushing them once per minute to InfluxDB. This ensures efficient data transfer and reduces load on InfluxDB.
For more details on InfluxDB, refer to the official InfluxDB documentationOpens in a new tab.
Enable TLS encryption for InfluxDB
InfluxData strongly recommends enabling TLS, especially if you plan on sending requests to InfluxDB over a network.
To enable TLS:
- Provide valid TLS certificates to InfluxDB.
- Ensure the JVM running ADS is configured to trust InfluxDB’s TLS certificate.
For more information refer to Enable TLS encryptionOpens in a new tab for information on how to enable and configure TLS encryption.