Monitoring and Alerts

• 1: Metrics and alerts
• 2: Monitoring and Alerting using Prometheus
• 3: Monitoring and Alerting using InfluxDB
• 4: Publish Server or Audit Logs to an External Service
• 5: Healthcheck API
• 6: Metrics version 2
• 7: Monitor a Silo Server with Grafana

1 - Metrics and alerts

MinIO publishes metrics using the Prometheus Data Model. You can use any scraping tool to pull metrics data from MinIO for further analysis and alerting.

Starting with MinIO Server RELEASE.2024-07-15T19-02-30Z and MinIO Client RELEASE.2024-07-11T18-01-28Z, metrics version 3 provides additional endpoints. MinIO recommends version 3 for new deployments.

Version 3 Endpoints

For metrics version 3, all metrics are available under the base /minio/metrics/v3 endpoint. You can scrape the base endpoint to collect all metrics in a single operation, or append an optional path to return a specific category.

For example, the following endpoint returns audit metrics:

http://HOSTNAME:PORT/minio/metrics/v3/audit

Replace HOSTNAME:PORT with the FQDN and port of the MinIO deployment. For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.

By default, MinIO requires authentication to scrape the metrics endpoints. To generate the needed bearer tokens, use mc admin prometheus generate. You can also disable metrics endpoint authentication by setting MINIO_PROMETHEUS_AUTH_TYPE to public.

MinIO provides the following scraping endpoints, relative to the base URL:

For a complete list of metrics for each endpoint, see Available version 3 metrics.

To enable historical data visualization in MinIO Console, set the following environment variables on each node in the MinIO deployment:

• Set MINIO_PROMETHEUS_URL to the URL of the Prometheus service
• Set MINIO_PROMETHEUS_JOB_ID to the unique job ID assigned to the collected metrics

Available version 3 metrics

MinIO publishes a number of metrics for clusters, API requests, buckets, and other aspects of the MinIO service:

• API Metrics
• Audit Metrics
• Cluster Metrics
• Debug Metrics
• ILM Metrics
• Logger webhook Metrics
• Notification Metrics
• Replication Metrics
• Scanner Metrics
• System Metrics

Many metrics include labels identifying the resource which generated that metric and other relevant details.

API metrics

Metrics about requests served by the current node.

/api/requests

/bucket/api

Audit metrics

Metrics about the MinIO audit functionality.

/audit

Cluster metrics

Metrics about an entire MinIO cluster.

/cluster/config

/cluster/erasure-set

/cluster/health

/cluster/iam

/cluster/usage/buckets

/cluster/usage/objects

Debug metrics

Standard Go runtime metrics from the Prometheus Go Client base collector.

ILM metrics

Metrics about the MinIO ILM functionality.

/ilm

Logger webhook metrics

Metrics about MinIO logger webhooks.

/logger/webhook

Notification metrics

Metrics about the MinIO notification functionality.

/notification

Replication metrics

Metrics about MinIO site and bucket replication.

/replication

/bucket/replication

Scanner metrics

Metrics about the MinIO scanner.

/scanner

System metrics

Metrics about the MinIO process and the node.

/system/drive

/system/memory

/system/cpu

/system/network/internode

/system/process

2 - Monitoring and Alerting using Prometheus

• Monitoring with MinIO and Prometheus: Overview
• Monitoring with MinIO and Prometheus: Lab

MinIO publishes cluster, node, bucket, and resource metrics using the Prometheus Data Model. The procedure on this page documents the following:

• Configuring a Prometheus service to scrape and display metrics from a MinIO deployment
• Configuring an Alert Rule on a MinIO Metric to trigger an AlertManager action

These instructions use version 2 metrics. For more about metrics API versions, see Metrics and alerts.

Configure Prometheus to Collect and Alert using MinIO Metrics

1) Generate the Scrape Configuration

Use the mc admin prometheus generate command to generate the scrape configuration for use by Prometheus in making scraping requests:

mc admin prometheus generate ALIAS

global:
   scrape_interval: 60s

scrape_configs:
   - job_name: minio-job
     bearer_token: TOKEN
     metrics_path: /minio/v2/metrics/cluster
     scheme: https
     static_configs:
     - targets: [minio.example.net]

mc admin prometheus generate ALIAS node

global:
   scrape_interval: 60s

scrape_configs:
   - job_name: minio-job-node
     bearer_token: TOKEN
     metrics_path: /minio/v2/metrics/node
     scheme: https
     static_configs:
     - targets: [minio-1.example.net, minio-2.example.net, minio-N.example.net]

mc admin prometheus generate ALIAS bucket

global:
   scrape_interval: 60s

scrape_configs:
   - job_name: minio-job-bucket
     bearer_token: TOKEN
     metrics_path: /minio/v2/metrics/bucket
     scheme: https
     static_configs:
     - targets: [minio.example.net]

mc admin prometheus generate ALIAS resource

global:
   scrape_interval: 60s

scrape_configs:
   - job_name: minio-job-resource
     bearer_token: TOKEN
     metrics_path: /minio/v2/metrics/resource
     scheme: https
     static_configs:
     - targets: [minio.example.net]

• Set an appropriate scrape_interval value to ensure each scraping operation completes before the next one begins. The recommended value is 60 seconds.

  Some deployments require a longer scrape interval due to the number of metrics being scraped. To reduce the load on your MinIO and Prometheus servers, choose the longest interval that meets your monitoring requirements.

• Set the job_name to a value associated to the MinIO deployment.

  Use a unique value to ensure isolation of the deployment metrics from any others collected by that Prometheus service.

• MinIO deployments started with MINIO_PROMETHEUS_AUTH_TYPE set to "public" can omit the bearer_token field.

• Set the scheme to http for MinIO deployments not using TLS.

• Set the targets array with a hostname that resolves to the MinIO deployment.

  This can be any single node, or a load balancer/proxy which handles connections to the MinIO nodes.

  For MinIO Tenants on Kubernetes infrastructure, when using a Prometheus cluster in that same cluster you can specify the service DNS name for the minio service. You can otherwise specify the ingress or load balancer endpoint configured to route connections to and from the MinIO Tenant.

2) Restart Prometheus with the Updated Configuration

Append the desired scrape_configs job generated in the previous step to the configuration file:

global:
   scrape_interval: 60s

scrape_configs:
   - job_name: minio-job
     bearer_token: TOKEN
     metrics_path: /minio/v2/metrics/cluster
     scheme: https
     static_configs:
     - targets: [minio.example.net]

global:
   scrape_interval: 60s

scrape_configs:
   - job_name: minio-job-node
     bearer_token: TOKEN
     metrics_path: /minio/v2/metrics/node
     scheme: https
     static_configs:
     - targets: [minio-1.example.net, minio-2.example.net, minio-N.example.net]

global:
   scrape_interval: 60s

scrape_configs:
   - job_name: minio-job-bucket
     bearer_token: TOKEN
     metrics_path: /minio/v2/metrics/bucket
     scheme: https
     static_configs:
     - targets: [minio.example.net]

global:
   scrape_interval: 60s

scrape_configs:
   - job_name: minio-job-resource
     bearer_token: TOKEN
     metrics_path: /minio/v2/metrics/resource
     scheme: https
     static_configs:
     - targets: [minio.example.net]

Start the Prometheus cluster using the configuration file:

prometheus --config.file=prometheus.yaml

3) Analyze Collected Metrics

Prometheus includes an expression browser. You can execute queries here to analyze the collected metrics.

minio_node_drive_free_bytes{job="minio-job"}[5m]
minio_node_drive_free_inodes{job="minio-job"}[5m]

minio_node_drive_latency_us{job="minio-job"}[5m]

minio_node_drive_offline_total{job="minio-job"}[5m]
minio_node_drive_online_total{job="minio-job"}[5m]

minio_node_drive_total{job="minio-job"}[5m]

minio_node_drive_total_bytes{job="minio-job"}[5m]
minio_node_drive_used_bytes{job="minio-job"}[5m]

minio_node_drive_errors_timeout{job="minio-job"}[5m]
minio_node_drive_errors_availability{job="minio-job"}[5m]

minio_node_drive_io_waiting{job="minio-job"}[5m]

4) Configure an Alert Rule using MinIO Metrics

You must configure Alert Rules on the Prometheus deployment to trigger alerts based on collected MinIO metrics.

The following example alert rule files provide a baseline of alerts for a MinIO deployment. You can modify or otherwise use these examples as guidance in building your own alerts.

groups:
- name: minio-alerts
  rules:
  - alert: NodesOffline
    expr: avg_over_time(minio_cluster_nodes_offline_total{job="minio-job"}[5m]) > 0
    for: 10m
    labels:
      severity: warn
    annotations:
      summary: "Node down in MinIO deployment"
      description: "Node(s) in cluster {{ $labels.instance }} offline for more than 5 minutes"

  - alert: DisksOffline
    expr: avg_over_time(minio_cluster_drive_offline_total{job="minio-job"}[5m]) > 0
    for: 10m
    labels:
      severity: warn
    annotations:
      summary: "Disks down in MinIO deployment"
      description: "Disks(s) in cluster {{ $labels.instance }} offline for more than 5 minutes"

In the Prometheus configuration, specify the path to the alert file in the rule_files key:

rule_files:
- minio-alerting.yml

Once triggered, Prometheus sends the alert to the configured AlertManager service.

Dashboards

MinIO provides Grafana Dashboards to display metrics collected by Prometheus. For more information, see Monitor a MinIO Server with Grafana

3 - Monitoring and Alerting using InfluxDB

MinIO publishes cluster and node metrics using the Prometheus Data Model. InfluxDB supports scraping MinIO metrics data for monitoring and alerting.

The procedure on this page documents the following:

• Configuring an InfluxDB service to scrape and display metrics from a MinIO deployment
• Configuring an Alert on a MinIO metric

For MinIO Deployments on Kubernetes, this procedure assumes all necessary network control components, such as Ingress or Load Balancers, to facilitate access between the MinIO Tenant and the InfluxDB service.

Configure InfluxDB to Collect and Alert using MinIO Metrics

1. Configure Public Access to MinIO Metrics

   Set the MINIO_PROMETHEUS_AUTH_TYPE environment variable to "public" for all nodes in the MinIO deployment. You can then restart the deployment to allow public access to MinIO metrics.

   You can validate the change by attempting to curl the metrics endpoint:

   curl https://HOSTNAME/minio/v2/metrics/cluster

   Replace HOSTNAME with the URL of the load balancer or reverse proxy through which you access the MinIO deployment. You can alternatively specify any single node as HOSTNAME:PORT, specifying the MinIO server API port in addition to the node hostname.

   The response body should include a list of collected MinIO metrics.

2. Log into the InfluxDB UI and Create a Bucket

   Select the Organization under which you want to store MinIO metrics.

   Create a New Bucket in which to store metrics for the MinIO deployment.

3. Create a new Scraping Source

   Create a new InfluxDB Scraper.

   Specify the full URL to the MinIO deployment, including the metrics endpoint:

   https://HOSTNAME/minio/v2/metrics/cluster

   Replace HOSTNAME with the URL of the load balancer or reverse proxy through which you access the MinIO deployment. You can alternatively specify any single node as HOSTNAME:PORT, specifying the MinIO server API port in addition to the node hostname.

4. Validate the Data

   Use the DataExplorer to visualize the collected MinIO data.

   For example, you can set a filter on minio_cluster_capacity_usable_total_bytes and minio_cluster_capacity_usable_free_bytes to compare the total usable against total free space on the MinIO deployment.

5. Configure a Check

   Create a new Check on a MinIO metric.

   The following example check rules provide a baseline of alerts for a MinIO deployment. You can modify or otherwise use these examples for guidance in building your own checks.

   • Create a Threshold Check named MINIO_NODE_DOWN.

     Set the filter for the minio_cluster_nodes_offline_total key.

     Set the Thresholds to WARN when the value is greater than 1

   • Create a Threshold Check named MINIO_QUORUM_WARNING.

     Set the filter for the minio_cluster_drive_offline_total key.

     Set the Thresholds to CRITICAL when the value is one less than your configured Erasure Code Parity setting.

     For example, a deployment using EC:4 should set this value to 3.

   Configure your Notification endpoints and Notification rules such that checks of each type trigger an appropriate response.

4 - Publish Server or Audit Logs to an External Service

MinIO publishes all minio server operations to the system console. Reading these logs depends on how the server process is managed. For example, if the server is managed through a systemd script, you can read the logs using journalctl -u SERVICENAME.service. Replace SERVICENAME with the name of the MinIO service.

MinIO also supports publishing server logs and audit logs to an HTTP webhook.

• Server logs contain the same minio server operations logged to the system console. Server logs support general monitoring and troubleshooting of operations.
• Audit logs are more granular descriptions of each operation on the MinIO deployment. Audit logging supports security standards and regulations which require detailed tracking of operations.

MinIO publishes logs as a JSON document as a PUT request to each configured endpoint. The endpoint server is responsible for processing each JSON document. MinIO requires explicit configuration of each webhook endpoint and does not publish logs to a webhook by default.

Publish Server Logs to HTTP Webhook

You can configure a new HTTP webhook endpoint to which MinIO publishes minio server logs using either environment variables or by setting runtime configuration settings.

   set MINIO_LOGGER_WEBHOOK_ENABLE_<IDENTIFIER>="on"
   set MINIO_LOGGER_WEBHOOK_ENDPOINT_<IDENTIFIER>="https://webhook-1.example.net"
   set MINIO_LOGGER_WEBHOOK_AUTH_TOKEN_<IDENTIFIER>="TOKEN"

   export MINIO_LOGGER_WEBHOOK_ENABLE_<IDENTIFIER>="on"
   export MINIO_LOGGER_WEBHOOK_ENDPOINT_<IDENTIFIER>="https://webhook-1.example.net"
   export MINIO_LOGGER_WEBHOOK_AUTH_TOKEN_<IDENTIFIER>="TOKEN"

set MINIO_LOGGER_WEBHOOK_AUTH_TOKEN_myendpoint="Bearer 1a2b3c4f5e"

export MINIO_LOGGER_WEBHOOK_AUTH_TOKEN_myendpoint="Bearer 1a2b3c4f5e"

set MINIO_LOGGER_WEBHOOK_AUTH_TOKEN_xyz="ServiceXYZ 1a2b3c4f5e"

export MINIO_LOGGER_WEBHOOK_AUTH_TOKEN_xyz="ServiceXYZ 1a2b3c4f5e"

mc admin config set ALIAS/ logger_webhook:IDENTIFIER  \
   endpoint="https://webhook-1.example.net"           \
   auth_token="TOKEN"

Publish Audit Logs to HTTP Webhook

You can configure a new HTTP webhook endpoint to which MinIO publishes audit logs using either environment variables or by setting runtime configuration settings:

set MINIO_AUDIT_WEBHOOK_ENABLE_<IDENTIFIER>="on"
set MINIO_AUDIT_WEBHOOK_ENDPOINT_<IDENTIFIER>="https://webhook-1.example.net"
set MINIO_AUDIT_WEBHOOK_AUTH_TOKEN_<IDENTIFIER>="TOKEN"
set MINIO_AUDIT_WEBHOOK_CLIENT_CERT_<IDENTIFIER>="cert.pem"
set MINIO_AUDIT_WEBHOOK_CLIENT_KEY_<IDENTIFIER>="cert.key"

export MINIO_AUDIT_WEBHOOK_ENABLE_<IDENTIFIER>="on"
export MINIO_AUDIT_WEBHOOK_ENDPOINT_<IDENTIFIER>="https://webhook-1.example.net"
export MINIO_AUDIT_WEBHOOK_AUTH_TOKEN_<IDENTIFIER>="TOKEN"
export MINIO_AUDIT_WEBHOOK_CLIENT_CERT_<IDENTIFIER>="cert.pem"
export MINIO_AUDIT_WEBHOOK_CLIENT_KEY_<IDENTIFIER>="cert.key"

set MINIO_AUDIT_WEBHOOK_AUTH_TOKEN_myendpoint="Bearer 1a2b3c4f5e"

export MINIO_AUDIT_WEBHOOK_AUTH_TOKEN_myendpoint="Bearer 1a2b3c4f5e"

set MINIO_AUDIT_WEBHOOK_AUTH_TOKEN_xyz="ServiceXYZ 1a2b3c4f5e"

export MINIO_AUDIT_WEBHOOK_AUTH_TOKEN_xyz="ServiceXYZ 1a2b3c4f5e"

mc admin config set ALIAS/ audit_webhook:IDENTIFIER  \
   endpoint="https://webhook-1.example.net"          \
   auth_token="TOKEN"                                \
   client_cert="cert.pem"                            \
   client_key="cert.key"

Audit Log Structure

MinIO audit logs resemble the following JSON document:

• The api.timeToFirstByte and api.timeToResponse fields are expressed in nanoseconds.

• For erasure coded setups tags.objectErasureMap provides per-object details on the following:

  • The Server Pool on which the object operation was performed.
  • The erasure set on which the object operation was performed.
  • The list of drives in the erasure set which participated in the object operation.

{
   "version": "1",
   "deploymentid": "8ca2b7ad-20cf-4d07-9efb-28b2f519f4a5",
   "time": "2024-02-29T19:39:25.744431903Z",
   "event": "",
   "trigger": "incoming",
   "api": {
      "name": "CompleteMultipartUpload",
      "bucket": "data",
      "object": "test-data.csv",
      "status": "OK",
      "statusCode": 200,
      "rx": 267,
      "tx": 358,
      "txHeaders": 387,
      "timeToFirstByte": "2096989ns",
      "timeToFirstByteInNS": "2096989",
      "timeToResponse": "2111986ns",
      "timeToResponseInNS": "2111986"
   },
   "remotehost": "127.0.0.1",
   "requestID": "17B86CB0ED88EBE9",
   "userAgent": "MinIO (linux; amd64) minio-go/v7.0.67 mc/RELEASE.2024-02-24T01-33-20Z",
   "requestPath": "/data/test-data.csv",
   "requestHost": "minio.example.net:9000",
   "requestQuery": {
      "uploadId": "OGNhMmI3YWQtMjBjZi00ZDA3LTllZmItMjhiMmY1MTlmNGE1LmU3MjNlNWI4LTNiYWYtNDYyNy1hNzI3LWMyNDE3NTVjMmMzNw"
   },
   "requestHeader": {
      "Accept-Encoding": "zstd,gzip",
      "Authorization": "AWS4-HMAC-SHA256 Credential=minioadmin/20240229/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=ccb3acdc1763509a88a7e4a3d7fe431ef0ee5ca3f66ccb430d5a09326e87e893",
      "Content-Length": "267",
      "Content-Type": "application/octet-stream",
      "User-Agent": "MinIO (linux; amd64) minio-go/v7.0.67 mc/RELEASE.2024-02-24T01-33-20Z",
      "X-Amz-Content-Sha256": "d61969719ee94f43c4e87044229b7a13b54cab320131e9a77259ad0c9344f6d3",
      "X-Amz-Date": "20240229T193925Z"
   },
   "responseHeader": {
      "Accept-Ranges": "bytes",
      "Content-Length": "358",
      "Content-Type": "application/xml",
      "ETag": "1d9fdc88af5e74f5eac0a3dd750ce58e-2",
      "Server": "MinIO",
      "Strict-Transport-Security": "max-age=31536000; includeSubDomains",
      "Vary": "Origin,Accept-Encoding",
      "X-Amz-Id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
      "X-Amz-Request-Id": "17B86CB0ED88EBE9",
      "X-Content-Type-Options": "nosniff",
      "X-Xss-Protection": "1; mode=block"
   },
   "tags": {
      "objectLocation": {
            "name": "Mousepad Template-v03final.jpg",
            "poolId": 1,
            "setId": 1,
            "disks": [
               "/mnt/drive-1",
               "/mnt/drive-2",
               "/mnt/drive-3",
               "/mnt/drive-4"
            ]
      }
   },
   "accessKey": "minioadmin"
}

5 - Healthcheck API

MinIO exposes unauthenticated endpoints for probing node uptime and cluster high availability for simple healthchecks. These endpoints return an HTTP status code indicating whether the underlying resource is healthy or satisfies read/write quorum. MinIO exposes no other data through these endpoints.

Node Liveness

Use the following endpoint to test if a MinIO server is online:

curl -I https://minio.example.net:9000/minio/health/live

Replace https://minio.example.net:9000 with the DNS hostname of the MinIO server to check.

A response code of 200 OK indicates the MinIO server is online and functional. Any other HTTP codes indicate an issue with reaching the server, such as a transient network issue or potential downtime.

The healthcheck probe alone cannot determine if a MinIO server is offline. Instead, the probe determines whether the current host machine can reach the server. Consider configuring a Prometheus alert using minio_cluster_health_nodes_offline_count for metrics v3 or minio_cluster_nodes_offline_total for metrics v2 to detect whether one or more MinIO nodes are offline.

Cluster Write Quorum

Use the following endpoint to test if a MinIO cluster has write quorum:

curl -I https://minio.example.net:9000/minio/health/cluster

Replace https://minio.example.net:9000 with the DNS hostname of a node in the MinIO cluster to check. For clusters using a load balancer to manage incoming connections, specify the hostname for the load balancer.

A response code of 200 OK indicates that the MinIO cluster has sufficient MinIO servers online to meet write quorum. A response code of 503 Service Unavailable indicates the cluster does not currently have write quorum.

The healthcheck probe alone cannot determine if a MinIO server is offline or processing write operations normally - only whether enough MinIO servers are online to meet write quorum requirements based on the configured erasure code parity. Consider configuring a Prometheus alert using one of the following metrics to detect potential issues or errors on the MinIO cluster:

• minio_cluster_nodes_offline_total to alert if one or more MinIO nodes are offline.
• minio_node_drive_free_bytes to alert if the cluster is running low on free drive space.

Cluster Read Quorum

Use the following endpoint to test if a MinIO cluster has read quorum:

curl -I https://minio.example.net:9000/minio/health/cluster/read

Replace https://minio.example.net:9000 with the DNS hostname of a node in the MinIO cluster to check. For clusters using a load balancer to manage incoming connections, specify the hostname for the load balancer.

A response code of 200 OK indicates that the MinIO cluster has sufficient MinIO servers online to meet read quorum. A response code of 503 Service Unavailable indicates the cluster does not currently have read quorum.

The healthcheck probe alone cannot determine if a MinIO server is offline or processing read operations normally - only whether enough MinIO servers are online to meet read quorum requirements based on the configured erasure code parity. Consider configuring a Prometheus alert using the minio_cluster_nodes_offline_total metric to detect whether one or more MinIO nodes are offline.

Cluster Maintenance Check

Use the following endpoint to test if the MinIO cluster can maintain both read and write if the specified MinIO server is taken down for maintenance:

curl -I https://minio.example.net:9000/minio/health/cluster?maintenance=true

Replace https://minio.example.net:9000 with the DNS hostname of a node in the MinIO cluster to check. For clusters using a load balancer to manage incoming connections, specify the hostname for the load balancer.

A response code of 200 OK indicates that the MinIO cluster has sufficient MinIO servers online to meet write quorum. A response code of 412 Precondition Failed indicates the cluster will lose quorum if the MinIO server goes offline.

The healthcheck probe alone cannot determine if a MinIO server is offline - only whether enough MinIO servers will be online after taking the node down for maintenance to meet read and write quorum requirements based on the configured erasure code parity. Consider configuring a Prometheus alert using the minio_cluster_nodes_offline_total metric to detect whether one or more MinIO nodes are offline.

6 - Metrics version 2

MinIO publishes cluster and node metrics using the Prometheus Data Model. You can use any scraping tool to pull metrics data from MinIO for further analysis and alerting.

Version 2 Endpoints

Metrics version 2 provides metrics organized into three categories:

• Cluster Metrics
• Bucket Metrics
• Resource Metrics

Each v2 endpoint returns all metrics for its category. For example, scraping the following endpoint returns all cluster metrics:

http://HOSTNAME:PORT/minio/v2/metrics/cluster

The base endpoint alone, /minio/v2/metrics/, returns cluster metrics.

For more flexible scraping and a wider range of metrics, use metrics version 3.

Existing deployments can continue to use version 2 metrics and Grafana dashboards.

MinIO Grafana dashboard

MinIO publishes two Grafana Dashboards for visualizing v2 metrics. For more complete documentation on configuring a Prometheus-compatible data source for Grafana, see the Prometheus documentation on Grafana Support.

Available version 2 metrics

The following sections describe the version 2 endpoints and metrics.

http://HOSTNAME:PORT/minio/v2/metrics/cluster

http://HOSTNAME:PORT/minio/v2/metrics/bucket

http://HOSTNAME:PORT/minio/v2/metrics/resource

• Cluster Metrics
• Bucket Metrics
• Resource Metrics

Cluster Metrics

MinIO collects the following metrics at the cluster level. Metrics may include one or more labels, such as the server that calculated that metric.

These metrics can be obtained from any MinIO server once per collection by using the following URL:

https://HOSTNAME:PORT/minio/v2/metrics/cluster

Replace HOSTNAME:PORT with the hostname of your MinIO deployment. For deployments behind a load balancer, use the load balancer hostname instead of a single node hostname.

Audit Metrics

Cluster Capacity Metrics

Cluster Drive Metrics

Cluster ILM Metrics

Cluster KMS Metrics

Cluster Health Metrics

Cluster Replication Metrics

Metrics marked as Site Replication Only only populate on deployments with Site Replication configurations. For deployments with bucket or batch replication configurations, these metrics populate instead under the Bucket Metrics endpoint.

Node Replication Metrics

Metrics marked as Site Replication Only only populate on deployments with Site Replication configurations. For deployments with bucket or batch replication configurations, these metrics populate instead under the Bucket Metrics endpoint.

Healing Metrics

Inter Node Metrics

Bucket Notification Metrics

S3 API Request Metrics

Software Metrics

Drive Metrics

Identity and Access Management (IAM) Metrics

Information Lifecycle Management (ILM) Metrics

Tier Metrics

System Metrics

Scanner Metrics

Note

Changed: RELEASE.2025-03-12T17-29-24Z

v2 metrics have a limit of 100 buckets for performance reasons. For metrics across a higher number of buckets, use v3 metrics instead.

Bucket Metrics

MinIO collects the following metrics at the bucket level. Each metric includes the bucket label to identify the corresponding bucket. Metrics may include one or more additional labels, such as the server that calculated that metric.

These metrics can be obtained from any MinIO server once per collection by using the following URL:

https://HOSTNAME:PORT/minio/v2/metrics/bucket

Replace HOSTNAME:PORT with the hostname of your MinIO deployment. For deployments behind a load balancer, use the load balancer hostname instead of a single node hostname.

Distribution Metrics

Replication Metrics

These metrics only populate on deployments with Bucket Replication or Batch Replication configurations. For deployments with Site Replication configured, select metrics populate under the Cluster Metrics endpoint.

Traffic Metrics

Usage Metrics

Requests Metrics

Resource Metrics

MinIO collects the following resource metrics at the node level. Each metric includes the server label to identify the corresponding node. Metrics may include one or more additional labels, such as the drive path, interface name, etc.

These metrics can be obtained from any MinIO server once per collection by using the following URL:

https://HOSTNAME:PORT/minio/v2/metrics/resource

Replace HOSTNAME:PORT with the hostname of your MinIO deployment. For deployments behind a load balancer, use the load balancer hostname instead of a single node hostname.

Drive Resource Metrics

Network Interface Metrics

CPU Metrics

Memory Metrics

7 - Monitor a Silo Server with Grafana

Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored.

Prerequisites

• An existing Prometheus deployment with backing Alert Manager
• An existing MinIO deployment with network access to the Prometheus deployment
• Grafana installed

MinIO Grafana Dashboard

MinIO provides several official Grafana Dashboards you can download from the Grafana Dashboard portal.

1. MinIO Server metrics
2. MinIO Bucket metrics
3. MinIO Replication metrics

To track changes to the Grafana dashboard, inspect the JSON files for the server or bucket dashboards in the MinIO Server GitHub repository.

MinIO Server Metrics Dashboard

Browse the maintained MinIO dashboards in the MinIO organization catalog on Grafana, then select a server dashboard compatible with the metrics version exposed by your deployment.

MinIO provides a Grafana Dashboard for MinIO Server metrics. For specifics on the dashboard’s configuration, see the JSON file on GitHub.

For MinIO Deployments running with Server-Side Encryption (SSE-KMS or SSE-S3), the dashboard includes metrics for the KMS. These metrics include status, request error rates, and request success rates.

MinIO Bucket Metrics Dashboard

Use the MinIO organization catalog on Grafana to select a bucket dashboard compatible with the metrics version exposed by your deployment.

Bucket metrics can be viewed in the Grafana dashboard using the bucket JSON file on GitHub.

MinIO Node Metrics Dashboard

Node metrics can be viewed in the Grafana dashboard using the node JSON file on GitHub.

MinIO Replication Metrics Dashboard

Use the MinIO organization catalog on Grafana to select a replication dashboard compatible with the metrics version exposed by your deployment.

Cluster replication metrics can be viewed in the Grafana dashboard using the cluster replication JSON file on GitHub.