Datafy Metrics Exporter

The Datafy Metrics Exporter exposes supplementary metrics about the Datafy agent's health and your autoscaling volumes, at two endpoints:

• /metrics - exposes metrics about the size and usage of autoscaling volumes managed by Datafy.

• /health - returns a health indication for the Datafy agent installed on the instance.

Installing Datafy Metrics Exporter

Datafy Metrics Exporter can be installed with a single command or as a Docker container.

The metrics are exposed by default on port 9908, and a custom port can be configured as desired.

One-line install

To install Datafy Metrics Exporter, run the command:

curl -sSfL https://bos.datafy.io/install_exporter | sh

To configure a custom port, install with the flag PORT. For example:

curl -sSfL https://bos.datafy.io/install_exporter | sh PORT=9100 sh

The exporter can be uninstalled in the same manner, using the command:

curl -sSfL https://bos.datafy.io/uninstall_exporter | sh

Docker

To install Datafy Metrics Exporter using a Docker image, use the following installation command:

docker run -d \
  --name datafy_exporter \
  --restart unless-stopped \
  --pid=host --privileged \
  -v /:/host:ro,rslave \
  -v /var/lib/datafy/pipes/geoms.state:/var/lib/datafy/pipes/geoms.state \
  -p 9908:9908 \
  public.ecr.aws/datafy-io/datafy-exporter:latest

To configure a custom port, install with the flag -port. For example:

docker run -d \
  --name datafy_exporter \
  --restart unless-stopped \
  --pid=host --privileged \
  -v /:/host:ro,rslave \
  -v /var/lib/datafy/pipes/geoms.state:/var/lib/datafy/pipes/geoms.state \
  -p 9202:9202 \
  public.ecr.aws/datafy-io/datafy-exporter:latest \
  -port=9202

Datafy Exporter Metrics

Viewing Datafy Metrics

Once the Datafy Metrics Exporter is installed and running, you can verify that metrics are being exported, and view help and type information by accessing the /metrics endpoint:

curl http://localhost:9908/metrics

Make sure to use the port specified in the installation

You should see output like this:

# HELP disk_total_bytes Total disk space in bytes
# TYPE disk_total_bytes gauge
# HELP disk_used_bytes Used disk space in bytes
# TYPE disk_used_bytes gauge
# HELP disk_available_bytes Available disk space in bytes
# TYPE disk_available_bytes gauge
# HELP disk_usage_percent Disk usage percent
# TYPE disk_usage_percent gauge
# HELP service_up Service liveness status (1 = up, 0 = down)
# TYPE service_up gauge
disk_total_bytes{mount="/mnt/data",fs="/dev/nvme1n1",vol="vol-00d9752e77b9ff5be"} 5.7982058496e+10
disk_used_bytes{mount="/mnt/data",fs="/dev/nvme1n1",vol="vol-00d9752e77b9ff5be"} 45056
disk_available_bytes{mount="/mnt/data",fs="/dev/nvme1n1",vol="vol-00d9752e77b9ff5be"} 5.798201344e+10
disk_usage_percent{mount="/mnt/data",fs="/dev/nvme1n1",vol="vol-00d9752e77b9ff5be"} 7.770679615162037e-05
disk_total_bytes{fs="/dev/nvme2n1",vol="vol-05c2dd6f056df3fc8",mount="/mnt/data2"} 2.10241560576e+11
disk_used_bytes{fs="/dev/nvme2n1",vol="vol-05c2dd6f056df3fc8",mount="/mnt/data2"} 36864
disk_available_bytes{mount="/mnt/data2",fs="/dev/nvme2n1",vol="vol-05c2dd6f056df3fc8"} 1.99487328256e+11
disk_usage_percent{fs="/dev/nvme2n1",vol="vol-05c2dd6f056df3fc8",mount="/mnt/data2"} 1
service_up{service="datafy-agent"} 1
service_up{service="datafy-agent-core"} 1

Success! The Datafy Metrics Exporter is now exposing metrics that a service like Prometheus can scrape.

Using Datafy Metrics

The Datafy Metrics Exporter exposes size and usage metrics for each volume on the instance. For autoscaling volumes, the actual current size of the autoscaling volume is exposed.

For out of space monitoring of autoscaling volumes use the utilization reported by the filesystem, do not use the metrics from Datafy Metrics Exporter.

Other metrics exporters (such as Prometheus Node Exporter) will continue to correctly report read, write and IOPS rates for autoscaling volumes. However, autoscaling volumes will have device names in the format dvolXXX.... This name corresponds to the vol field in the Datafy metrics, which you can use to enrich other metrics with the relevant mount and filesystem paths.