Deployment Examples

Examples for deploying Datafy in your environment using common deployment tools

Datafy can be installed on EC2 instances with a single shell command, and on Kubernetes clusters with a Helm chart. In environments with many instances or clusters, we recommend deploying Datafy Sensor and AutoScaler with the same tools you already use to deploy packages in your environment.

This page provides examples for integrating Datafy installation into common deployment and management tools. Each example assumes you're already using the tool - refer to its own documentation for initial setup and general usage.

For the standard installation steps and parameters, see the installation guide.

The examples on this page use the following parameters:

• $TOKEN - Your Datafy security token, generated from the Datafy dashboard.

• $VERSION - The Datafy agent version to install (used in EC2 examples).

• $HELM_VERSION - The Datafy Helm chart version to deploy (used in Kubernetes examples).

This page contains examples for deploying Datafy using different tools - they do not cover all of the ways to deploy Datafy, with these tools or otherwise. For the best outcome, follow the standards and best practices used in your environment.

EC2 Instances

On EC2, Datafy is installed by running a shell command on each instance. The tools below let you run that command across many instances at once.

Docker must be installed on each target instance before installing Datafy. See the EC2 installation prerequisites for details.

Ansible

Ansible is an agentless automation tool that connects to your instances over SSH and runs tasks defined in a playbook. You can use it to install Datafy across a group of EC2 instances in a single run.

Follow the steps below to create and run a playbook that will install Datafy on the desired hosts.

1

Create Playbook

Create a new playbook called install_datafy.yaml, and add it to your ansible project. The following example playbook downloads the Datafy install script once on your Ansible control machine, copies it to each target instance, and runs it.

Install Datafy Playbook

Create an install_datafy.yaml file:

---
- name: Install Datafy Agent
  # Set the hosts field to target the instances you want to install Datafy on
  hosts: all 
  become: yes

  vars:
    # Replace $VERSION and $TOKEN with your values.
    # To install Sensor instead of AutoScaler, change agent_mode to "sensor".
    version: "$VERSION" 
    datafy_token: "$TOKEN"
    agent_mode: "autoscaler"

  tasks:
    - name: Download Datafy install script to control machine
      ansible.builtin.get_url:
        url: "https://agent.datafy.io/install?version={{ version }}"
        dest: "/tmp/install_datafy_agent.sh"
        mode: '0755'
      delegate_to: localhost
      run_once: true
      become: no

    - name: Copy install script to target instance
      ansible.builtin.copy:
        src: /tmp/install_datafy_agent.sh
        dest: /tmp/install_datafy_agent.sh
        mode: '0755'

    - name: Run Datafy install script
      ansible.builtin.shell: /tmp/install_datafy_agent.sh
      environment:
        TOKEN: "{{ datafy_token }}"
        AGENT_MODE: "{{ agent_mode }}"

2

Run the Playbook

Run the playbook against your inventory, with:

ansible-playbook install_datafy.yaml

Ansible will connect to each instance, copy the install script, and run it. You'll see output for each task on each host, with a summary at the end showing how many succeeded.

AWS Systems Manager (SSM) Run Command

AWS Systems Manager Run Command lets you run shell commands on your EC2 instances remotely, without needing SSH access. It uses the SSM Agent, which comes pre-installed on most Amazon Linux and Ubuntu AMIs.

This is a good option when your instances are already managed through Systems Manager, or when you want to run the install across instances without setting up SSH keys.

Prerequisites

• The SSM Agent must be installed and running on each target instance. Most Amazon-provided AMIs include it by default. See AWS documentation on SSM Agent for details.

• The instances must have an IAM instance profile with the AmazonSSMManagedInstanceCore policy (or equivalent permissions) to communicate with Systems Manager.

• Docker installed on each target instance.

Installation

You can set up and run the shell command from the AWS console, or through the AWS CLI.

AWS Console

1. Open the Run Command page in the AWS Systems Manager console.

2. In Command document, search for AWS-RunShellScript and select it, then select Run Command.

3. In Command parameters:

   1. Define the $VERSION and $TOKEN variables

   2. Enter the Datafy install command:

      curl -sSfL https://agent.datafy.io/install?version="$VERSION" | AGENT_MODE="autoscaler" TOKEN="$TOKEN" sh

4. In Target selection, choose the instances to install on. You can select targets by:

   • Tags - for example, all instances with a specific Environment or Role tag.

   • Manual selection - pick individual instances from the list.

   • Resource group - target all instances in an AWS resource group.

5. Optionally, in Output options, configure output logging to an S3 bucket or CloudWatch Logs. This is helpful for reviewing results across many instances.

6. Click Run to execute the command.

AWS CLI

Verifying the results

After running the command, you can check the results in the Systems Manager console:

• Running commands - view commands currently in progress under Executing Commands.

• Completed commands - view results under Command History. Select a command to see the overall status, and drill into individual instances to view their output and any errors.

Kubernetes

On Kubernetes, Datafy is installed via its Helm chart. GitOps tools can manage this Helm release declaratively, keeping your cluster configuration in sync with your desired state.

ArgoCD

ArgoCD is a declarative GitOps continuous delivery tool for Kubernetes. It monitors a source (such as a Helm chart repository) and automatically syncs your cluster to match the desired state. You can use it to deploy and manage the Datafy Helm chart.

Prerequisites

• ArgoCD installed and running on your cluster. See the ArgoCD getting started guide.

• Access to create ArgoCD Applications (via the UI or kubectl).

Deploy Datafy with ArgoCD

Create an ArgoCD Application that points to the Datafy Helm chart. You can use the ArgoCD UI or CLI to create an application based on the following manifest:

Datafy Application manifest

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: datafy
  namespace: argocd
spec:
  project: default
  destination:
    namespace: datafy-agent
    server: https://kubernetes.default.svc
  source:
    repoURL: https://helm.datafy.io/datafy-agent
    chart: datafy-agent
    targetRevision: $HELM_VERSION # define variable or replace with correct version
    helm:
      parameters:
        - name: agent.token
          value: $TOKEN # define variable or replace with correct value
        - name: agent.mode
          value: "autoscaler" #To install Sensor instead of AutoScaler, change to "sensor".
  syncPolicy:
    syncOptions:
      # Create the "datafy-agent" namespace if it doesn't already exist
      - CreateNamespace=true
    automated:
      # Don't automatically delete resources removed from the chart.
      # Adjust based on your team's GitOps practices.
      prune: false
      # Don't automatically revert manual changes made to resources.
      # Adjust based on your team's GitOps practices.
      selfHeal: false

ArgoCD UI

1. Open the ArgoCD dashboard and click New App.

2. Click Edit as YAML and paste the above manifest.

3. Click Save, then Create.

CLI

Once created, ArgoCD will sync the application and deploy Datafy to your cluster. You can monitor the sync status in the ArgoCD dashboard.