Integrations ยป Monitoring
Datadog
You can add operational visibility to your Rack with Datadog.
Configure kubectl to Point at Your Rack
Convox allows you to securely connect your kubectl
to your Convox created Kubernetes cluster by exporting a kubeconfig that will connect you to a Kubernetes API Proxy running inside your Rack. This allows you to use kubectl
without directly exposing the credentials for your Kubernetes cluster. For example if your Rack is called myrack
you could point your local kubectl
to your Rack cluster as follows
$ convox rack kubeconfig > /tmp/myrack-config
$ export KUBECONFIG=/tmp/myrack-config
This will export the proxy configuration to a temporary file and then point your local kubectl
environment at that location so you can connect to your Rack’s cluster. You will need to perform this step before you can execute any kubectl
commands against your cluster.
Deploy the Datadog Agent
Once you have kubectl
pointing at your Rack you can deploy the datadog agent as a Kubernetes Daemonset. The following is based on the Datadog Documentation so please refer back there for any specific tweaks you may want to make.
Configure Agent Permissions
The following commands will create the necessary roles in your cluster for the Datadog Agent to monitor your cluster and your Apps
$ kubectl apply -f "https://raw.githubusercontent.com/DataDog/datadog-agent/master/Dockerfiles/manifests/rbac/clusterrole.yaml"
$ kubectl apply -f "https://raw.githubusercontent.com/DataDog/datadog-agent/master/Dockerfiles/manifests/rbac/serviceaccount.yaml"
$ kubectl apply -f "https://raw.githubusercontent.com/DataDog/datadog-agent/master/Dockerfiles/manifests/rbac/clusterrolebinding.yaml"
Store Your API Key as a Kubernetes Secret
Retrieve your API Key from the Datadog console and store it as a secret in your cluster by replacing <DATADOG_API_KEY>
with your API key and running the following command.
$ kubectl create secret generic datadog-secret --from-literal api-key="<DATADOG_API_KEY>" --namespace="default"
You will then need to grab the Base64 encoded version of your API key to put in the Datadog Agent configuration file. You can retrieve the Base64 encoded value by running
$ kubectl get secret datadog-secret -o yaml
and grabbing the value for api-key
$ kubectl get secret datadog-secret -o yaml
apiVersion: v1
data:
api-key: [BASE64 Encoded Key]
kind: Secret
metadata:
creationTimestamp: "2020-07-27T20:42:29Z"
name: datadog-secret
namespace: default
resourceVersion: "8898"
selfLink: /api/v1/namespaces/default/secrets/datadog-secret
uid: XXXXX-XXXXXX-XXXXX-XXXXXX
type: Opaque
Create a Datadog Agent Manifest
Datadog has several example manifests - Manifest with Logs, APM, process, metrics collection enabled. - Manifest with Logs, APM, and metrics collection enabled. - Manifest with Logs and metrics collection enabled. - Manifest with APM and metrics collection enabled. - Manifest with Network Performance Monitoring enabled - Vanilla manifest with just metrics collection enabled.
Whichever one of these examples you use as your starting point, you will need to insert the Base64 encoded key from the previous step in the section at the top where it says:
# Source: datadog/templates/secrets.yaml
# API Key
apiVersion: v1
kind: Secret
metadata:
name: datadog-agent
labels: {}
type: Opaque
data:
api-key: PUT_YOUR_BASE64_ENCODED_API_KEY_HERE
We also recommend adding the following to the environment section of the Daemonset spec:
- name: DD_CONTAINER_EXCLUDE
value: "name:datadog-agent"
to remove extra noise from the Datadog Agent itself.
Create the Daemonset
If you save your customized Agent Manifest as a file called datadog-agent.yaml
you can then create the daemonset by running
$ kubectl apply -f datadog-agent.yaml
You can verify the daemonset by running
$ kubectl get daemonset
Once your DESIRED
CURRENT
and READY
counts are all equal your Agents should be up and running. To make any changes to your Agent configuration simply modify your manifest and repeat the steps above.
For further customization and troubleshooting please refer to the Datadog Docs
Metrics and Traces
In order to use Datadog’s APM, Distributed Tracing, or Runtime Metrics you will need to connect to the Datadog agent.
The agent configuration above will be listening to 8125/udp
and 8126/tcp
on the instance
IP address. This IP address will be available to your Processes
in the INSTANCE_IP
environment variable.