Skip to content

Argo CD

Learn how and why deployKF uses Argo CD. Learn how to use your existing Argo CD with deployKF and Kubeflow.


What is Argo CD?

Argo CD is an extremely widely-used tool that helps you programmatically manage the applications deployed on a Kubernetes cluster.

Please note that Argo CD is a completely different tool from Argo Workflows, they just have similar names.

Argo CD Manages the state of Kubernetes resources.
Argo Workflows Runs DAG workflows in Pods on Kubernetes.
(Used by Kubeflow Pipelines)

Argo CD Applications

The main config for Argo CD is the Application, a Kubernetes custom resource that specifies Kubernetes manifests for Argo CD to deploy and manage (typically from a git repository).

An "app of apps" is a pattern where a single Argo CD Application contains other Application definitions, this is typically done to make bootstrapping large applications easier.

Argo CD Interfaces

The Argo CD Web Interface lets you visually manage your cluster, allowing you to view and make changes to Kubernetes resources (including Applications). While the CLI and REST API allow you to manage everything programmatically.

Argo CD Web UI (Dark Mode) Argo CD Web UI (Light Mode)

The ArgoCD Web UI allows you to view and manage your cluster.


How does deployKF use Argo CD?

Argo CD is a core part of deployKF, it helps us manage the lifecycle and state of all components in the platform. For example, we use Argo CD for:

  • Applying manifests to the cluster, in a defined order
  • Detecting when applications are out-of-sync
  • Syncing applications to their desired state
  • Pruning old manifests which are no longer needed
  • Programmatically managing all of the above

What is the deployKF ArgoCD Plugin?

The deployKF ArgoCD Plugin is an optional part of deployKF which removes the need to commit manifests to a Git repository. The plugin adds a special kind of Argo CD Application that produces deployKF manifests internally, similar to how Helm charts are used in Argo CD.

With the plugin, you manage the whole platform from a single "app of apps" Application whose specification only needs your values, and a specified source version of deployKF. For an example of this, see this section of the local quickstart.

Can I use <other tool> instead of Argo CD?

No, not yet. While we believe that Argo CD is currently the best in its category, we recognize that it's not the only option. In the future, we may support other Kubernetes GitOps tools (like Flux CD), or even build a deployKF-specific solution.

We think deployKF is good enough to try, even if you don't love Argo CD!

Info

To learn more about this decision, and participate in the discussion, see deployKF/deployKF#110.


Can I use my existing Argo CD?

Yes, you must.

See our version matrix for a list of supported Argo CD versions, then follow the Getting Started guide to install deployKF with your existing Argo CD.

Can I use an off-cluster ArgoCD?

Yes. deployKF supports the Argo CD "management cluster" pattern, where multiple target clusters are managed by a single Argo CD.

Step 1 - Configure destination and appNamePrefix

When using an off-cluster ArgoCD, you must set the argocd.destination value to target the correct cluster.

You must also set the argocd.appNamePrefix value to avoid conflicting ArgoCD application names (which is needed because multiple sets of them may exist in the management cluster).

For example, say you have defined a remote cluster named "my-cluster1" on your Argo CD management cluster. The following values will prefix all application names with "cluster1-" and target them to the named destination "my-cluster1":

argocd:
  ## a prefix to use for argocd application names
  appNamePrefix: "cluster1-"

  ## the destination used for deployKF argocd applications
  destination:
    name: "my-cluster1"

Destination MUST be remote

When the argocd.appNamePrefix value is non-empty, the argocd.destination MUST be a remote cluster (that is, you should not run deployKF on your management cluster).

Step 2 - Update App-of-Apps

Your app-of-apps Application MUST target the management cluster, NOT the remote cluster, only the internal Applications will target the remote cluster.

Also, you must set the app.kubernetes.io/part-of label to {argocd.appNamePrefix}deploykf, so the sync script works correctly.

For example, your app-of-apps Application might look like this:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: {argocd.appNamePrefix}deploykf-app-of-apps
  namespace: argocd
  labels:
    app.kubernetes.io/name: deploykf-app-of-apps
    app.kubernetes.io/part-of: {argocd.appNamePrefix}deploykf
spec:
  ## NOTE: This project ONLY applies to the app-of-apps itself, not the internal Applications.
  ##       It needs to create Applications in the management cluster and Namespaces in the target.
  ##       The project used by internal Applications is set by the `argocd.project` value.
  project: default

  source:
    ...
    ...
    ...

  destination:
    ## OPTION 1: target the management cluster with `server`
    server: "https://kubernetes.default.svc"

    ## OPTION 2: target the management cluster with `name`
    #name: "in-cluster"
Step 3 - Update Sync Script

By default, the sync_argocd_apps.sh script assumes that argocd.appNamePrefix is not set.

Update the ARGOCD_APP_NAME_PREFIX variable at the top of the script to match your argocd.appNamePrefix value.

ARGOCD_APP_NAME_PREFIX="cluster1-"

TIP: make sure you have your kubectl context set to the management cluster (NOT your target cluster), before running the sync script.


Last update: 2024-05-09
Created: 2024-05-09