Getting Started¶

This tutorial will bring up three gitops-automated clusters running in the following domains visible on the internet:

karrots-helloworld.[your-domain].com
dev.[your-domain].com
dev-feature-1234.[your-domain].com

These clusters are functionally equivalent, fully gitops-automated and maintained in a single git repo. You will see that pushing changes to that repo automatically updates running clusters, or through git branching, allows you to quickly bring up new clusters that are functionally equivalent to production. Moreover, you could run them in separate regions, AZs or even hosting providers!

If you Already Use FluxCD¶

Karrots builds clusters that run with gitops-automation provided by FluxCD. If you don't already use FluxCD then if you follow this tutorial as-is then you will become a FluxCD user.

If you already use FluxCD there is no need to worry that you will damage your current setup. Karrots creates new clusters using a new FluxCD repo, so the work below will have no impact on your current clusters or FluxCD setup. See our FluxCD Migration Guide when you complete this tutorial.

Install Karrots Binary¶

Linux (Debian)

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B4D285C0003B4D71
sudo add-apt-repository "deb https://zero-diff.github.io/karrots/debian-repo/ karrots-github main"
sudo apt-get install karrots
karrots --help

MacOS (Brew)

brew tap zero-diff/karrots
brew install karrots
karrots --help

Cluster Setup (Karrots Seed)¶

We need to change to the parent directory where we want to create our FluxCD gitops repo.

cd ~/git

From this repo directory, execute the seed command:

karrots seed --directory=karrots-cluster

This will create a new directory ~git/karrots-cluster and populate it using the Karrots-Seed github repo as a template. It will prompt you to specify a git origin/main push configuration such as ssh:git@github.com:<your-team>/karrots-cluster.git. Once complete, the directory will have the following structure:

ls
clusters/
  all/
    deployments/
    charts/
    helm-releases/
  karrots-helloworld/
    deployments/
    charts/
    helm-releases/
    karrots.yaml
docs/
README.md

Once seeded, your FluxCD repo will contain two important directories that will drive cluster creation and automation.

clusters/all:

This directory contains the elements that FluxCD applies to all clusters. The charts and helm-releases directories come populated with everything needed to bring up the base cluster services listed in: Base Services Inventory

clusters/karrots-helloworld:

This directory contains the elements that FluxCD applies to the karrots-helloworld cluster. It also contains the starting karrots.yaml file which you need to edit with the values that apply to where you want to run the cluster.

cd ~/git/fluxcd/clusters/karrots-helloworld
vi karrots.yaml

kubernetes:
  provider: # eks, gke, azure
  region: # us-east-1, us-west-2, etc
  cluster: karrots-helloworld
  domain: # zerodiff.org

Cluster Bring-up (Karrots Create Cluster)¶

Once you have saved your values, execute the following command from the karrots-helloworld directory:

cd ~/git/karrots-cluster/clusters/karrots-helloworld
karrots create-cluster

Behind the scenes, Karrots does the following:

Create a build directory
Populate build with Terraform scripts customized to build this cluster
Run terraform init in the build directory to setup the workspace
Run terraform validate in the build directory to verify the setup
Add, commit and push all changes from the local repo to the central repo seen by FluxCD in the new cluster
Run terraform apply in the build directory to build the cluster
Test that the cluster has come up

In about 20-30 minutes you should have a fully operational Kubernetes cluster with gitops-automation. The cluster will run FluxCD which will bring up the Karrots base services (Ambassador, Istio, etc) and the Karrots Test app. If you visit https://karrots-helloworld.[your-domain].com. You should see the message:

Welcome to Karrots-HelloWorld version 0.1.0

Push a Change to your New Karrots-HelloWorld Cluster¶

At this point your karrots-helloworld cluster is running with FluxCD gitops automation. In the cluster, FluxCD periodically scans the central git repo, e.g. at Github, to determine if there are differences between the repo and the running cluster. If it detects differences, for example the cluster is one git commit behind the repo, FluxCD will apply those changes to the cluster. In this example step we will commit a simple change and push it to the remote repo to see that the cluster detects the change and updates itself shortly after.

To update the Karrots deployment running in the cluster, you change it's helm-release and then push the change to the remote repo:

cd ~/git/karrots-cluster/clusters/karrots-helloworld/releases/karrots-helloworld
vi karrots-helloworld.yaml

In the karrots-helloworld.yaml we will change the Karrots Test release container from v1.0.0 to v1.0.1 by setting image:tag to 1.0.1 . See the note near the bottom of the YAML file.

---
apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
  name: karrots-helloworld
  namespace: karrots-helloworld
  labels:
    revision: "1"
  annotations:
    fluxcd.io/automated: "true"
spec:
  releaseName: karrots-helloworld
  chart:
    git: git@github.com:zero-diff/karrots-helloworld
    path: clusters/karrots-helloworld/charts/karrots-helloworld
    version: 1.0.0
  values:
    serviceAccount:
      create: false
    springProfile: dev
    image:
      repository: us.gcr.io/zero-diff-karrots/karrots-helloworld
      tag: 1.0.0 # <-- change this to 1.0.1
    replicaCount: 1

To bring this change live you add, commit and push the change:

git add karrots-helloworld.yaml
git commit -m "Upgrade karrots-helloworld to v1.0.0"
git push origin/primary

A few minutes later FluxCD will detect this change and apply it to the cluster. If you visit https://karrots-helloworld.[your-domain].com. You should now see the message:

Welcome to Karrots-HelloWorld version 0.1.0

Port Your Old FluxCD Setup (Optional)¶

If you already use FluxCD then you will need to port your old setup to the Karrots idiomatic layout in your new karrots-cluster repo. See the post FluxCD for Busy People for why we recommend this organization structure for FluxCD. During this reorganization you need to add a karrots.yaml file to any new cluster you add to the karrots-cluster/clusters directory.

Adding a Development Cluster¶

We will now see how easy it is to build a dev cluster that contains the development version. You can easily add an automated cluster for any part of the software delivery pipeline: Dev, QA, Integration, Stage and, yes, even Production! In a later step we will show how you can even create ephemeral variants any of your clusters using git branch.

To create the new dev cluster we start by having Karrots initialize the cluster's directory structure. From the karrots-cluster directory execute the add-cluster command:

cd ~/git/karrots-cluster
karrots add-cluster dev

This will create a new cluster directory with an idiomatic structure to hold your deployments, helm-releases and other cluster resources.

cd ~/git/karrots-cluster/clusters/dev
ls
/deployments
  /central-config
    /namespaces
    /configMaps
    /cronjobs
    /secrets
/charts
/helm-releases

See the post FluxCD for Busy People for why we recommend this organization structure for FluxCD.

Populating the New Dev Cluster¶

At this point if you're comfortable with FluxCD you can populate the dev cluster with your organization's deployments, configs, helm-releases, etc., and then skip to the next step. If you simply want to continue with this example using the Karrots Test application then continue with this step. (You can always go back later an update your dev cluster for real.)

The easiest way to do this is to simply copy its helm-release from the karrots-helloworld cluster:

cp ~/git/karrots-cluster/clusters/karrots-helloworld/releases/* ~/git/karrots-cluster/clusters/dev/releases/

Now edit the karrots.yaml with the values that apply for where you want to run the cluster.

cd ~/git/fluxcd/clusters/dev
vi karrots.yaml

kubernetes:
  provider: # eks, gke, azure
  region: # us-east-1, us-west-2, etc
  cluster: dev
  domain: # zerodiff.org

Bring up the New Dev Cluster¶

To bring-up the dev cluster, run the karrots create-cluster command from the dev cluster directory.

cd ~/git/karrots-cluster/clusters/dev
karrots create-cluster

About 20-30 minutes later the dev cluster should be live on the internet at: https://dev.[your-domain].com. You should now see the message:

Welcome to Karrots-HelloWorld version 0.1.0

Branch the Dev Cluster for Story Work¶

When it comes time for a team to do story work in dev the team can now share it's own ephemeral cluster dedicated to this work! Let's say the team is working on story feature/JIRA-1234 then the first step is to run the Karrots branch command to update all of the git and FluxCD references so there is a git branch dedicated to this ephemeral cluster.

cd ~/git/karrots-cluster/clusters/dev
karrots branch feature/JIRA-1234 dev-feature-1234

The branch command leaves the structures intact, but creates a git branch and changes all of the FluxCD references to point at that branch. It also updates your karrots.yaml file for the new branch. You will want to verify these changes before you create the dev branch cluster.

cd ~/git/karrots-cluster/clusters/dev
vi karrots.yaml

kubernetes:
  provider: # eks, gke, azure
  region: # us-east-1, us-west-2, etc
  cluster: dev-feature-1234
  domain: # zerodiff.org

Now make your cluster changes and push them to the origin/primary central repo:

cd ~/git/karrots-cluster/clusters/dev/releases/karrots-helloworld
vi karrots-helloworld.yaml

Change the image:release tag to 1.0.2.

git add karrots-helloworld.yaml
cd ~/git/karrots-cluster
git commit -m "Update Karrots Test in the dev cluster"
git push origin/primary

To spin-up the dev-feature-1234 cluster, run the karrots create-cluster command from the dev cluster directory.

cd ~/git/karrots-cluster/clusters/dev
karrots create-cluster

About 20-30 minutes later the dev-feature-1234 cluster should be live on the internet at: https://dev-feature-1234.[your-domain].com. You should now see the message:

Welcome to Karrots-HelloWorld version 0.2.0

Dev Story Iteration¶

If we setuup the HelmRelease semver to automatically update

Wrapping Up and Tear Down¶

We haven't spent too much money running these clusters for the last few hours, but you may want to bring them down until you're ready to adopt Karrots for real in your software delivery pipeline. Since we're already in the dev-feature-1234 branch, we'll tear down the dev-feature-1234 cluster first:

cd ~/git/karrots-cluster/clusters/dev
karrots destroy-cluster

This will run a similar set of Terraform steps as Karrots create-cluster, but it will invoke terraform destroy instead. Once the command completes you will want to switch back to the primary branch to tear down the other clusters:

cd ~/git/karrots-cluster/clusters/dev
karrots branch primary

Since the primary branch already exists, Karrots simply changes to it using git commands. Now you can tear down the dev cluster:

cd ~/git/karrots-cluster/clusters/dev
karrots destroy-cluster

And to tear down the karrots-helloworld cluster:

cd ~/git/karrots-cluster/clusters/karrots-helloworld
karrots destroy-cluster