What is vcluster.yaml

The vcluster.yaml file contains all configurations when deploying and upgrading virtual clusters. It's optionally passed to the CLI or as the values file for deployments using Helm. It is also possible to create a vCluster without a vcluster.yaml file which will be created with a set of default values.

Example configurations include the vCluster control plane's backing store, resource syncing behavior from and to the host cluster, and exporting the virtual cluster's kubeconfig file.

• Unified Configuration: All settings for vCluster in one place, simplifying management.
• Schema Validation: Automatically validate the configuration when using the vCluster CLI.
• IDE Support: Integrated schema with popular IDEs for autocompletion and validation.

Example vcluster.yaml file

controlPlane:
  backingStore:
    etcd:
      embedded:
        enabled: true
  ingress:
    enabled: true
    host: vcluster-k8s-api.example.com
sync:
  toHost:
    serviceAccounts:
      enabled: true
  fromHost:
    nodes:
      enabled: true
      clearImageStatus: true
exportKubeConfig:
  context: my-vcluster-context
  server: https://vcluster-k8s-api.example.com
  secret:
    name: my-vcluster-kubeconfig

This example configuration defines:

1. Embedded etcd as the vCluster control plane's backing store to reduce management overhead (Pro feature).
2. Syncing ServiceAccount resources from the virtual to the host cluster, to support, for example, AWS IRSA.
3. Deploying an Ingress resource on the host cluster to expose the vCluster control plane's API server at a hostname.
4. Syncing real node data, rather than the default "pseudo" nodes, but clears the ImageStatuses field within the NodeStatus of the synced nodes.
5. How to export the kubeconfig file to a secret, so you can use it, for example, in your ArgoCD or Terraform pipelines.

How to Use vcluster.yaml

Creating a New Virtual Cluster

• Define your configurations in the vcluster.yaml file.
• Deploy your virtual cluster.

vCluster CLI

1. Install the vCluster CLI.

   Homebrew

   brew install loft-sh/tap/vcluster-experimental

   If you installed the CLI using brew install vcluster, you should brew uninstall vcluster and then install the experimental version. The binaries in the tap are signed using the Sigstore framework for enhanced security.

   Mac (Intel/AMD)

   curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.12/vcluster-darwin-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

   Mac (Silicon/ARM)

   curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.12/vcluster-darwin-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

   Linux (AMD)

   curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.12/vcluster-linux-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

   Linux (ARM)

   curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.12/vcluster-linux-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

   Download Binary

   Download the binary for your platform from the GitHub Releases page and add this binary to your $PATH.

   Windows Powershell

   md -Force "$Env:APPDATA\vcluster"; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Tls,Tls11,Tls12';
   Invoke-WebRequest -URI "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.12/vcluster-windows-amd64.exe" -o $Env:APPDATA\vcluster\vcluster.exe;
   $env:Path += ";" + $Env:APPDATA + "\vcluster";
   [Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::User);

   Reboot Required

   You may need to reboot your computer to use the CLI due to changes to the PATH variable (see below).

   Check Environment Variable $PATH

   Line 4 of this install script adds the install directory %APPDATA%\vcluster to the $PATHenvironment variable. This is only effective for the current Powershell session, i.e. when opening a new terminal window,vcluster may not be found.

   Make sure to add the folder %APPDATA%\vcluster to the PATH environment variable after installing vcluster CLI via Powershell. Afterward, a reboot might be necessary.

   Confirm that you've installed the correct version of the vCluster CLI.

   vcluster --version

2. Deploy vCluster.

   vcluster create my-vcluster --namespace team-x --values vcluster.yaml

   When the installation finishes, you are automatically connected to the virtual cluster. You Kubernetes context is updated to point to your new virtual cluster. You can run local kubectl commands for the new virtual cluster.

Helm

1. Create the "team-x" namespace.

   kubectl create namespace team-x

2. Deploy vCluster using helm upgrade, which can be used to deploy a new vCluster or re-deploy an existing vCluster with new vcluster.yaml values.

   helm upgrade --install my-vcluster vcluster \
   --values vcluster.yaml \
   --repo https://charts.loft.sh \
   --namespace team-x \
   --repository-config='' \
   --version 0.20.0-beta.12

Terraform

1. Create a main.tf file.

   provider "helm" {
     kubernetes {
       config_path = "~/.kube/config"
     }
   }
   
   resource "helm_release" "my_vcluster" {
     name             = "my-vcluster"
     namespace        = "team-x"
     create_namespace = true
   
     repository       = "https://charts.loft.sh"
     chart            = "vcluster"
     version          = "0.20.0-beta.12"
   
     values = [
       file("${path.module}/vcluster.yaml")
     ]
   }

2. Install the required Helm provider.

   terraform init

3. Generate a plan.

   terraform plan

   Verify that the provider can access your cluster and that the proposed changes are correct.

4. Deploy vCluster.

   terraform apply

Argo CD

To deploy vCluster using ArgoCD, you need the following files:

• vcluster.yaml for your configuration options of your vCluster.
• my-vcluster-app.yaml for your ArgoCD Application definition.

1. Create the ArgoCD Application file my-vcluster-app.yaml, which refers to the vCluster Helm chart.

   apiVersion: argoproj.io/v1alpha1
   kind: Application
   metadata:
     name: my-vcluster
     namespace: argocd
   spec:
     project: default
     source:
       chart: vcluster
       repoURL: https://charts.loft.sh
       targetRevision: 0.20.0-beta.12
       helm:
         releaseName: my-vcluster
         valueFiles:
           - vcluster.yaml
     destination:
       server: https://kubernetes.default.svc
       namespace: team-x

2. Commit and push these files to your configured ArgoCD repository and synchronize it with your configured cluster.

Cluster API

1. Install the clusterctl CLI.

2. Install the vCluster provider.

   clusterctl init --infrastructure vcluster:v0.2.0-alpha.2

3. Generate the required manifests and apply using clusterctl generate cluster and kubectl.

   export CLUSTER_NAME=my-vcluster
   export CLUSTER_NAMESPACE=team-x
   export KUBERNETES_VERSION=1.29.3
   export HELM_VALUES=$(awk '{printf "%s\\n", $0}' vcluster.yaml)
   export CHART_VERSION=0.20.0-beta.12
   
   kubectl create namespace ${CLUSTER_NAMESPACE}
   
   clusterctl generate cluster ${CLUSTER_NAME} \
     --infrastructure vcluster \
     --kubernetes-version ${KUBERNETES_VERSION} \
     --target-namespace ${CLUSTER_NAMESPACE} | kubectl apply -f -

4. Execute the following command to wait for the vCluster custom resource to report a ready status.

   kubectl wait --for=condition=ready vcluster -n $CLUSTER_NAMESPACE $CLUSTER_NAME --timeout=300s

Learn more about Cluster API Provider for vCluster.

Updating Existing Virtual Clusters

• Simply update your vcluster.yaml file with the new configurations.
• Apply the updates.

vCluster CLI

vcluster create --upgrade VCLUSTER_NAME -n VCLUSTER_NAMESPACE -f vcluster.yaml

Replace:

• VCLUSTER_NAME with your vCluster instance name.
• VCLUSTER_NAMESPACE with the namespace where you deployed vCluster.

Helm

Since the v0.20 release is a beta, you need to specify the version or Helm uses the latest GA version.

helm upgrade --install VCLUSTER_NAME vcluster \
  --values vcluster.yaml \
  --repo https://charts.loft.sh \
  --namespace VCLUSTER_NAMESPACE \
  --repository-config='' \
  --version 0.20.0-beta.12

Replace:

• VCLUSTER_NAME with your vCluster instance name.
• VCLUSTER_NAMESPACE with the namespace where you deployed vCluster.

kubectl

Since the v0.20 release is a beta, you need to specify the version or Helm uses the latest GA version.

helm template VCLUSTER_NAME vcluster --repo https://charts.loft.sh --version 0.20.0-beta.12 -n VCLUSTER_NAMESPACE -f vcluster.yaml | kubectl apply -f -

Replace:

• VCLUSTER_NAME with your vCluster instance name.
• VCLUSTER_NAMESPACE with the namespace where you deployed vCluster.

Terraform

Apply vCluster config changes by editing the vcluster.yaml file and running terraform plan:

terraform plan

Review the planned changes and apply them if they look appropriate:

terraform apply

Argo CD

Add your vcluster.yaml config file to the valueFiles array in your ArgoCD Application file.

Since the v0.20 release is a beta, you need to specify the targetRevision or Helm uses the latest GA version.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: VCLUSTER_NAME
  namespace: argocd
spec:
  project: default
  source:
    chart: vcluster
    repoURL: https://charts.loft.sh
    targetRevision: 0.20.0-beta.12
    helm:
      releaseName: VCLUSTER_NAME
      valueFiles:
        - vcluster.yaml
  destination:
    server: https://kubernetes.default.svc
    namespace: VCLUSTER_NAMESPACE

Replace:

• VCLUSTER_NAME with your vCluster instance name.
• VCLUSTER_NAMESPACE with the namespace where you deployed vCluster.

Cluster API

Apply Cluster API changes by regenerating the cluster custom resource using clusterctl.

Since the v0.20 release is a beta, you need to export the chart version, or Helm uses the latest GA version.

export CLUSTER_NAME=VCLUSTER_NAME
export CLUSTER_NAMESPACE=VCLUSTER_NAMESPACE
export KUBERNETES_VERSION=1.29.3
export HELM_VALUES=$(cat vcluster.yaml)
export CHART_VERSION=0.20.0-beta.12

clusterctl generate cluster ${CLUSTER_NAME} \
    --infrastructure vcluster \
    --kubernetes-version ${KUBERNETES_VERSION} \
    --target-namespace ${CLUSTER_NAMESPACE} | kubectl apply -f -

Replace:

• VCLUSTER_NAME with your vCluster instance name.
• VCLUSTER_NAMESPACE with the namespace where you deployed vCluster.

After the changes have been applied, wait for the vCluster custom resource to report a ready status:

kubectl wait --for=condition=ready vcluster -n $CLUSTER_NAMESPACE $CLUSTER_NAME --timeout=300s

FAQs

How do I upgrade my 0.19.x and earlier virtual cluster configurations?

If you have existing 0.19.x virtual clusters, you can convert the old configuration values to the new v0.20 format. View the guide for more details.

Is it required to provide a vcluster.yaml file?

No, if a vcluster.yaml file isn't provided, the vCluster will be created using a set of default values.

Can I change the configuration for existing virtual clusters?

Yes, you can run vcluster create VCLUSTER_NAME --upgrade -f vcluster.yaml