Building a Kubernetes Platform on Azure: Clusters from Code, Pipelines That Halved Deployment Time

The problem: slow deployments and environments that could drift

• Deployments took twice as long as they needed to.
• Environments weren't yet defined consistently in code, so dev, staging and production could drift apart, and "works in staging" didn't guarantee "works in production".
• A key messaging platform (Confluent) ran as an unsupported manual deployment, with no version control and no clean way to roll back.

The approach: one definition, many environments

Everything that makes up an environment comes from code, and every change flows through the same pipeline:

Commit → Pipeline → Image → Helm release → AKS (dev → staging → prod)

What I built

1. Clusters from code

Kubernetes clusters built from scratch with Terraform across dev, staging and production, with reusable AKS Terraform modules that became the blueprint other infrastructure projects adopted.

Illustrative: the core of an AKS module (azurerm provider v4 syntax).

resource "azurerm_kubernetes_cluster" "this" {
  name                = "aks-${var.app}-${var.env}"
  location            = var.location
  resource_group_name = var.resource_group_name
  dns_prefix          = "aks-${var.app}-${var.env}"

  default_node_pool {
    name                 = "system"
    vm_size              = var.node_size
    auto_scaling_enabled = true
    min_count            = var.min_nodes
    max_count            = var.max_nodes
  }

  identity {
    type = "SystemAssigned"
  }

  role_based_access_control_enabled = true
}

The same module, with different variables, produces every environment, so differences between environments are deliberate, not accidental.

2. Workloads with Helm

Applications deployed with Helm: autoscaling (HPA), ingress, RBAC, health probes and persistent storage, all in versioned charts.

3. Azure DevOps pipelines

Azure DevOps pipelines (Git, Docker, Maven, Jenkins) for infrastructure and applications, with infrastructure defined in Terraform, ARM and Bicep.

Illustrative: build once, then deploy the same image through environments.

trigger: [main]

stages:
  - stage: Build
    jobs:
      - job: image
        pool: { vmImage: ubuntu-latest }
        steps:
          - task: Docker@2
            inputs:
              command: buildAndPush
              containerRegistry: acr-connection
              repository: app
              tags: $(Build.BuildId)

  - stage: DeployDev
    dependsOn: Build
    jobs:
      - deployment: helm
        environment: dev
        strategy:
          runOnce:
            deploy:
              steps:
                - task: HelmDeploy@0
                  inputs:
                    connectionType: Azure Resource Manager
                    azureSubscription: arm-connection
                    azureResourceGroup: rg-app-dev
                    kubernetesCluster: aks-app-dev
                    command: upgrade
                    chartType: FilePath
                    chartPath: charts/app
                    releaseName: app
                    overrideValues: image.tag=$(Build.BuildId)
                    install: true

Staging and production repeat the deploy stage with their own environments, which is where Azure DevOps approvals and checks gate promotion.

4. Confluent onto Helm

Confluent moved from an unsupported manual deployment to Helm-packaged releases, adding version control and rollback across dev, QA and production.

5. Standard monitoring

Prometheus and Grafana deployed through Helm as the standard monitoring stack for every cluster.

Things to watch

• Build once, promote the same artifact. Rebuilding the image per environment means production runs something staging never tested. Tag once, deploy that tag everywhere.
• Gate promotion, not just deployment. Azure DevOps environments support approvals and checks. Use them on staging and production so promotion is a decision, not a side effect.
• Provider versions change attribute names. The azurerm v4 provider renamed several AKS arguments (for example enable_auto_scaling → auto_scaling_enabled). Pin provider versions in modules.
• Rollback needs history. helm rollback only works if releases are versioned and retained. Moving Confluent onto Helm is what made rollback possible at all.

What it solves

• Deployment time cut by 50%.
• Every environment rebuilt from the same code, so drift is visible and deliberate.
• Upgrades with a rollback path instead of manual changes.
• A reusable blueprint other projects adopted, instead of each team building clusters its own way.

What it doesn't solve

• Security by default. Clusters from code are consistent, not automatically secure. Admission policies, image scanning and workload identity are separate layers (see the Kubernetes guardrails case study).
• Secrets in pipelines. Service connections still need credential hygiene. Workload identity federation is the modern answer (see the Azure identity case study).
• Application-level drift. Terraform and Helm define the platform; manual kubectl changes can still drift a live cluster unless access is restricted.

Checklist

• [ ] One reusable cluster module, with variables per environment
• [ ] Provider versions pinned
• [ ] Images built once and promoted unchanged through environments
• [ ] Approvals and checks on staging and production environments
• [ ] Every workload deployed as a versioned Helm release, with rollback tested
• [ ] Standard monitoring deployed to every cluster
• [ ] Direct kubectl changes to production restricted

Further reading

• Azure Kubernetes Service documentation: https://learn.microsoft.com/azure/aks/
• Azure Pipelines environments, approvals and checks (Microsoft Learn)
• Helm documentation: https://helm.sh/docs/
• Terraform azurerm_kubernetes_cluster (Terraform Registry)