GitOps Best Practices

What is GitOps?

GitOps is an operational model in which Git holds the declared state of what runs in your environments. Changes are made in Git, reviewed through pull requests, and a controller running in the cluster keeps the running state in line with what Git declares.

How It Works

• Configuration lives in Git — application manifests, environment settings, policies.
• Changes go through pull requests — code review, approval, audit trail.
• A GitOps controller watches the repository — typically ArgoCD or Flux.
• Sync — the controller compares the cluster with Git and, where allowed, applies the difference.
• Self-healing, depending on the tool and settings — in ArgoCD, a manual change in the cluster is by default only reported as OutOfSync and waits for a person. Automated sync and self-heal are switched on per application, and pruning resources that are no longer in Git is a separate decision with its own risk. Flux takes the opposite default for plain manifests: its Kustomization controller re-applies what Git declares on every interval and reverts manual changes to the fields it manages. For Helm releases, drift detection and correction are opt-in.

Key Advantages

GitOps for Applications, IaC for Infrastructure

Infrastructure as Code builds the environment — servers, networks, cloud resources, the cluster itself — and runs when the infrastructure changes. GitOps delivers applications into that environment and keeps reconciling them for as long as they run. Tools such as Crossplane or Cluster API reconcile infrastructure continuously too; the boundary between building the environment and delivering applications into it still holds, even when one team owns both.

In practice: separate repositories or clearly separated paths, separate pipelines and, where teams are split, separate owners. The IaC pipeline needs credentials to your cloud or virtualisation layer; the GitOps controller needs access to the cluster and the repository, not to provision infrastructure.

If your IaC is Terraform, choose it knowing its licence. HashiCorp moved Terraform to the Business Source License in August 2023; every release from 1.6 (October 2023) onwards ships under it. OpenTofu is the open-source fork under MPL 2.0, now a CNCF project. The difference matters to procurement and to organisations with an open-source-first policy. Record which one your pipelines use, and why.

Common Use Cases

• Promotion between environments — the same version moving from dev to staging to production through pull requests, not manual copying.
• Compliance and governance — policies checked in the pull request and enforced at admission, with an audit trail in Git history.
• Multi-cluster consistency — one declared configuration applied to many clusters, with the differences kept explicit.

Architecture Components

GitOps vs Manual or Push-Based Deployment

Implementation Recommendations

Start with:

• One non-critical application or environment
• ArgoCD or Flux as the GitOps controller — ArgoCD if teams need a UI and per-application access control, Flux if you want a lighter controller that reconciles on an interval
• A repository structure that separates application configuration from infrastructure code
• A decision on how secrets reach the application before the first application with a password — "everything in Git" and "no secrets in Git" collide on day one
• Automated sync, self-heal and alerting on drift set per application, on purpose

Repository structure example

apps/
├── app-a/
│   ├── base/
│   └── overlays/
│       ├── dev/
│       ├── staging/
│       └── production/
└── app-b/
    ├── base/
    └── overlays/
        ├── dev/
        ├── staging/
        └── production/

Keep this configuration in its own repository, separate from the application source code, and require extra review for changes under overlays/production, for example with CODEOWNERS. Helm-based applications follow the same idea with a values file per environment. Infrastructure code — networks, storage, the cluster itself — lives in its own repository, with its own pipeline.

Common Mistakes in GitOps Implementation

• Not using linting or schema checks: errors reach the cluster instead of failing in the pull request.
• Poor secrets management: secrets committed to the repository, or no external secrets store, complicate audits, create security risk and make rotation hard to automate.
• The "all-in-one" approach: every application and every environment in one file or one large manifest is difficult to read, review and change safely.
• Insufficient protection of the main branch: direct pushes, merges without approval or without required checks let untested changes reach production.
• Poor separation of environments: a change meant for staging lands in production because the structure does not make the boundary explicit.
• Mutable image tags: with latest or re-pushed tags, Git no longer says which image runs, and reverting a commit may not bring the old version back. Pin versions or digests.
• Declared state without reconciliation: the controller reports drift, nobody is alerted, and the repository slowly stops describing what runs.
• Manual fixes on the cluster: without automatic correction they stay until the next sync overwrites them; with it, they are reverted automatically — in ArgoCD within seconds, in Flux at the next reconciliation interval. Decide per application which behaviour you want, and tell the team before someone fixes production by hand.

How Long Does a One-Line Change Take to Reach Production?

Once changes travel through Git, a pipeline and a controller, every step of the path leaves a timestamp. That makes the path measurable, and the quickest way to measure it is the smallest change that still goes all the way to production.

1. Pick a trivial change that goes the whole way — for example, a text string or version label in the application itself, so it is built, tested, published and rolled out like any other change.
2. Note when the pull request is opened and when it is merged. The time between the two is your review queue; the time from merge to production is your delivery path.
3. Record each step until it runs in production: the build and tests, publishing the artefact, every promotion between environments, every approval, the controller sync, the rollout. The timestamps are already there: the merge time in Git, CI job logs, the approval tool and the controller's sync and rollout events.
4. For each step, write down two numbers: how long the work took and how long the change waited before the step started.

The result is a short list. Add up each column separately: the work column is what faster tooling can shorten, the waiting column is what process decides. Waiting can come from approvals in a queue, a change board that meets on fixed days, deployment windows, and manual promotion or switchover between environments. A trivial change measures the floor of your path; a real change adds review and testing on top. Repeat it on a different day of the week, because change boards and deployment windows make a single sample misleading.

In a regulated environment the controls stay. For each waiting step, ask whether the control can run inside the path instead of beside it: approval proportional to the risk of the change, the pull request as the change record, and an audit trail produced as the change moves rather than assembled afterwards.

The Bottom Line

GitOps applies the review and history you already use for application code to the way applications reach production. It delivers on that when reconciliation is switched on, drift is visible and secrets have a path. Run the one-line measurement once before you decide what to automate next.

Where We Apply This

A DevOps Capability Assessment starts from three questions: how long a change takes from merge to production, how many manual steps sit in the path, and whether what runs in production can be traced back to Git and to an approval. For ČSOB CZ we ran a structured DevOps capabilities assessment: the change path mapped as it was, a target state, and a prioritised roadmap.

Related reading: ČSOB CZ — DevOps Capabilities Assessment · KvaPay — infrastructure and GitOps · DevOps, CI/CD & GitOps