Kubernetes Cluster - Nixidy + ArgoCD

GitOps-managed Kubernetes cluster using Nixidy for type-safe manifest generation and ArgoCD for continuous deployment.

Overview

This repository implements the "Rendered Manifests Pattern":

• Nix configuration (env/{dev,infra,prod}/*.nix) defines all Kubernetes resources in a type-safe way
• Nixidy generates plain YAML manifests and commits them to manifests/{dev,infra,prod}/
• ArgoCD watches the manifest directories and automatically syncs changes to the cluster
• App-of-apps pattern: ArgoCD manages itself and all other applications

Key Benefits: Type safety, reproducible builds, Git-based audit trail, declarative infrastructure.

Repository Structure

cluster/
├── flake.nix                   # Nix flake with dependencies and env configuration
├── flake.lock                  # Locked dependency versions
├── justfile                    # Common operations (run `just` to see all)
├── treefmt.nix                 # Formatter configuration
├── modules/
│   ├── common.nix              # Global networking options + nixidy defaults + port assertion
│   ├── templates.nix           # Reusable application templates (webApplication, route)
│   └── generators.nix          # CRD resource option generators (Traefik, Prometheus Operator)
├── lib/
│   └── default.nix             # lib.my.mkApplication, lib.my.checkServicePortUniqueness
├── tests/
│   ├── default.nix             # Unit tests (nix-unit, all systems)
│   └── integration.nix         # Integration tests (x86_64-linux only)
├── env/
│   ├── dev/                    # Dev application definitions (auto-discovered by import-tree)
│   │   ├── *.nix               # Application configs (argocd, traefik, etc.)
│   │   └── *.sops.yaml         # Encrypted secrets
│   ├── infra/                  # Infrastructure application definitions
│   │   ├── *.nix               # Infrastructure apps (monitoring, logging, etc.)
│   │   └── *.sops.yaml         # Encrypted secrets
│   └── prod/                   # Prod application definitions
│       └── *.nix
└── manifests/
    ├── dev/                    # Generated YAML manifests for dev (committed, do not edit)
    │   ├── apps/               # ArgoCD Application definitions
    │   └── */                  # Application manifests
    ├── infra/                  # Generated YAML manifests for infra (committed, do not edit)
    │   ├── apps/               # ArgoCD Application definitions
    │   └── */                  # Application manifests
    └── prod/                   # Generated YAML manifests for prod (committed, do not edit)
        ├── apps/               # ArgoCD Application definitions
        └── */                  # Application manifests

Prerequisites

• Nix with flakes enabled
• Kubernetes cluster (k3s recommended)
• SOPS with age keys for secrets management (optional)

Ingress & Routing

The cluster uses Traefik as the ingress controller, deployed via Helm chart.

Configuration

• LoadBalancer IP: 192.168.10.228 (configured in modules/common.nix)
• Domain (dev): *.dev.martials.no / Domain (infra): *.infra.martials.no / Domain (prod): *.martials.no (configured in modules/common.nix)
• Entry Points:
  • web: HTTP (port 80)
  • websecure: HTTPS (port 443)

Routing Methods

Traefik IngressRoute (type-safe Nix resources preferred; YAML also supported):

resources.ingressRoutes.myapp-route.spec = {
  entryPoints = [ "web" ];
  routes = [
    {
      match = "Host(`myapp.dev.martials.no`)";
      kind = "Rule";
      services = [ { name = "myapp-svc"; port = 80; } ];
    }
  ];
};

See toolbox.nix for a complete example.

Customizing Networking

Edit modules/common.nix to configure:

• networking.localIp: LoadBalancer IP address
• networking.domain.root: Base domain
• networking.subDomain: Subdomain for the environment (set automatically by mkEnv in flake.nix)

Quick Start

# Enter development shell (provides nixidy, kubectl, sops)
nix develop

# View available commands
just

# Build manifests (preview, doesn't modify files)
just build dev

# Generate manifests to manifests/dev/
just switch dev

# Review changes and commit
git diff manifests/
git add env/ manifests/
git commit -m "Update configuration"
git push

Note: ArgoCD automatically deploys changes pushed to the repository.

Common Operations

Run just or just --list to see all available commands. Most common:

just build dev          # Preview generated manifests
just switch dev         # Generate and write manifests
just diff dev           # Compare current vs generated
just argocd-status      # Check ArgoCD application status
just pods               # View all pods
just sops-edit          # Edit encrypted secrets

See the justfile for the complete list of operations.

Development Workflow

Making Changes

1. Edit configuration in env/dev/*.nix, env/infra/*.nix, or env/prod/*.nix
2. Build and validate: just build {env}
3. Generate manifests: echo y | just switch {env}
4. Review: git diff manifests/
5. Commit and push
6. ArgoCD auto-syncs within minutes

Adding a New Application

1. Create env/{dev,infra,prod}/myapp.nix — it is automatically discovered by import-tree, no registration needed:

{ lib, ... }:

{
  applications.myapp = {
    namespace = "myapp";
    createNamespace = true;

    resources = let
      labels = { "app.kubernetes.io/name" = "myapp"; };
    in {
      deployments.myapp.spec = {
        replicas = 2;
        selector.matchLabels = labels;
        template = {
          metadata.labels = labels;
          spec.containers.myapp = {
            image = "registry.example.com/myapp:v1.0.0";
            ports.http.containerPort = 8080;
          };
        };
      };

      services.myapp.spec = {
        selector = labels;
        ports.http = { port = 80; targetPort = 8080; };
      };
    };
  };
}

2. Build, switch, and commit: just build dev → echo y | just switch dev → commit env/ + manifests/

See DEPLOYMENT_GUIDE.md for detailed examples including ingress, secrets, databases, and CI/CD setup.

Adding a New Environment

To add a new environment (e.g., staging, testing), follow these steps:

1. Create Environment Directory

mkdir -p env/staging

2. Register Environment in flake.nix

Update the mkEnv function or the envs list in flake.nix to include your new environment. The mkEnv function automatically:

• Sets networking.subDomain to the environment name (or null for prod)
• Computes domain.fqdn as subDomain.domain.root (e.g., staging.martials.no)

Add your environment name to the environments list in the envs section:

[
  "dev"      # → subdomain "dev"   → dev.martials.no
  "infra"    # → subdomain "infra" → infra.martials.no
  "prod"     # → subdomain null    → martials.no
  "staging"  # → subdomain "staging" → staging.martials.no
]

3. Add Your Applications

Create application configuration files in the new environment directory. For example, create env/staging/toolbox.nix:

{ config, ... }:

{
  applications.toolbox = {
    namespace = "staging";
    templates.webApplication.toolbox = {
      image = "${config.networking.registry}/toolbox:1.0.0";
      internalPort = 8080;
    };
  };
}

Add additional .nix files for other applications as needed. Each file is automatically discovered by import-tree.

4. (Optional) Update modules/common.nix

If your environment needs different domain root or other networking settings, update modules/common.nix:

# In modules/common.nix, update domain.root or other networking config
networking.domain.root = "example.com";  # Override for all envs

The FQDN is automatically computed from networking.subDomain + domain.root. For the staging environment, the resulting FQDN would be staging.example.com.

5. Bootstrap the Environment

The bootstrap recipe deploys the initial ArgoCD manifests to your cluster:

# Validate configuration first
just build staging

# Bootstrap the environment (deploys ArgoCD and initial apps)
just bootstrap staging

# Verify deployment
kubectl get applications -n argocd

6. Generate and Commit Manifests

# Generate manifests to manifests/staging/
just switch staging

# Review changes
git diff manifests/

# Commit both .nix and manifests/
git add env/ manifests/
git commit -m "Add staging environment"
git push

7. Verify ArgoCD Sync

Once bootstrapped, ArgoCD automatically watches the manifests/staging/ directory and syncs applications:

# Watch application status
just watch-apps

# Or check specific app
just argocd-describe apps-staging

Key Points:

• The bootstrap recipe is the one-time setup that deploys ArgoCD and the app-of-apps
• After bootstrap, all subsequent deployments use just switch {env} → push → ArgoCD auto-syncs
• Each environment gets its own manifests/{env}/ directory
• Environment-specific configuration is stored in env/{env}/*.nix

Using Helm Charts

helm.releases.myapp = {
  chart = lib.helm.downloadHelmChart {
    repo = "https://charts.example.com";
    chart = "app";
    version = "1.0.0";
    chartHash = "";  # Leave empty; build error provides correct hash
  };

  values = {
    replicaCount = 2;
    # ... other values
  };
};

Managing Secrets with SOPS

# Edit encrypted secrets (requires age key configured)
just sops-edit

# Secrets are automatically decrypted by SOPS operator in cluster

Documentation

• AGENTS.md: Guidelines for AI agents and contributors
• justfile: All available commands with descriptions

Architecture

• Type Safety: Nixidy validates resources against Kubernetes schemas
• Reproducibility: flake.lock pins all dependencies
• GitOps: Single source of truth in Git, ArgoCD handles deployment
• Declarative: Entire cluster state defined in Nix configuration
• Secrets: SOPS encrypts secrets at rest in Git, decrypted in-cluster

Resources

• Nixidy Documentation - Official docs and options reference
• ArgoCD Documentation - GitOps deployment
• k3s Documentation - Lightweight Kubernetes
• SOPS - Secrets management

License

Personal cluster configuration. Use as reference for your own setup.