Networking8 min read

User-Data & Cloud-Config

Configuring instances with YAML and scripts

User-Data Formats

User-data is the blob you pass to an instance at launch time. Cloud-init inspects the first bytes to determine the format and dispatch accordingly.

Cloud-Config (YAML)

Starts with #cloud-config on the first line. Declarative YAML — the primary and recommended format. Cloud-init maps keys to modules that handle the work.

yaml

#cloud-config
packages:
  - nginx
  - curl
runcmd:
  - systemctl enable nginx

Shell Script

Starts with a shebang (#!/bin/bash, #!/usr/bin/env python3, etc.). Executed as a script during the final stage. Stdout/stderr logged to /var/log/cloud-init-output.log.

bash

#!/bin/bash
set -euo pipefail
apt-get update
apt-get install -y nginx
systemctl enable --now nginx

Multi-Part MIME

Combines multiple formats in one user-data payload. Each part has its own MIME type and content-type header. Cloud-init dispatches each part to the appropriate handler.

MIME types:

text/cloud-config
text/x-shellscript
text/cloud-boothook
text/x-include-url

Note

Size limit: Most clouds limit user-data to 16 KB (AWS) or 64 KB (GCE, Azure). Cloud-init supports gzip compression — if user-data starts with a gzip magic number, it's decompressed automatically. For large configs, use #include to reference URLs that cloud-init fetches at boot.

Key Cloud-Config Modules

users: Create Users & Groups

The users module creates system users with full control over groups, SSH keys, sudo, and shell. It runs during the config stage.

yaml

#cloud-config
users:
  - default                         # keep the distro's default user (ubuntu, ec2-user, etc.)
  - name: deploy
    gecos: Deploy User
    groups: [sudo, docker]
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    lock_passwd: true              # disable password login (SSH keys only)
    ssh_authorized_keys:
      - ssh-ed25519 AAAA...key1 deploy@bastion
      - ssh-ed25519 AAAA...key2 deploy@ci
  - name: monitoring
    system: true                   # system user (no home dir, low UID)
    shell: /usr/sbin/nologin

Tip

Important: If you define the users key, you must include - default to keep the distro's default user. Otherwise, cloud-init replaces the default user list entirely and you lose SSH access via the cloud-provided key pair.

packages: Install Packages

Cloud-init auto-detects the package manager (apt, yum, dnf, zypper). The packages module runs during the config stage.

yaml

#cloud-config
package_update: true              # apt-get update / yum makecache
package_upgrade: true             # apt-get upgrade / yum update (all packages)
package_reboot_if_required: true  # reboot if kernel was upgraded

packages:
  - nginx
  - curl
  - jq
  - unzip
  - [python3-pip, 3.10*]          # version pinning (package manager syntax)

Warning

Caution: package_upgrade: true upgrades all packages on first boot. This is great for security but adds significant time to instance startup (minutes, not seconds). For latency-sensitive workloads, bake updated packages into the AMI/image instead.

write_files: Write Arbitrary Files

Write files to the filesystem with full control over content, path, permissions, owner, and encoding. Runs during the config stage.

yaml

#cloud-config
write_files:
  - path: /etc/nginx/conf.d/app.conf
    content: |
      server {
          listen 80;
          server_name app.example.com;
          location / {
              proxy_pass http://127.0.0.1:8080;
          }
      }
    owner: root:root
    permissions: '0644'

  - path: /opt/app/config.env
    content: |
      DATABASE_URL=postgres://db.internal:5432/app
      REDIS_URL=redis://cache.internal:6379
    permissions: '0600'
    owner: deploy:deploy

  - path: /usr/local/bin/healthcheck.sh
    encoding: b64                  # base64-encoded content (for binaries or complex text)
    content: IyEvYmluL2Jhc2gKY3VybCAtZiAtcyBsb2NhbGhvc3Q6ODA4MC9oZWFsdGg=
    permissions: '0755'

Note

Encoding options: text/plain (default), b64 (base64), gz (gzip), gz+b64 (gzip + base64). Use b64 when your content has characters that would break YAML parsing.

runcmd vs bootcmd

runcmd (Final Stage)

Runs once, during the final stage (stage 5). Executed after users, packages, and write_files are all done.

yaml

runcmd:
  - systemctl enable --now nginx
  - [sh, -c, 'echo "boot done" >> /var/log/init.log']
  - /opt/app/setup.sh

Use case: One-time initialization — start services, run setup scripts, register with config management.

Execution: Each item runs via sh -c unless given as a list (which is exec'd directly).

bootcmd (Init Stage)

Runs on every boot, during the init-network stage (stage 3). Runs before users, packages, or write_files.

yaml

bootcmd:
  - echo 1 > /proc/sys/net/ipv4/ip_forward
  - [cloud-init-per, once, setup-swap, fallocate, -l, 2G, /swapfile]
  - modprobe br_netfilter

Use case: Kernel parameters, module loading, early filesystem setup. Things that must happen before the rest of cloud-init.

Note: Use cloud-init-per to make a bootcmd run only once.

ca_certs: Custom CA Certificates

yaml

#cloud-config
ca_certs:
  trusted:
    - |
      -----BEGIN CERTIFICATE-----
      MIIFazCCA1OgAwIBAgIUE... (your internal CA cert)
      -----END CERTIFICATE-----
  remove_defaults: false          # don't remove system default CAs

Tip

Corporate environments: If your org uses an internal CA for TLS interception or internal services, this module is critical. Without it, apt, curl, and any HTTPS connection to internal endpoints will fail with certificate validation errors.

apt / yum_repos: Configure Package Repositories

APT (Debian/Ubuntu)

bash

apt:
  sources:
    docker.list:
      source: "deb [arch=amd64] https://download.docker.com/linux/ubuntu $RELEASE stable"
      keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
      keyserver: hkps://keyserver.ubuntu.com
    hashicorp.list:
      source: "deb [arch=amd64] https://apt.releases.hashicorp.com $RELEASE main"
      key: |
        -----BEGIN PGP PUBLIC KEY BLOCK-----
        ...
        -----END PGP PUBLIC KEY BLOCK-----

YUM (RHEL/CentOS/Fedora)

bash

yum_repos:
  docker-ce-stable:
    name: Docker CE Stable
    baseurl: https://download.docker.com/linux/centos/$releasever/$basearch/stable
    enabled: true
    gpgcheck: true
    gpgkey: https://download.docker.com/linux/centos/gpg
  hashicorp:
    name: HashiCorp Stable
    baseurl: https://rpm.releases.hashicorp.com/RHEL/$releasever/$basearch/stable
    enabled: true
    gpgcheck: true
    gpgkey: https://rpm.releases.hashicorp.com/gpg

disk_setup / fs_setup / mounts

Partition, format, and mount attached disks. This runs during the init-network stage, so disks are ready before packages are installed or files are written.

yaml

#cloud-config
disk_setup:
  /dev/xvdf:
    table_type: gpt
    layout: true                   # single partition using entire disk
    overwrite: false                # don't overwrite if already partitioned

fs_setup:
  - device: /dev/xvdf1
    filesystem: ext4
    label: data-vol
    overwrite: false

mounts:
  - [/dev/xvdf1, /data, ext4, "defaults,nofail", "0", "2"]
  - [tmpfs, /tmp, tmpfs, "defaults,noatime,size=2G", "0", "0"]

Warning

Always use nofail in mount options for non-root disks. Without it, if the disk is missing (e.g., wrong instance type, detached EBS volume), the system will hang at boot waiting for a mount that can never succeed.

Other Common Modules

console

#cloud-config

# Add SSH keys to the default user
ssh_authorized_keys:
  - ssh-ed25519 AAAA...key1 ops@bastion
  - ssh-rsa AAAA...key2 ci@jenkins

# Set timezone
timezone: UTC

# Configure NTP
ntp:
  enabled: true
  pools:
    - 169.254.169.123              # AWS time sync (chrony)
  servers:
    - ntp.internal.corp

Jinja2 Templating

Since cloud-init 22.x, cloud-config YAML supports Jinja2 templates. This lets you write a single cloud-config that adapts dynamically based on instance metadata — no external templating layer needed.

How It Works

Add ## template: jinja as the very first line (before #cloud-config)
Access instance metadata via {{ ds.meta_data.* }}
Access user-data variables via {{ ds.user_data.* }}
Full Jinja2 syntax: {% if %}, {% for %}, {{ variable | filter }}
Rendered before YAML parsing, so the output must be valid YAML

console

## template: jinja
#cloud-config

# Dynamic hostname based on instance ID
hostname: web-{{ ds.meta_data.instance_id[-8:] }}

# Region-specific package mirror
{% if ds.meta_data.region == 'us-east-1' %}
apt:
  primary:
    - arches: [amd64]
      uri: http://us-east-1.ec2.archive.ubuntu.com/ubuntu/
{% elif ds.meta_data.region == 'eu-west-1' %}
apt:
  primary:
    - arches: [amd64]
      uri: http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/
{% endif %}

# Tag-based role assignment
packages:
  - curl
  - jq
{% if v1.instance_id %}
  - amazon-ssm-agent
{% endif %}

# Expose metadata for debugging
write_files:
  - path: /etc/instance-info
    content: |
      INSTANCE_ID={{ ds.meta_data.instance_id }}
      REGION={{ ds.meta_data.region }}
      AZ={{ ds.meta_data.availability_zone }}
      INSTANCE_TYPE={{ ds.meta_data.instance_type }}

Note

Available variables: The exact variables depend on the datasource. Use cloud-init query --all on a running instance to see what's available. Common ones: ds.meta_data.instance_id, ds.meta_data.region, ds.meta_data.availability_zone, ds.meta_data.local_ipv4, v1.instance_id, v1.cloud_name, v1.region.

Complete Cloud-Config Example

A production-grade cloud-config that creates a deploy user, installs packages, writes configuration, and runs a setup script:

console

#cloud-config

# --- System ---
hostname: web-prod-01
timezone: UTC
locale: en_US.UTF-8

# --- Users ---
users:
  - default
  - name: deploy
    groups: [sudo, docker]
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    lock_passwd: true
    ssh_authorized_keys:
      - ssh-ed25519 AAAA...key deploy@bastion

# --- Packages ---
package_update: true
packages:
  - nginx
  - docker.io
  - curl
  - jq
  - unzip
  - fail2ban

# --- NTP ---
ntp:
  enabled: true
  pools:
    - 169.254.169.123

# --- Files ---
write_files:
  - path: /etc/nginx/conf.d/app.conf
    content: |
      upstream app_backend {
          server 127.0.0.1:8080;
      }
      server {
          listen 80;
          server_name _;
          location / {
              proxy_pass http://app_backend;
              proxy_set_header Host $host;
              proxy_set_header X-Real-IP $remote_addr;
          }
          location /health {
              return 200 'ok';
          }
      }
    permissions: '0644'

  - path: /opt/app/deploy.sh
    permissions: '0755'
    content: |
      #!/bin/bash
      set -euo pipefail
      echo "[$(date)] Starting application deployment..."
      docker pull registry.internal/app:latest
      docker run -d --restart=unless-stopped \
        -p 8080:8080 \
        --name app \
        registry.internal/app:latest
      echo "[$(date)] Deployment complete."

# --- Run commands ---
runcmd:
  - systemctl enable --now docker
  - systemctl enable --now fail2ban
  - rm /etc/nginx/sites-enabled/default
  - systemctl enable --now nginx
  - /opt/app/deploy.sh

# --- Final message ---
final_message: "Cloud-init complete. Uptime: $UPTIME seconds."

Multi-Part MIME Example

When you need to combine cloud-config YAML with a shell script (or multiple configs), use multi-part MIME. The standard tool is cloud-init devel make-mime or you can construct it manually.

Creating Multi-Part MIME with cloud-init CLI

console


      $ cloud-init devel make-mime \

        --attach cloud-config.yaml:cloud-config \

        --attach setup.sh:x-shellscript \

        > user-data.mime

      

      $ cat user-data.mime

console

Content-Type: multipart/mixed; boundary="===============BOUNDARY=="
MIME-Version: 1.0

--===============BOUNDARY==
Content-Type: text/cloud-config; charset="utf-8"
MIME-Version: 1.0

#cloud-config
packages:
  - nginx
  - docker.io

users:
  - default
  - name: deploy
    groups: [sudo, docker]
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL

--===============BOUNDARY==
Content-Type: text/x-shellscript; charset="utf-8"
MIME-Version: 1.0

#!/bin/bash
set -euo pipefail

# This script runs AFTER cloud-config modules
echo "Running post-config setup..."
docker pull registry.internal/app:latest
docker run -d --restart=unless-stopped -p 8080:8080 registry.internal/app:latest

--===============BOUNDARY==--

Note

When to use MIME: Multi-part MIME is useful when different teams own different parts of the initialization. For example, the platform team provides the cloud-config (users, security, monitoring), and the app team provides a shell script (app deployment). The MIME bundle combines both without either team needing to modify the other's YAML.

Validating User-Data

Always validate your cloud-config before deploying. Syntax errors in user-data fail silently at boot — no error in the console, just missing configuration.

console


    $ cloud-init schema --config-file user-data.yaml

    Valid cloud-config: user-data.yaml

    

    $ cloud-init schema --config-file bad-config.yaml

    Error: cloud-config failed schema validation:

      packages.0: 123 is not of type 'string'

    

    # Validate the running instance's cloud-config

    $ cloud-init schema --system

    Valid schema /var/lib/cloud/instance/cloud-config.txt

Tip

CI pipeline tip: Add cloud-init schema --config-file to your CI pipeline that builds infrastructure. Catch cloud-config errors before they reach production. This works without cloud-init being installed — the schema validation is available in the cloud-init Python package.