Updraft ExtensionsPreview
ExampleIntermediate 20 min

Driving a stack from a versioned configuration file

One template, three environments, and no parameter list — with the configuration in a reviewed file and the deployment pinned to a specific version of it.

Resource types used Updraft::Config::Document, Updraft::Config::Lookup, Updraft::Config::ParameterSet, AWS::S3::Bucket

The scenario #

A payments service deploys to development, staging and production. The three environments differ in about twenty values: retention periods, instance sizes, feature flags, downstream hostnames, alarm thresholds.

The usual approaches all have the same shape of problem:

ApproachProblem
Twenty template parametersThe parameter list becomes the interface. Adding a value means touching every pipeline that calls the template.
A Mappings block keyed by environmentConfiguration lives inside the template, so changing a threshold means redeploying the template that defines the infrastructure.
Three copies of the templateThey diverge. Always.
A parameter file per environment in the pipelineBetter — but the file is only visible to the pipeline, and nothing else can read it.

Putting the configuration in S3 keeps the file reviewable and diffable, lets things other than CloudFormation read it, and — with a pinned version — makes each deployment record exactly which configuration it ran against.

The document is read once per stack operation. Lookups feed template properties at deploy time; the parameter set publishes the same values for things that read at runtime.

The configuration file #

# s3://acme-platform-config/environments/production.yaml
retention:
  days: 90
  tier: STANDARD_IA

database:
  host: payments-db.internal.acme.example
  port: 5432
  poolSize: 40

featureFlags:
  newCheckout: true
  legacyExport: false

alarms:
  errorRateThreshold: 0.01
  latencyP99Ms: 800

The template #

Parameters:
  Environment:
    Type: String
    AllowedValues: [development, staging, production]
  ConfigVersion:
    Type: String
    Description: >-
      S3 object version of the configuration file. Supplied by the pipeline,
      which reads it after uploading. Pinning it here is what makes a
      deployment reproducible.

Resources:
  # --- Read the configuration -----------------------------------------

  AppConfig:
    Type: Updraft::Config::Document
    Properties:
      Source:
        Bucket: acme-platform-config
        Key: !Sub 'environments/${Environment}.yaml'
        ExpectedOwner: !Ref AWS::AccountId
      VersionId: !Ref ConfigVersion
      Format: Yaml

      # Values every environment gets unless it says otherwise.
      Defaults:
        retention:
          days: 30
          tier: STANDARD
        featureFlags: {}

      # Fail the deployment on a malformed file, before anything is built.
      Schema:
        Inline:
          type: object
          required: [database, retention]
          properties:
            database:
              type: object
              required: [host, port]
        AllowAdditionalProperties: false

      # Never let a credential reach an attribute, an event, or a log.
      Redact:
        - Path: /database/password
          Strategy: Remove

  # --- Pull out what the infrastructure needs -------------------------

  RetentionDays:
    Type: Updraft::Config::Lookup
    Properties:
      Document: !Ref AppConfig
      Path: /retention/days
      Type: Number

  RetentionTier:
    Type: Updraft::Config::Lookup
    Properties:
      Document: !Ref AppConfig
      Path: /retention/tier
      Type: String
      Default: STANDARD

  # --- Use it -----------------------------------------------------------

  ArchiveBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Properties:
      BucketName: !Sub 'acme-payments-archive-${Environment}-${AWS::AccountId}'
      VersioningConfiguration:
        Status: Enabled
      LifecycleConfiguration:
        Rules:
          - Id: archive-then-expire
            Status: Enabled
            Transitions:
              - StorageClass: !Ref RetentionTier
                TransitionInDays: 30
            ExpirationInDays: !Ref RetentionDays
          - Id: abort-incomplete-uploads
            Status: Enabled
            AbortIncompleteMultipartUpload:
              DaysAfterInitiation: 7

  # --- Publish it for things that read at runtime -----------------------

  PublishedConfig:
    Type: Updraft::Config::ParameterSet
    Properties:
      Document: !Ref AppConfig
      Prefix: !Sub '/acme/payments/${Environment}'
      Exclude:
        - /alarms          # only the template cares about these
      Tier: Standard
      DeleteOnRemove: true

Outputs:
  ConfigChecksum:
    Description: >-
      Changes whenever effective configuration changes. Record it with the
      deployment so a later incident can be tied to a configuration state.
    Value: !GetAtt AppConfig.Checksum

  ConfigVersionUsed:
    Description: The S3 object version this deployment actually read.
    Value: !GetAtt AppConfig.ResolvedVersionId

  PublishedParameterCount:
    Value: !GetAtt PublishedConfig.ParameterCount

What it produces #

!Ref RetentionDays → 90from the file, not the Defaults block

!Ref RetentionTier → STANDARD_IA

!GetAtt AppConfig.Checksum → 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

!GetAtt PublishedConfig.ParameterCount → 7alarms excluded

The pipeline half #

Pinning ConfigVersion only works if something supplies it. The pipeline uploads the file and passes back the version it created:

VERSION=$(aws s3api put-object \
  --bucket acme-platform-config \
  --key "environments/${ENVIRONMENT}.yaml" \
  --body "config/${ENVIRONMENT}.yaml" \
  --query VersionId --output text)

aws cloudformation deploy \
  --stack-name "payments-${ENVIRONMENT}" \
  --template-file template.yaml \
  --parameter-overrides "Environment=${ENVIRONMENT}" "ConfigVersion=${VERSION}"

Things to watch #

Rolling back a configuration change #

Because the version is a stack parameter, rolling back configuration is a deployment with an older version and nothing else changed:

aws s3api list-object-versions \
  --bucket acme-platform-config \
  --prefix environments/production.yaml \
  --query 'Versions[*].[VersionId,LastModified]' --output table

aws cloudformation deploy \
  --stack-name payments-production \
  --template-file template.yaml \
  --parameter-overrides Environment=production ConfigVersion=<previous>

The infrastructure changes back with it, because the lookups feed real properties. That is the payoff for reading configuration at deploy time rather than at runtime: it participates in the same rollback the rest of the stack does.