Region-staggered maintenance windows
One template, deployed to eight regions, that gives each region a different maintenance window in a deliberate order — so a bad change reaches the regions you watch first.
Updraft::Schedule::CronExpression, Updraft::Region::Stagger, Updraft::Region::MetadataThe scenario #
A platform team deploys the same maintenance automation to eight regions. The job takes about forty minutes and briefly degrades the service it runs against.
Deployed naively, all eight run at 02:00 UTC on Sunday. That produces three distinct problems:
- Throttling. Eight regions calling the same account-level APIs simultaneously hit rate limits that none of them would hit alone.
- No observation window. A change that breaks the job breaks it everywhere in the same minute. Nobody finds out until Monday.
- Wrong local time. 02:00 UTC is a reasonable hour in Europe and the middle of the afternoon in Sydney.
Staggering fixes all three, and the ordering turns it from a spreading mechanism into a rollout strategy.
The template #
Identical in every region. Nothing is passed in that varies by region — each deployment works out its own position.
Resources:
# Refuse to deploy into a region too small to run the workload.
Here:
Type: Updraft::Region::Metadata
Properties:
RequireAvailabilityZones: 2
Fields: [AvailabilityZones, Partition]
# This region's position in a deliberate rollout order.
MaintenanceOffset:
Type: Updraft::Region::Stagger
Properties:
Interval: 90
Unit: Minutes
Anchor: "02:00"
# Explicit, not Alphabetical. The order is a blast-radius decision:
# the regions with the best observability go first.
Order: Explicit
Regions:
- us-west-2
- us-east-1
- eu-west-1
- eu-central-1
- ap-south-1
- ap-southeast-2
# Eight regions x 90 minutes is nine hours; wrapping keeps the whole
# rollout inside one day.
Wrap: true
MaintenanceSchedule:
Type: Updraft::Schedule::CronExpression
Properties:
Behavior: Weekly
DaysOfWeek: [SUN]
Time: !GetAtt MaintenanceOffset.StartTime
# UTC, deliberately: the exact local hour does not matter here, and
# picking a named zone would force a daylight-saving decision for
# no benefit.
TimeZone: UTC
Jitter: 5
JitterSeed: !Ref AWS::StackId
MaintenanceRule:
Type: AWS::Events::Rule
Properties:
Description: !Sub
- 'Weekly maintenance for ${AWS::Region} — ${Desc}'
- Desc: !GetAtt MaintenanceSchedule.LocalDescription
ScheduleExpression: !GetAtt MaintenanceSchedule.Expression
State: ENABLED
Targets:
- Id: maintenance
Arn: !GetAtt MaintenanceFunction.Arn
Outputs:
Window:
Description: Human-readable schedule, for review and for the runbook.
Value: !GetAtt MaintenanceSchedule.LocalDescription
Position:
Description: This region's position in the rollout.
Value: !Sub
- '${Index} of ${Total}'
- Index: !GetAtt MaintenanceOffset.Index
Total: !GetAtt MaintenanceOffset.Total
NextRuns:
Value: !Join [", ", !GetAtt MaintenanceSchedule.NextOccurrences]
{
"Resources": {
"Here": {
"Type": "Updraft::Region::Metadata",
"Properties": {
"RequireAvailabilityZones": 2,
"Fields": ["AvailabilityZones", "Partition"]
}
},
"MaintenanceOffset": {
"Type": "Updraft::Region::Stagger",
"Properties": {
"Interval": 90,
"Unit": "Minutes",
"Anchor": "02:00",
"Order": "Explicit",
"Regions": [
"us-west-2", "us-east-1", "eu-west-1",
"eu-central-1", "ap-south-1", "ap-southeast-2"
],
"Wrap": true
}
},
"MaintenanceSchedule": {
"Type": "Updraft::Schedule::CronExpression",
"Properties": {
"Behavior": "Weekly",
"DaysOfWeek": ["SUN"],
"Time": { "Fn::GetAtt": ["MaintenanceOffset", "StartTime"] },
"TimeZone": "UTC",
"Jitter": 5,
"JitterSeed": { "Ref": "AWS::StackId" }
}
}
}
}
What each region computes #
The same template, three different answers:
| Deployed in | Index | StartTime | Expression |
|---|---|---|---|
us-west-2 | 0 | 02:00 | cron(3 2 ? * SUN *) |
eu-west-1 | 2 | 05:00 | cron(41 5 ? * SUN *) |
ap-southeast-2 | 5 | 09:30 | cron(18 9 ? * SUN *) |
The minute differs per region because the jitter is seeded from the stack ID, which differs per deployment. It is stable across updates of the same stack.
!GetAtt MaintenanceOffset.StartTime
→
05:00in eu-west-1
!GetAtt MaintenanceSchedule.LocalDescription
→
Every Sunday at 05:41 UTC
Why Order: Explicit
#
Alphabetical would put ap-south-1 and ap-southeast-2 first, and us-west-2
last. That is a defensible spread and a poor rollout: the regions that go first
are the ones with the least tooling and the fewest engineers awake.
Ordering explicitly makes the sequence a decision rather than an accident. It also means adding a region is a reviewed template change with a visible consequence, instead of a silent renumbering.
Verifying before you trust it #
Deploy to one region first and read the outputs. NextOccurrences is computed
at deploy time and is the fastest way to confirm the schedule is what you meant:
aws cloudformation describe-stacks \
--stack-name maintenance \
--query 'Stacks[0].Outputs[?OutputKey==`NextRuns`].OutputValue' \
--output text