Opinionated Rails Generator

2020-08-11T19:15:42+00:00

Recently I made long overdue updates to a Rails app generator I’ve been using on and off since 2009. The choices have changed over the years, but they aren’t shocking. Nothing you couldn’t do yourself, but it may save you a few minutes. Depending on your needs, check out the Generator or the Output. And please let me know if you see something you think is outdated or an anti-pattern.

Generates a Rails 6 application with these choices:

PostgreSQL database
Bootstrap V5 front-end toolkit
Devise authentication
Pundit authorization
Slim templating engine
RSpec & Cucumber testing
Bullet N+1 detection
FactoryBot test data
Kaminari pagination
Ransack search
MetaTags SEO
Pry console
Gravatar with fallback to LetterAvatar
RuboCop & ESLint linters, with Prettier formatting
Model annotation
Code coverage
Ready for cloud deploy at Heroku

Deploy Static Website to Amazon S3

2020-07-20T22:43:42+00:00

⚠️ Update (Out of Date): This post reflects an older CloudFormation-based setup and is kept for historical reference. For new projects, prefer AWS CDK so your infrastructure is best practices based, easier to test, and simpler to evolve over time. Start here: AWS CDK Developer Guide.

TL;DR — CloudFormation template that autodeploys MacksMind.io. Download from GitHub.

Why CloudFormation?

Setting up static deploy to S3 usually involves many steps in the AWS UI, then using aws s3 sync locally to deploy changes. Here I demonstrate how to automate jekyll build in CodePipeline/CodeBuild, and also how to create all the resources in a single CloudFormation template. It creates between 12 and 20 resources depending on your selections.

Yes it’s overkill for a Jekyll site, but it’s also a starting point for related use cases that only need a single pipeline from a single GitHub branch. What follows is section by section commentary to help you with that adaptation.

Parameters

Template sharing & reuse is all about parameters. If you’re deploying a Jekyll site, you might not have to change a thing. Some of the less obvious bits are:

GitHubSecret
- Create an OAuth token for GitHub using steps 1-6 of these instructions
- Store the token in AWS Secrets Manager as “Other type of secrets” without automatic rotation
- Construct a parameter resembling {{resolve:secretsmanager:macksmind.io:SecretString:github-token}}, in which macksmind.io is the secret name, and github-token is the key pointing to the token
- Of course you can paste the token directly into the parameter if you remove AllowedPattern, but you shouldn’t
ChatbotSlackArn
- To receive Slack notifications when deploys start and finish, link to Slack using these instructions
- Once the channel is configured, paste the ARN into the parameter

AWSTemplateFormatVersion: 2010-09-09
Parameters:
  GitHubOwner:
    Type: String
    Description: GitHub repo owner
    MinLength: 1
  GitHubRepo:
    Type: String
    Description: GitHub repo name
    MinLength: 1
  GitHubBranch:
    Type: String
    Default: master
    Description: GitHub branch name
    MinLength: 1
  GitHubSecret:
    Type: String
    Description: Reference to AWS Secrets Manager
    AllowedPattern: "\\{\\{resolve:secretsmanager:.+:SecretString:.+\\}\\}"
  Subdomain:
    Type: String
    Default: www
    Description: Subdomain of URL (optional)
  RootDomain:
    Type: String
    Description: Root domain of URL
  IndexPage:
    Type: String
    Default: index.html
    Description: Default page in any directory
    MinLength: 1
  ErrorPage:
    Type: String
    Default: 404.html
    Description: Page not found path (optional, but highly recommended)
  Route53:
    Type: String
    Description: Automatically configure DNS in Route53?
    AllowedValues:
    - true
    - false
    Default: false
  RedirectRoot:
    Type: String
    Description: Redirect RootDomain to Subdomain?
    AllowedValues:
    - true
    - false
    Default: false
  JekyllEnv:
    Type: String
    Description: Jekyll build environment
    Default: production
    AllowedValues:
    - production
    - development
  ChatbotSlackArn:
    Type: String
    Description: AWS Chatbot Channel ARN (optional)

Metadata

Controls parameter display in CloudFormation UI. Without this, parameters display alphabetically. 🙁

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
    - Label:
        default: GitHub Configuration
      Parameters:
      - GitHubOwner
      - GitHubRepo
      - GitHubBranch
      - GitHubSecret
    - Label:
        default: Website Info
      Parameters:
      - Subdomain
      - RootDomain
      - IndexPage
      - ErrorPage
    - Label:
        default: DNS Details
      Parameters:
      - Route53
      - RedirectRoot

Conditions

Conditions and parameters are separate namespaces, permitting a Boolean such as Route53 to match the String parameter it’s derived from
Note the condition chaining using Fn::And

Conditions:
  HasSubdomain:
    Fn::Not:
    - Fn::Equals:
      - ''
      - Ref: Subdomain
  HasErrorPage:
    Fn::Not:
    - Fn::Equals:
      - ''
      - Ref: ErrorPage
  RedirectRoot:
    Fn::And:
    - Condition: HasSubdomain
    - Fn::Equals:
      - 'true'
      - Ref: RedirectRoot
  Route53:
    Fn::Equals:
    - 'true'
    - Ref: Route53
  Route53Redirect:
    Fn::And:
    - Condition: Route53
    - Condition: RedirectRoot
  Chatbot:
    Fn::Not:
    - Fn::Equals:
      - ''
      - Ref: ChatbotSlackArn

Certificate

Manual Step Alert!!

Certificate validation via DNS requires creation of a cryptic CNAME in the root domain
Once creation of your stack is under way, visit the Route53 console to view the pending certificate
Expand the certificate to view the CNAME
Create the CNAME in your DNS or if your’re using Route53, AWS will create it for you with the push of a button

Resources:
  Certificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName:
        Fn::If:
        - HasSubdomain
        - Fn::Sub: "${Subdomain}.${RootDomain}"
        - Ref: RootDomain
      DomainValidationOptions:
      - DomainName:
          Fn::If:
          - HasSubdomain
          - Fn::Sub: "${Subdomain}.${RootDomain}"
          - Ref: RootDomain
        ValidationDomain:
          Ref: RootDomain
      ValidationMethod: DNS

Deploy Bucket

WebsiteConfiguration defines handling for requests that don’t have an exact match
AWS::NoValue essentially unsets ErrorDocument, but be warned the default 404 is cryptic
DeployBucketPolicy bravely lets all the world read, but not list

  DeployBucket:
    Type: AWS::S3::Bucket
    Properties:
      WebsiteConfiguration:
        IndexDocument:
          Ref: IndexPage
        ErrorDocument:
          Fn::If:
          - HasErrorPage
          - Ref: ErrorPage
          - Ref: AWS::NoValue
  DeployBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket:
        Ref: DeployBucket
      PolicyDocument:
        Statement:
        - Action:
          - s3:GetObject
          Effect: Allow
          Principal: "*"
          Resource:
            Fn::Sub: "${DeployBucket.Arn}/*"

CloudFront Distribution

Compress turns on automatic compression
MinTTL sets the cache to 24 hours even though browsers see cache-control: no-cache
ViewerProtocolPolicy handles https redirection while OriginProtocolPolicy avoids attempting https with S3
DomainName uses the public hostname of the bucket, because WebsiteConfiguration has no effect on S3 access using AWS internals

  Distribution:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Aliases:
        - Fn::If:
          - HasSubdomain
          - Fn::Sub: "${Subdomain}.${RootDomain}"
          - Ref: RootDomain
        DefaultCacheBehavior:
          AllowedMethods:
          - GET
          - HEAD
          Compress: true
          ForwardedValues:
            QueryString: false
          MinTTL: 86400
          TargetOriginId: S3Origin
          ViewerProtocolPolicy: redirect-to-https
        Enabled: true
        Origins:
        - DomainName:
            Fn::Sub: "${DeployBucket}.s3-website-${AWS::Region}.amazonaws.com"
          Id: S3Origin
          CustomOriginConfig:
            OriginProtocolPolicy: http-only
        HttpVersion: http2
        PriceClass: PriceClass_100
        ViewerCertificate:
          AcmCertificateArn:
            Ref: Certificate
          MinimumProtocolVersion: TLSv1.2_2018
          SslSupportMethod: sni-only

DNS RecordSets

Condition: Route53 makes these resources optional
A & AAAA to cover IPv4 and IPv6
Using Route53 aliases instead of CNAME for a shorter lookup
- If your DNS is not at Route53, you’ll need CNAMEs pointing to the CloudFront DomainName

  DistributionIP4:
    Type: AWS::Route53::RecordSet
    Condition: Route53
    Properties:
      AliasTarget:
        DNSName:
          Fn::GetAtt:
          - Distribution
          - DomainName
        HostedZoneId: Z2FDTNDATAQYW2
      HostedZoneName:
        Fn::Sub: "${RootDomain}."
      Name:
        Fn::If:
        - HasSubdomain
        - Fn::Sub: "${Subdomain}.${RootDomain}"
        - Ref: RootDomain
      Type: A
  DistributionIP6:
    Type: AWS::Route53::RecordSet
    Condition: Route53
    Properties:
      AliasTarget:
        DNSName:
          Fn::GetAtt:
          - Distribution
          - DomainName
        HostedZoneId: Z2FDTNDATAQYW2
      HostedZoneName:
        Fn::Sub: "${RootDomain}."
      Name:
        Fn::If:
        - HasSubdomain
        - Fn::Sub: "${Subdomain}.${RootDomain}"
        - Ref: RootDomain
      Type: AAAA

CodePipeline

The magic starts here.

PollForSourceChanges turns off polling since we define a Webhook below

  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      ArtifactStore:
        Location:
          Ref: PipelineBucket
        Type: S3
      RoleArn:
        Fn::GetAtt:
        - PipelineRole
        - Arn
      Stages:
      - Actions:
        - ActionTypeId:
            Category: Source
            Owner: ThirdParty
            Provider: GitHub
            Version: 1
          Configuration:
            Owner:
              Ref: GitHubOwner
            Repo:
              Ref: GitHubRepo
            Branch:
              Ref: GitHubBranch
            OAuthToken:
              Ref: GitHubSecret
            PollForSourceChanges: false
          Name: GitHubCommit
          OutputArtifacts:
          - Name: SourcePipe
        Name: Source
      - Actions:
        - Name: JekyllDeploy
          ActionTypeId:
            Category: Build
            Owner: AWS
            Provider: CodeBuild
            Version: 1
          Configuration:
            ProjectName:
              Ref: Project
          InputArtifacts:
          - Name: SourcePipe
        Name: Build

Pipeline Bucket

CodePipeline stores the code here for CodeBuild
CodeBuild also caches here as you’ll see below

  PipelineBucket:
    Type: AWS::S3::Bucket
    Properties:
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true

Pipeline Role & Policy

All the perms needed to orchestrate the build

  PipelineRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action: sts:AssumeRole
          Effect: Allow
          Principal:
            Service: codepipeline.amazonaws.com
        Version: 2012-10-17
  PipelineRoleDefaultPolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyDocument:
        Statement:
        - Action:
          - s3:GetObject*
          - s3:GetBucket*
          - s3:List*
          - s3:DeleteObject*
          - s3:PutObject*
          - s3:Abort*
          Effect: Allow
          Resource:
          - Fn::GetAtt:
            - PipelineBucket
            - Arn
          - Fn::Sub: "${PipelineBucket.Arn}/*"
        - Action:
          - codebuild:BatchGetBuilds
          - codebuild:StartBuild
          - codebuild:StopBuild
          Effect: Allow
          Resource:
            Fn::GetAtt:
            - Project
            - Arn
        Version: 2012-10-17
      PolicyName: PipelineRoleDefaultPolicy
      Roles:
      - Ref: PipelineRole

Webhook

Automatically manages webhooks from your GitHub repo.

  PipelineWebhook:
    Type: AWS::CodePipeline::Webhook
    Properties:
      Authentication: GITHUB_HMAC
      AuthenticationConfiguration:
        SecretToken:
          Ref: GitHubSecret
      Filters:
      - JsonPath: "$.ref"
        MatchEquals: refs/heads/{Branch}
      TargetAction: GitHubCommit
      TargetPipeline:
        Ref: Pipeline
      TargetPipelineVersion:
        Fn::GetAtt:
        - Pipeline
        - Version
      RegisterWithThirdParty: true

CodeBuild Project

The magic continues here.

Cache property, bundle config set path, and cache key in BuildSpec avoid unneeded gem installs
- node_modules and npm install would work in a similar way
- Simulate rm -rf node_modules by deleting the cache from the Pipeline Bucket
--cache-control='no-cache' is set during aws s3 sync
` aws cloudfront create-invalidation` refreshes the cache

  Project:
    Type: AWS::CodeBuild::Project
    Properties:
      Name:
        Ref: AWS::StackName
      Artifacts:
        Type: CODEPIPELINE
      Cache:
        Location:
          Fn::Sub: "${PipelineBucket}/build_cache"
        Type: S3
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/standard:4.0
        Type: LINUX_CONTAINER
      ServiceRole:
        Fn::GetAtt:
        - ProjectRole
        - Arn
      Source:
        BuildSpec:
          Fn::Sub: |
            version: 0.2
            phases:
              install:
                runtime-versions:
                  ruby: 2.7
                commands:
                  - mkdir -p _bundle_cache
                  - bundle config set path _bundle_cache
                  - bundle config set without 'development test'
                  - bundle install
              build:
                commands:
                  - JEKYLL_ENV=${JekyllEnv} bundle exec jekyll build
              post_build:
                commands:
                  - aws s3 sync --cache-control='no-cache' _site s3://${DeployBucket}/ --delete
                  - aws cloudfront create-invalidation --distribution-id ${Distribution} --paths '/*'
            cache:
              paths:
              - '_bundle_cache/**/*'
        Type: CODEPIPELINE

CodeBuild Role & Policy

Lots of perms for all the CodeBuild things.

  ProjectRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action: sts:AssumeRole
          Effect: Allow
          Principal:
            Service: codebuild.amazonaws.com
        Version: 2012-10-17
  ProjectRoleDefaultPolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyDocument:
        Statement:
        - Action:
          - s3:GetObject*
          - s3:GetBucket*
          - s3:List*
          - s3:DeleteObject*
          - s3:PutObject*
          - s3:Abort*
          Effect: Allow
          Resource:
          - Fn::GetAtt:
            - PipelineBucket
            - Arn
          - Fn::Sub: "${PipelineBucket.Arn}/*"
        - Action:
          - s3:List*
          - s3:DeleteObject*
          - s3:PutObject*
          - s3:Abort*
          Effect: Allow
          Resource:
          - Fn::GetAtt:
            - DeployBucket
            - Arn
          - Fn::Sub: "${DeployBucket.Arn}/*"
        - Action:
          - logs:CreateLogGroup
          - logs:CreateLogStream
          - logs:PutLogEvents
          Effect: Allow
          Resource:
          - Fn::Sub: arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/${Project}
          - Fn::Sub: arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/${Project}:*
        - Action: cloudfront:CreateInvalidation
          Effect: Allow
          Resource:
            Fn::Sub: arn:${AWS::Partition}:cloudfront::${AWS::AccountId}:distribution/${Distribution}
        Version: 2012-10-17
      PolicyName: ProjectRoleDefaultPolicy
      Roles:
      - Ref: ProjectRole

Slack Notification

AWS sends status updates and a link to the build
Love to see that ✅

  ProjectNotification:
    Type: AWS::CodeStarNotifications::NotificationRule
    Condition: Chatbot
    Properties:
      DetailType: FULL
      EventTypeIds:
      - codebuild-project-build-state-failed
      - codebuild-project-build-state-succeeded
      - codebuild-project-build-state-in-progress
      - codebuild-project-build-state-stopped
      Name:
        Fn::Sub: "${AWS::StackName}-notification"
      Resource:
        Fn::GetAtt:
        - Project
        - Arn
      Targets:
      - TargetAddress:
          Ref: ChatbotSlackArn
        TargetType: AWSChatbotSlack

Redirect Root Domain

A few key changes allow redirection to the primary hostname.

This Certificate also needs a CNAME for validation
RedirectAllRequestsTo in S3 is the key difference from the deploy bucket
https redirect is handled by S3 instead of CloudFront for fewer round trips
Oddly the bucket doesn’t need to be public ¯\_(ツ)_/¯

  CertificateRedirect:
    Type: AWS::CertificateManager::Certificate
    Condition: RedirectRoot
    Properties:
      DomainName:
        Ref: RootDomain
      DomainValidationOptions:
      - DomainName:
          Ref: RootDomain
        ValidationDomain:
          Ref: RootDomain
      ValidationMethod: DNS
  DeployBucketRedirect:
    Type: AWS::S3::Bucket
    Condition: RedirectRoot
    Properties:
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      WebsiteConfiguration:
        RedirectAllRequestsTo:
          HostName:
            Fn::Sub: "${Subdomain}.${RootDomain}"
          Protocol: https
  DistributionRedirect:
    Type: AWS::CloudFront::Distribution
    Condition: RedirectRoot
    Properties:
      DistributionConfig:
        Aliases:
        - Ref: RootDomain
        DefaultCacheBehavior:
          AllowedMethods:
          - GET
          - HEAD
          ForwardedValues:
            QueryString: false
          TargetOriginId: S3OriginRedirect
          ViewerProtocolPolicy: allow-all
        Enabled: true
        Origins:
        - DomainName:
            Fn::Sub: "${DeployBucketRedirect}.s3-website-${AWS::Region}.amazonaws.com"
          Id: S3OriginRedirect
          CustomOriginConfig:
            OriginProtocolPolicy: http-only
        HttpVersion: http2
        PriceClass: PriceClass_100
        ViewerCertificate:
          AcmCertificateArn:
            Ref: CertificateRedirect
          MinimumProtocolVersion: TLSv1.2_2018
          SslSupportMethod: sni-only
  DistributionIP4Redirect:
    Type: AWS::Route53::RecordSet
    Condition: Route53Redirect
    Properties:
      AliasTarget:
        DNSName:
          Fn::GetAtt:
          - DistributionRedirect
          - DomainName
        HostedZoneId: Z2FDTNDATAQYW2
      HostedZoneName:
        Fn::Sub: "${RootDomain}."
      Name:
        Ref: RootDomain
      Type: A
  DistributionIP6Redirect:
    Type: AWS::Route53::RecordSet
    Condition: Route53Redirect
    Properties:
      AliasTarget:
        DNSName:
          Fn::GetAtt:
          - DistributionRedirect
          - DomainName
        HostedZoneId: Z2FDTNDATAQYW2
      HostedZoneName:
        Fn::Sub: "${RootDomain}."
      Name:
        Ref: RootDomain
      Type: AAAA

Respawn: Live to Play Another Day

2020-07-18T17:01:11+00:00

Once upon a time, there was a blog.

It got an OK amount of traffic from people who were facing similar challenges, but I moved on to other things and let it decay. Then at some point it went away.

As I contemplate changes I’d like to make in my life, I realize I miss sharing solutions.

It’s not fancy, but it’s a start.

-M