Continuous Integration

Dagger seems nice.

Notes

• Fully-automated seamless CI/CD is the goal.
• At Stripe my PR is often reviewed within 10 minutes. That makes a huge difference for shipping fast.
• I feel like if I’m sitting on someone’s PR, I’m blocking them from moving forward. Rarely the case, but it’s a good motivator.
• The real missing part for CI/CD automation is a local development experience. Sounds counter-intuitive but being able to run the E2E CI/CD pipeline locally and validate failures saves engineers from the 20 minute push, coffee, failure, fix loop.

Links

• Concourse CI - Automation system written in Go. It is most commonly used for CI/CD, and is built to scale to any kind of automation pipeline, from simple to complex. (Web)
• HN: Jenkins Is Getting Old (2019)
• Buildkite - Platform for running fast, secure, and scalable continuous integration pipelines on your own infrastructure. (CLI) (Docs) (Docs Code)
• Buildkite Agent - Open-source toolkit written in Golang for securely running build jobs on any device or network.
• Buildkite Builder - Ruby gem for programmatically creating Buildkite pipeline YML.
• Buildkite Terraform Provider
• buildx - Docker CLI plugin for extended build capabilities with BuildKit.
• Lobsters: What CI platforms do you use? (2019)
• huskyCI - Performing security tests inside your CI.
• Strider - Open Source Continuous Integration & Deployment Server. (Web)
• Sleuth - Bring full visibility and reporting across your remote team's DevOps stack.
• Snyk - Enables businesses to easily build security into their continuous development process. (Community) (Code)
• See 13 Netlify features for the best control of development workflow (2020)
• Netlify Build - Pluggable tool for running builds locally and in Netlify CI.
• Netlify Build Plugins
• GitOps Engine - Reusable library that implements core GitOps features.
• Automating safe, hands-off deployments (2020)
• Platform.sh - End-to-end cloud PaaS for hosting, development, and deployment. (Docs)
• Drone CI - Automate Software Testing and Delivery.
• PipeCD - Continuous Delivery for Declarative Kubernetes, Serverless and Infrastructure Applications. (Web)
• Conducto - Radically Better CI/CD. Python, not YAML. Lightning quick debugging. Seamless GitHub and Slack integration. (Lobsters)
• Waypoint - Build, deploy, and release applications across any platform. (Code) (Article) (HN) (Waypoint Plugin SDK) (Waypoint Example Apps) (Waypoint Horizon API) (Waypoint Plugin Kustomize)
• Buddy - DevOps Automation Platform.
• CDS - Enterprise-Grade Continuous Delivery & DevOps Automation Platform written in Go. (Web)
• A tiny CI system (2020) (Lobsters)
• CI for performance: Reliable benchmarking in noisy environments (2020) (Lobsters)
• Quad CI - CI server written in Simple Haskell.
• CI/CD Workflow for AWS ECS via Terragrunt and GitHub Actions (2020)
• CI/CD for ARM Development (2021)
• Abstruse CI - Free and open-source CI/CD platform for companies and individuals who value independence and simplicity. Written in Go. (Web)
• Fulfilling the promise of CI/CD (2021) (HN)
• tbls - CI-Friendly tool for document a database, written in Go.
• Setting up a CI system part 1: Preparing your test machines (2021)
• Delta CI - Bare Metal CI/CD with much faster builds. (HN)
• ProgressiveDeliveryConf 2021 - Learn how to go beyond CI/CD.
• The Tao of Continuous Integration (2021)
• Speeding up Docker builds in CI with BuildKit (2021)
• Continuous Integration With Interactive Notebooks
• Modern CI is Too Complex and Misdirected (2021) (Lobsters) (HN)
• Ask HN: How Long Is Your CI Process? (2021)
• Mega-Linter - Open-Source tool for CI/CD workflows that analyzes consistency and quality.
• Woodpecker - Simple CI engine with great extensibility. (Docs)
• BuildKit integration for Rust
• DevOps CI/CD Explained in 100 Seconds
• Werft - Kubernetes-native CI system. It knows no pipelines, just jobs and each job is a Kubernetes pod. (Code)
• How to Learn CircleCI in Simple Words (2021)
• Deliveroo CircleCI helper image - Container with useful helpers for CircleCI 2.0.
• Vela - Pipeline Automation (CI/CD) framework built on Linux container technology written in Go. (Docs)
• CircleCI CLI - Use CircleCI from the command line.
• Tekton Pipelines - Provides k8s-style resources for declaring CI/CD-style pipelines. (Web) (Buildkit Tekton)
• Tekton Chains - Kubernetes Custom Resource Definition (CRD) controller that allows you to manage your supply chain security in Tekton.
• Bitrise - Mobile Continuous Integration and Delivery. (GitHub) (Bitrise Step Libs)
• Bitrise (offline) CLI (Web)
• Zero Downtime Deployments (2021)
• Common Threat Matrix for CI/CD Pipeline
• Visual Studio App Center SDK for iOS and macOS
• App Center fastlane plugin
• snowpatch - Enabling continuous integration for patch-based development workflows.
• ci-env - Environment variables exposed by CI tools.
• go-circleci - CircleCI API (V2) Client Library in Go.
• bkl - Run Buildkite Pipelines locally.
• Example Pipelines | Buildkite
• What we learnt by migrating from CircleCI to Buildkite (2021)
• Spinnaker - Open-source continuous delivery platform for releasing software changes with high velocity and confidence. (Web)
• Danger JS - Runs after your CI, automating your team's conventions surrounding code review.
• Faster Gitlab CI/CD pipelines (2021) (HN)
• Developing Databricks' Runbot CI Solution (2021)
• Conform - Policy enforcement for your pipelines.
• YourBase - New build tool optimized for local + remote development. (Code)
• Hercules CI Agent - Runs on your infrastructure and executes tasks (evaluation, builds, etc) for your Hercules CI account.
• Bob the Builder - What CI/CD should've been. (Web)
• Cirrus CI - Modern Continuous Integration system built for the era of cloud computing. (GitHub) (CLI) (Docs Code)
• Awesome CI/CD Security
• List of Continuous Integration services
• Lightning-fast CI with nixbuild.net (2022)
• Homu - Bot that integrates with GitHub and your favorite continuous integration service.
• Dagger - Build powerful CI/CD pipelines quickly, then run them anywhere. (Code) (Intro) (HN)
• We don’t use a staging environment (2022) (HN)
• Spacelift - Flexible CI/CD for Infrastructure as Code. (GitHub) (CLI)
• Creating one CI to rule them all, with Fedor Korotkov, founder and CTO of Cirrus Labs (2022)
• Polar Signals - Continuous profiling project for applications and infrastructure. (GitHub)
• Zadig - Cloud native, distributed, developer-oriented continuous delivery product.
• Tekton Results - Aims to help users logically group CI/CD workload history and separate out long term result storage away from the Pipeline controller.
• Ratchet - Tool for improving the security of CI/CD workflows by automating the process of pinning and unpinning upstream versions.
• CI Free Tier Showdown (2022)
• goldboot - Provisioning bare-metal with CI.
• Scribe - Framework to write flexible CI pipelines in Go that have consistent behavior when ran locally or in a CI server.
• Ask HN: CI in 2022