CI/CD 流水线构建
大约 11 分钟约 3351 字
CI/CD 流水线构建
简介
持续集成/持续部署(CI/CD)自动化了构建、测试和部署流程。理解 GitHub Actions/GitLab CI 的流水线配置、多环境部署策略和发布管理,有助于实现高效可靠的软件交付。
特点
GitHub Actions 工作流
完整 CI/CD 流水线
# .github/workflows/ci-cd.yml
name: CI/CD Pipeline
on:
push:
branches: [main, develop]
tags: ['v*']
pull_request:
branches: [main]
env:
DOTNET_VERSION: '8.0.x'
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
# 1. 构建与测试
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
- name: Cache NuGet packages
uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: ${{ runner.os }}-nuget-${{ hashFiles('**/*.csproj') }}
restore-keys: |
${{ runner.os }}-nuget-
- name: Restore dependencies
run: dotnet restore
- name: Build
run: dotnet build --no-restore -c Release
- name: Run Unit Tests
run: dotnet test --no-build -c Release --filter "Category!=Integration"
--logger "trx;LogFileName=test-results.trx"
--collect:"XPlat Code Coverage"
--results-directory ./coverage
- name: Run Integration Tests
run: dotnet test --no-build -c Release --filter "Category=Integration"
- name: Upload Test Results
uses: actions/upload-artifact@v4
if: always()
with:
name: test-results
path: ./coverage/
- name: Code Coverage Report
uses: danielpalme/ReportGenerator-GitHub-Action@5
with:
reports: ./coverage/**/coverage.cobertura.xml
targetdir: ./coveragereport
reporttypes: 'HtmlInline;Cobertura'
# 2. 安全扫描
security:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v4
- name: Run Snyk Security Scan
uses: snyk/actions/dotnet@master
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
- name: Container Scan (Trivy)
if: github.event_name == 'push'
uses: aquasecurity/trivy-action@master
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
format: 'sarif'
output: 'trivy-results.sarif'
# 3. 构建并推送 Docker 镜像
docker:
runs-on: ubuntu-latest
needs: [build, security]
if: github.event_name == 'push'
permissions:
contents: read
packages: write
outputs:
image_tag: ${{ steps.meta.outputs.tags }}
steps:
- uses: actions/checkout@v4
- name: Login to Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=sha,prefix=
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
# 4. 部署到开发环境
deploy-dev:
runs-on: ubuntu-latest
needs: docker
if: github.ref == 'refs/heads/develop'
environment:
name: development
steps:
- name: Deploy to Dev
run: |
echo "Deploying to development..."
kubectl set image deployment/user-api \
api=${{ needs.docker.outputs.image_tag }} \
--namespace=dev
# 5. 部署到生产环境
deploy-prod:
runs-on: ubuntu-latest
needs: docker
if: startsWith(github.ref, 'refs/tags/v')
environment:
name: production
steps:
- name: Deploy to Production
run: |
echo "Deploying to production..."
kubectl set image deployment/user-api \
api=${{ needs.docker.outputs.image_tag }} \
--namespace=prod
kubectl rollout status deployment/user-api --namespace=prod --timeout=300s发布管理
# .github/workflows/release.yml
name: Create Release
on:
push:
tags: ['v*']
jobs:
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # 获取完整历史
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- name: Determine Version
id: version
run: |
TAG=${GITHUB_REF#refs/tags/}
echo "version=${TAG#v}" >> $GITHUB_OUTPUT
- name: Pack NuGet Packages
run: |
dotnet pack -c Release -o ./artifacts \
-p:Version=${{ steps.version.outputs.version }}
- name: Generate Changelog
id: changelog
uses: mikepenz/release-changelog-builder-action@v4
with:
configuration: ".github/changelog-config.json"
- name: Create GitHub Release
uses: softprops/action-gh-release@v1
with:
tag_name: ${{ github.ref_name }}
name: Release ${{ github.ref_name }}
body: ${{ steps.changelog.outputs.changelog }}
files: |
./artifacts/*.nupkg
- name: Push NuGet Packages
run: |
dotnet nuget push ./artifacts/*.nupkg \
--source https://api.nuget.org/v3/index.json \
--api-key ${{ secrets.NUGET_API_KEY }} \
--skip-duplicate质量门禁
测试与覆盖率配置
// 测试项目配置
// Directory.Build.props
// <Project>
// <PropertyGroup>
// <TargetFramework>net8.0</TargetFramework>
// <ImplicitUsings>enable</ImplicitUsings>
// <Nullable>enable</Nullable>
// <IsPackable>false</IsPackable>
// <CollectCoverage>true</CollectCoverage>
// <CoverletOutputFormat>cobertura</CoverletOutputFormat>
// <CoverletOutput>./coverage/</CoverletOutput>
// <Threshold>80</Threshold>
// <ThresholdType>line</ThresholdType>
// </PropertyGroup>
// </Project>
// 集成测试使用 WebApplicationFactory
public class OrderApiIntegrationTests : IClassFixture<WebApplicationFactory<Program>>
{
private readonly WebApplicationFactory<Program> _factory;
private readonly HttpClient _client;
public OrderApiIntegrationTests(WebApplicationFactory<Program> factory)
{
_factory = factory.WithWebHostBuilder(builder =>
{
builder.ConfigureServices(services =>
{
// 替换为测试数据库
services.RemoveAll<DbContextOptions<AppDbContext>>();
services.AddDbContext<AppDbContext>(options =>
options.UseInMemoryDatabase("TestDb"));
});
});
_client = _factory.CreateClient();
}
[Fact]
public async Task CreateOrder_ReturnsCreated()
{
// Arrange
var request = new CreateOrderRequest("Product A", 10, 99.99m);
// Act
var response = await _client.PostAsJsonAsync("/api/orders", request);
// Assert
Assert.Equal(HttpStatusCode.Created, response.StatusCode);
var result = await response.Content.ReadFromJsonAsync<OrderDto>();
Assert.NotNull(result);
Assert.Equal("Product A", result.ProductName);
}
[Fact]
public async Task GetOrder_WhenNotFound_Returns404()
{
var response = await _client.GetAsync("/api/orders/999");
Assert.Equal(HttpStatusCode.NotFound, response.StatusCode);
}
}构建优化深入
Docker 多阶段构建与镜像瘦身
# Dockerfile — 多阶段构建最佳实践
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src
# 先拷贝 csproj 并 restore,利用 Docker 层缓存
COPY ["src/App/App.csproj", "src/App/"]
COPY ["src/App.Tests/App.Tests.csproj", "src/App.Tests/"]
RUN dotnet restore "src/App/App.csproj"
COPY . .
RUN dotnet publish "src/App/App.csproj" -c Release -o /app/publish \
--no-restore /p:DebugType=None /p:DebugSymbols=false
# 运行时镜像 — 使用 chiseled 镜像减小体积
FROM mcr.microsoft.com/dotnet/aspnet:8.0-chiseled AS runtime
WORKDIR /app
# 非 root 用户运行
USER $APP_UID
COPY --from=build /app/publish .
# 健康检查
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD curl -f http://localhost:8080/health || exit 1
ENV ASPNETCORE_URLS=http://+:8080
EXPOSE 8080
ENTRYPOINT ["dotnet", "App.dll"]构建缓存策略
# GitHub Actions — 多层缓存配置
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# 缓存 NuGet 包
- name: Cache NuGet
uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj', '**/Directory.Packages.props') }}
restore-keys: nuget-${{ runner.os }}-
# 缓存 .NET 本地工具
- name: Cache Tools
uses: actions/cache@v4
with:
path: .dotnet-tools
key: tools-${{ runner.os }}-${{ hashFiles('.config/dotnet-tools.json') }}
# Docker 层缓存 — 使用 GHA 缓存后端
- name: Build with cache
uses: docker/build-push-action@v5
with:
context: .
push: false
load: true
tags: app:test
cache-from: type=gha
cache-to: type=gha,mode=max
# 并行运行多个测试项目
- name: Run Tests in Parallel
run: |
dotnet test tests/UnitTests -c Release --no-build &
dotnet test tests/IntegrationTests -c Release --no-build &
wait矩阵构建 — 多目标并行
# 矩阵策略 — 同时测试多个运行时版本
jobs:
test-matrix:
strategy:
fail-fast: false # 不因单个失败取消其他
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
dotnet: ['7.0.x', '8.0.x']
exclude:
- os: macos-latest
dotnet: '7.0.x'
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ matrix.dotnet }}
- run: dotnet test -c ReleaseGitLab CI 流水线
GitLab CI 配置
# .gitlab-ci.yml
stages:
- build
- test
- package
- deploy
variables:
DOTNET_IMAGE: mcr.microsoft.com/dotnet/sdk:8.0
REGISTRY: $CI_REGISTRY
# 缓存配置
.cache: &cache
cache:
key: "${CI_COMMIT_REF_SLUG}"
paths:
- ~/.nuget/packages
- .gitlab/cache
build:
stage: build
image: $DOTNET_IMAGE
<<: *cache
script:
- dotnet restore
- dotnet build -c Release --no-restore
artifacts:
paths:
- **/bin/Release/
expire_in: 1 hour
test:
stage: test
image: $DOTNET_IMAGE
<<: *cache
needs: [build]
script:
- dotnet test -c Release --no-build
--logger "trx;LogFileName=test-results.trx"
--collect:"XPlat Code Coverage"
artifacts:
reports:
junit: '**/test-results.trx'
coverage_report:
coverage_format: cobertura
path: '**/coverage.cobertura.xml'
coverage: '/Line coverage: \d+\.\d+/'
docker-build:
stage: package
image: docker:24
services:
- docker:24-dind
needs: [test]
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin $CI_REGISTRY
script:
- docker build -t $REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHORT_SHA .
- docker push $REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHORT_SHA
rules:
- if: $CI_COMMIT_BRANCH == "main"
- if: $CI_COMMIT_TAG
deploy-staging:
stage: deploy
image: bitnami/kubectl
needs: [docker-build]
script:
- kubectl set image deployment/user-api api=$REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHORT_SHA -n staging
- kubectl rollout status deployment/user-api -n staging --timeout=300s
environment:
name: staging
url: https://staging.example.com
rules:
- if: $CI_COMMIT_BRANCH == "main"
deploy-production:
stage: deploy
image: bitnami/kubectl
needs: [docker-build]
script:
- kubectl set image deployment/user-api api=$REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHORT_SHA -n production
- kubectl rollout status deployment/user-api -n production --timeout=300s
environment:
name: production
url: https://api.example.com
rules:
- if: $CI_COMMIT_TAG
when: manual高级部署策略
蓝绿部署与金丝雀发布
# 蓝绿部署 — GitHub Actions
jobs:
deploy-blue-green:
runs-on: ubuntu-latest
needs: docker
environment: production
steps:
- name: Determine Active Slot
id: slot
run: |
ACTIVE=$(kubectl get service user-api -n prod -o jsonpath='{.spec.selector.slot}')
if [ "$ACTIVE" = "blue" ]; then
echo "target=green" >> $GITHUB_OUTPUT
else
echo "target=blue" >> $GITHUB_OUTPUT
fi
- name: Deploy to Inactive Slot
run: |
kubectl set image deployment/user-api-${{ steps.slot.outputs.target }} \
api=${{ needs.docker.outputs.image_tag }} -n prod
- name: Warm Up New Version
run: |
# 等待新 Pod 就绪并预热
kubectl rollout status deployment/user-api-${{ steps.slot.outputs.target }} \
-n prod --timeout=120s
sleep 10
curl -sf http://user-api-${{ steps.slot.outputs.target }}:8080/health || exit 1
- name: Run Smoke Tests
run: |
# 对新版本执行冒烟测试
pytest tests/smoke/ --target-slot=${{ steps.slot.outputs.target }}
- name: Switch Traffic
run: |
kubectl patch service user-api -n prod -p \
'{"spec":{"selector":{"slot":"${{ steps.slot.outputs.target }}"}}}'
- name: Rollback on Failure
if: failure()
run: |
ACTIVE=${{ steps.slot.outputs.target == 'blue' && 'green' || 'blue' }}
kubectl patch service user-api -n prod -p \
"{\"spec\":{\"selector\":{\"slot\":\"$ACTIVE\"}}}"GitOps 风格部署 — ArgoCD 同步
# 推送镜像标签到 Git 仓库,触发 ArgoCD 同步
jobs:
update-manifests:
runs-on: ubuntu-latest
needs: docker
steps:
- uses: actions/checkout@v4
with:
repository: myorg/k8s-manifests
token: ${{ secrets.PAT_TOKEN }}
- name: Update Image Tag
run: |
# 使用 kustomize 编辑镜像
cd overlays/production
kustomize edit set image \
user-api=${{ needs.docker.outputs.image_tag }}
- name: Commit and Push
run: |
git config user.name "CI Bot"
git config user.email "ci@example.com"
git add .
git commit -m "chore: update image to ${{ needs.docker.outputs.image_tag }}"
git push数据库迁移策略
# 安全的数据库迁移步骤
jobs:
migrate:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v4
- name: Generate Migration Script
run: |
dotnet ef migrations script \
--idempotent \
--output migrate.sql \
--project src/App \
--startup-project src/App
- name: Dry Run — Check Migration
run: |
# 对生产库执行 dry-run 检查(只读)
sqlcmd -S ${{ secrets.DB_HOST }} -d ${{ secrets.DB_NAME }} \
-U ${{ secrets.DB_USER }} -P ${{ secrets.DB_PASS }} \
-i migrate.sql -r 0 -b -t 30 2>&1 | head -50
env:
DB_CONNECTION: ${{ secrets.PROD_DB_CONNECTION }}
- name: Apply Migration
run: dotnet ef database update --project src/App --no-build
env:
ConnectionStrings__Default: ${{ secrets.PROD_DB_CONNECTION }}环境配置管理
多环境配置分离
// .github/environments/development.json
{
"ASPNETCORE_ENVIRONMENT": "Development",
"ConnectionStrings__Default": "${{ secrets.DEV_DB_CONNECTION }}",
"Redis__Host": "redis-dev.internal",
"Logging__LogLevel__Default": "Debug"
}
// .github/environments/production.json
{
"ASPNETCORE_ENVIRONMENT": "Production",
"ConnectionStrings__Default": "${{ secrets.PROD_DB_CONNECTION }}",
"Redis__Host": "redis-prod.internal",
"Logging__LogLevel__Default": "Warning"
}Kubernetes Secret 与 ConfigMap 管理
# 使用 Sealed Secrets 或 External Secrets Operator
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: app-secrets
spec:
refreshInterval: 1h
secretStoreRef:
name: vault-backend
kind: ClusterSecretStore
target:
name: app-secrets
data:
- secretKey: db-connection
remoteRef:
key: secret/data/app/production
property: connection_string
- secretKey: api-key
remoteRef:
key: secret/data/app/production
property: api_key监控与回滚
自动化健康检查与回滚
# 部署后自动验证,失败自动回滚
jobs:
deploy-with-verify:
runs-on: ubuntu-latest
needs: docker
steps:
- name: Deploy
run: |
kubectl set image deployment/user-api \
api=${{ needs.docker.outputs.image_tag }} -n prod
- name: Wait for Rollout
run: |
kubectl rollout status deployment/user-api \
-n prod --timeout=300s
- name: Verify Deployment Health
run: |
# 等待应用启动
sleep 15
# 检查健康端点
for i in $(seq 1 5); do
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
https://api.example.com/health)
if [ "$STATUS" = "200" ]; then
echo "Health check passed"
exit 0
fi
echo "Attempt $i: HTTP $STATUS, retrying..."
sleep 10
done
echo "Health check failed, rolling back..."
kubectl rollout undo deployment/user-api -n prod
exit 1
- name: Run Post-Deploy Smoke Tests
run: |
pytest tests/smoke/ --base-url=https://api.example.com \
--max-retries=3 --retry-delay=5
- name: Notify on Failure
if: failure()
uses: slackapi/slack-github-action@v1
with:
payload: |
{
"text": "Production deployment failed for ${{ github.repository }}",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": ":red_circle: *Deployment Failed*\nRepo: ${{ github.repository }}\nCommit: ${{ github.sha }}\nRun: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
}
}
]
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}流水线安全最佳实践
安全扫描集成
# 综合安全扫描流水线
jobs:
security-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# 依赖漏洞扫描
- name: SCA — Dependency Check
run: dotnet list package --vulnerable --include-transitive
# 静态代码分析
- name: SAST — Semgrep
uses: returntocorp/semgrep-action@v1
with:
config: >-
p/security-audit
p/secrets
p/owasp-top-ten
# 密钥泄露检测
- name: Secret Detection — TruffleHog
uses: trufflesecurity/trufflehog@main
with:
extra_args: --only-verified
# 容器镜像扫描
- name: Container Scan — Trivy
uses: aquasecurity/trivy-action@master
with:
image-ref: app:latest
format: 'table'
exit-code: '1'
severity: 'CRITICAL,HIGH'
ignore-unfixed: true分支保护与审批策略
# 环境保护规则 — production 环境需要人工审批
# 在 GitHub 仓库 Settings > Environments 中配置:
# - Required reviewers: 至少 2 人审批
# - Wait timer: 5 分钟冷却期
# - Deployment branches: 仅允许 main 和 release/* 分支
# - Environment secrets: 生产密钥仅在此环境可用流水线性能优化
常见性能瓶颈与优化
# 优化前后对比
# 优化前:串行执行所有测试,约 15 分钟
# 优化后:并行 + 增量构建,约 5 分钟
jobs:
# 使用路径过滤实现增量构建
changes:
runs-on: ubuntu-latest
outputs:
backend: ${{ steps.filter.outputs.backend }}
frontend: ${{ steps.filter.outputs.frontend }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v2
id: filter
with:
filters: |
backend:
- 'src/**'
- 'tests/**'
frontend:
- 'web/**'
# 仅后端变更时运行
backend-test:
needs: changes
if: needs.changes.outputs.backend == 'true'
runs-on: ubuntu-latest
steps:
- run: dotnet test -c Release
# 仅前端变更时运行
frontend-test:
needs: changes
if: needs.changes.outputs.frontend == 'true'
runs-on: ubuntu-latest
steps:
- run: npm test优点
缺点
总结
GitHub Actions 通过 YAML 工作流定义 CI/CD 流水线,Jobs 之间通过 needs 定义依赖关系。构建阶段利用 NuGet 缓存加速,测试阶段收集覆盖率和测试结果。Docker 镜像使用多阶段缓存和 docker/metadata-action 自动打标签。部署分为开发(develop 分支自动)、预发布(main 分支自动)和生产(Tag 手动审批)三个环境。质量门禁包括单元测试覆盖率阈值和安全扫描。
关键知识点
- 先分清这个主题位于请求链路、后台任务链路还是基础设施链路。
- 服务端主题通常不只关心功能正确,还关心稳定性、性能和可观测性。
- 任何框架能力都要结合配置、生命周期、异常传播和外部依赖一起看。
项目落地视角
- 画清请求进入、业务执行、外部调用、日志记录和错误返回的完整路径。
- 为关键链路补齐超时、重试、熔断、追踪和结构化日志。
- 把配置与敏感信息分离,并明确不同环境的差异来源。
常见误区
- 只会堆中间件或组件,不知道它们在链路中的执行顺序。
- 忽略生命周期和线程池、连接池等运行时资源约束。
- 没有监控和测试就对性能或可靠性下结论。
进阶路线
- 继续向运行时行为、可观测性、发布治理和微服务协同深入。
- 把主题和数据库、缓存、消息队列、认证授权联动起来理解。
- 沉淀团队级模板,包括统一异常处理、配置约定和基础设施封装。
适用场景
- 当你准备把《CI/CD 流水线构建》真正落到项目里时,最适合先在一个独立模块或最小样例里验证关键路径。
- 适合 API 服务、后台任务、实时通信、认证授权和微服务协作场景。
- 当需求开始涉及稳定性、性能、可观测性和发布流程时,这类主题会成为基础设施能力。
落地建议
- 先定义请求链路与失败路径,再决定中间件、过滤器、服务边界和依赖方式。
- 为关键链路补日志、指标、追踪、超时与重试策略。
- 环境配置与敏感信息分离,避免把生产参数写死在代码或镜像里。
排错清单
- 先确认问题发生在路由、模型绑定、中间件、业务层还是基础设施层。
- 检查 DI 生命周期、配置来源、序列化规则和认证上下文。
- 查看线程池、连接池、缓存命中率和外部依赖超时。
复盘问题
- 如果把《CI/CD 流水线构建》放进你的当前项目,最先要验证的输入、输出和失败路径分别是什么?
- 《CI/CD 流水线构建》最容易在什么规模、什么边界条件下暴露问题?你会用什么指标或日志去确认?
- 相比默认实现或替代方案,采用《CI/CD 流水线构建》最大的收益和代价分别是什么?
