DeOldify服务CI/CD流水线：GitHub Actions自动构建镜像+部署验证

DeOldify服务CI/CD流水线：GitHub Actions自动构建镜像+部署验证

1. 项目概述

DeOldify是一款基于深度学习技术的图像上色工具，能够将黑白照片自动转换为彩色照片。本文将详细介绍如何通过GitHub Actions构建完整的CI/CD流水线，实现DeOldify服务的自动化构建、测试和部署。

1.1 技术架构

• 核心模型：基于U-Net架构的DeOldify深度学习模型
• 服务框架：Flask + Gunicorn
• 容器化：Docker镜像
• CI/CD工具：GitHub Actions
• 部署平台：支持Kubernetes、ECS等主流容器平台

2. CI/CD流水线设计

2.1 整体流程

1. 代码提交：开发者推送代码到GitHub仓库
2. 自动触发：GitHub Actions检测到变更并启动流水线
3. 构建阶段：构建Docker镜像并推送到镜像仓库
4. 测试阶段：运行自动化测试验证服务功能
5. 部署阶段：将镜像部署到测试环境
6. 验证阶段：执行端到端测试验证部署结果
7. 生产发布：人工审核后发布到生产环境

2.2 关键组件

• workflow文件：.github/workflows/deoldify-ci-cd.yml
• Dockerfile：定义容器构建规范
• 测试脚本：包含单元测试和集成测试
• 部署脚本：Kubernetes/ECS部署配置

3. 实现步骤详解

3.1 准备工作

3.1.1 项目结构

deoldify-service/ ├── app/ │ ├── main.py # Flask应用入口 │ ├── model.py # 模型加载与推理 │ └── utils.py # 工具函数 ├── tests/ │ ├── unit/ # 单元测试 │ └── integration/ # 集成测试 ├── Dockerfile # 容器构建文件 ├── requirements.txt # Python依赖 └── .github/ └── workflows/ └── deoldify-ci-cd.yml # CI/CD工作流

3.1.2 Dockerfile配置

FROM python:3.9-slim WORKDIR /app # 安装系统依赖 RUN apt-get update && apt-get install -y \ libgl1-mesa-glx \ libglib2.0-0 \ && rm -rf /var/lib/apt/lists/* # 复制项目文件 COPY requirements.txt . COPY . . # 安装Python依赖 RUN pip install --no-cache-dir -r requirements.txt # 下载模型 RUN python -c "from app.model import load_model; load_model()" # 暴露端口 EXPOSE 7860 # 启动命令 CMD ["gunicorn", "--bind", "0.0.0.0:7860", "--workers", "4", "app.main:app"]

3.2 GitHub Actions配置

3.2.1 基础工作流

name: DeOldify CI/CD Pipeline on: push: branches: [ main ] pull_request: branches: [ main ] env: IMAGE_NAME: deoldify-service REGISTRY: ghcr.io REPOSITORY: ${{ github.repository }} jobs: build-and-test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up Python uses: actions/setup-python@v4 with: python-version: '3.9' - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt pip install pytest - name: Run unit tests run: | pytest tests/unit -v - name: Build Docker image run: docker build -t $IMAGE_NAME . - name: Run integration tests run: | docker run -d -p 7860:7860 --name deoldify-test $IMAGE_NAME sleep 10 # 等待服务启动 pytest tests/integration -v docker stop deoldify-test docker rm deoldify-test

3.2.2 镜像构建与推送

build-and-push: needs: build-and-test runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' steps: - uses: actions/checkout@v3 - name: Log in to GitHub Container Registry uses: docker/login-action@v2 with: registry: ${{ env.REGISTRY }} username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - name: Build and push Docker image uses: docker/build-push-action@v4 with: context: . push: true tags: | ${{ env.REGISTRY }}/${{ env.REPOSITORY }}:latest ${{ env.REGISTRY }}/${{ env.REPOSITORY }}:${{ github.sha }}

3.2.3 部署到测试环境

deploy-to-test: needs: build-and-push runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v2 with: aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: us-east-1 - name: Deploy to ECS run: | # 更新ECS任务定义 aws ecs register-task-definition \ --family deoldify-test \ --network-mode awsvpc \ --execution-role-arn ${{ secrets.ECS_TASK_ROLE }} \ --container-definitions '[ { "name": "deoldify", "image": "${{ env.REGISTRY }}/${{ env.REPOSITORY }}:${{ github.sha }}", "essential": true, "portMappings": [ { "containerPort": 7860, "hostPort": 7860 } ], "environment": [ { "name": "ENVIRONMENT", "value": "test" } ] } ]' # 更新服务 aws ecs update-service \ --cluster deoldify-cluster \ --service deoldify-test \ --task-definition deoldify-test \ --force-new-deployment

3.3 测试验证

3.3.1 单元测试示例

# tests/unit/test_model.py import pytest from app.model import colorize_image from PIL import Image import numpy as np def test_colorize_image(): # 创建测试用的黑白图像 test_img = Image.new('L', (100, 100), color=128) # 调用上色函数 colored_img = colorize_image(test_img) # 验证结果 assert colored_img.mode == 'RGB' assert colored_img.size == (100, 100) assert np.array(colored_img).shape == (100, 100, 3)

3.3.2 集成测试示例

# tests/integration/test_api.py import requests import base64 from io import BytesIO from PIL import Image BASE_URL = "http://localhost:7860" def test_health_check(): response = requests.get(f"{BASE_URL}/health") assert response.status_code == 200 assert response.json()["status"] == "healthy" def test_colorize_api(): # 创建测试图像 test_img = Image.new('L', (100, 100), color=128) img_byte_arr = BytesIO() test_img.save(img_byte_arr, format='JPEG') img_byte_arr.seek(0) # 调用API files = {'image': ('test.jpg', img_byte_arr, 'image/jpeg')} response = requests.post(f"{BASE_URL}/colorize", files=files) # 验证响应 assert response.status_code == 200 result = response.json() assert result["success"] is True assert "output_img_base64" in result # 验证图像 img_data = base64.b64decode(result["output_img_base64"]) img = Image.open(BytesIO(img_data)) assert img.mode == 'RGB' assert img.size == (100, 100)

4. 高级配置与优化

4.1 多阶段构建优化

# 构建阶段 FROM python:3.9-slim as builder WORKDIR /app COPY requirements.txt . RUN apt-get update && apt-get install -y gcc python3-dev RUN pip install --user -r requirements.txt # 运行时阶段 FROM python:3.9-slim WORKDIR /app COPY --from=builder /root/.local /root/.local COPY . . ENV PATH=/root/.local/bin:$PATH ENV PYTHONPATH=/app # 其余配置与之前相同...

4.2 缓存优化

# 在GitHub Actions中添加缓存步骤 - name: Cache Python dependencies uses: actions/cache@v3 with: path: ~/.cache/pip key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }} restore-keys: | ${{ runner.os }}-pip-

4.3 安全扫描

- name: Scan for vulnerabilities uses: aquasecurity/trivy-action@master with: image-ref: ${{ env.IMAGE_NAME }} format: 'table' exit-code: '1' ignore-unfixed: true severity: 'CRITICAL,HIGH'

5. 部署验证

5.1 自动化验证脚本

#!/bin/bash # 验证服务健康状态 health_check() { local retries=5 local delay=5 for ((i=1; i<=retries; i++)); do response=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:7860/health) if [ "$response" -eq 200 ]; then echo "Service is healthy" return 0 fi echo "Attempt $i: Service not ready, waiting $delay seconds..." sleep $delay done echo "Health check failed after $retries attempts" return 1 } # 验证图像上色功能 test_colorize() { local test_image="test.jpg" local output_file="output.jpg" # 创建测试图像 convert -size 100x100 xc:gray -quality 90 "$test_image" # 调用API response=$(curl -s -X POST -F "image=@$test_image" http://localhost:7860/colorize) # 检查响应 if echo "$response" | jq -e '.success == true' >/dev/null; then echo "Colorize API responded successfully" # 保存结果图像 echo "$response" | jq -r '.output_img_base64' | base64 -d > "$output_file" if [ -s "$output_file" ]; then echo "Colorized image saved to $output_file" return 0 else echo "Failed to save colorized image" return 1 fi else echo "Colorize API failed: $response" return 1 fi } # 主验证流程 main() { if health_check; then if test_colorize; then echo "Deployment validation PASSED" exit 0 fi fi echo "Deployment validation FAILED" exit 1 } main

5.2 监控与告警

- name: Setup monitoring run: | # 安装Prometheus exporter pip install prometheus-flask-exporter # 修改应用代码添加监控端点 echo "from prometheus_flask_exporter import PrometheusMetrics" >> app/main.py echo "metrics = PrometheusMetrics(app)" >> app/main.py

6. 总结

通过本文介绍的CI/CD流水线，我们实现了DeOldify服务的自动化构建、测试和部署流程。这套方案具有以下优势：

1. 自动化程度高：从代码提交到部署验证全流程自动化
2. 质量保障：包含多层次的测试验证
3. 可扩展性强：易于扩展到其他深度学习服务
4. 安全可靠：包含安全扫描和健康检查
5. 高效部署：支持快速迭代和回滚

实际部署时，可以根据具体需求调整各阶段配置，如增加性能测试、蓝绿部署等高级特性，进一步提升交付质量和效率。

获取更多AI镜像

想探索更多AI镜像和应用场景？访问 CSDN星图镜像广场，提供丰富的预置镜像，覆盖大模型推理、图像生成、视频生成、模型微调等多个领域，支持一键部署。