Полная настройка CI/CD для Python API: тесты, линтеры, деплой.
Полная настройка CI/CD для Python API: от тестов и линтеров до деплоя на сервер или в Kubernetes.
В этом кейсе мы настроим полный CI/CD пайплайн для Python API на FastAPI. Пайплайн будет включать:
project/
├── .github/
│ └── workflows/
│ ├── ci.yml # CI: тесты, линтеры
│ └── deploy.yml # CD: сборка и деплой
├── app/
│ ├── __init__.py
│ ├── main.py # FastAPI приложение
│ ├── models.py # Pydantic модели
│ └── routers/ # API роутеры
├── tests/
│ ├── __init__.py
│ ├── conftest.py # pytest fixtures
│ └── test_api.py # тесты API
├── Dockerfile
├── docker-compose.yml
├── pyproject.toml # зависимости (poetry)
├── poetry.lock
├── .pre-commit-config.yaml
└── README.md
Создайте pytest.ini или настройте в pyproject.toml:
# pytest.ini
[pytest]
testpaths = tests
python_files = test_*.py
python_functions = test_*
addopts = -v --tb=short --strict-markers
markers =
slow: slow running tests
integration: integration tests requiring database# .github/workflows/ci.yml
name: CI
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test_db
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Install dependencies
run: |
pip install poetry
poetry install --with dev
- name: Run linting
run: |
poetry run black --check app/
poetry run flake8 app/
poetry run mypy app/
- name: Run tests with coverage
env:
DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test_db
run: |
poetry run pytest \
--cov=app \
--cov-report=xml \
--cov-report=term-missing \
--cov-fail-under=80
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
with:
files: ./coverage.xml
fail_ci_if_error: falseКэширование зависимостей:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip' # Автоматическое кэширование pipGitHub Actions автоматически кэширует pip-кэш на основе хэша requirements.txt или poetry.lock. Это ускоряет установку зависимостей на 30-60%.
Запуск линтеров:
# black — форматирование кода
poetry run black --check app/
# flake8 — стиль кода и ошибки
poetry run flake8 app/ --max-line-length=100 --ignore=E501,W503
# mypy — статический анализ типов
poetry run mypy app/ --strictПокрытие кода:
poetry run pytest \
--cov=app \ # Измерять покрытие модуля app
--cov-report=xml \ # XML отчёт для Codecov
--cov-report=term-missing \ # Вывод пропущенных строк
--cov-fail-under=80 # Fail если покрытие < 80%# Stage 1: Build
FROM python:3.12-slim as builder
WORKDIR /app
# Устанавливаем poetry
RUN pip install --no-cache-dir poetry
# Копируем зависимости
COPY pyproject.toml poetry.lock ./
# Устанавливаем зависимости в виртуальное окружение
RUN poetry export -f requirements.txt --output requirements.txt
RUN pip install --user --no-cache-dir -r requirements.txt
# Stage 2: Runtime
FROM python:3.12-slim as runtime
WORKDIR /app
# Создаём non-root пользователя
RUN useradd --create-home --shell /bin/bash appuser
# Копируем зависимости из builder
COPY /root/.local /home/appuser/.local
COPY /app/requirements.txt ./
# Копируем приложение
COPY app/ ./app/
# Устанавливаем PATH
ENV PATH=/home/appuser/.local/bin:$PATH
ENV PYTHONPATH=/app
# Переключаемся на non-root пользователя
USER appuser
# Экспортируем порт
EXPOSE 8000
# Health check
HEALTHCHECK \
CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
# Запуск приложения
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]| До оптимизации | После оптимизации |
|---|---|
| Размер: ~1.2GB | Размер: ~180MB |
| Содержит poetry, build tools | Только runtime зависимости |
| Root пользователь | Non-root пользователь |
# .github/workflows/build-and-push.yml
name: Build and Push
on:
push:
branches: [main]
tags:
- 'v*'
pull_request:
branches: [main]
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=sha,prefix=sha-
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=maxtags: |
type=ref,event=branch # main → main
type=ref,event=pr # pr-42 → pr-42
type=semver,pattern={{version}} # v1.2.3 → 1.2.3
type=sha,prefix=sha- # abc123 → sha-abc123ssh-keygen -t ed25519 -C "github-actions" -f github_actions_keyДобавьте публичный ключ на сервер в ~/.ssh/authorized_keys
Добавьте приватный ключ в GitHub Secrets как SSH_PRIVATE_KEY
# .github/workflows/deploy-ssh.yml
name: Deploy to Server
on:
push:
branches: [main]
workflow_dispatch:
jobs:
deploy:
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v4
- name: Deploy via SSH
uses: appleboy/ssh-action@v1.0.3
with:
host: ${{ secrets.SERVER_HOST }}
username: ${{ secrets.SERVER_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
port: 22
script: |
set -e
# Pull latest changes
cd /opt/myapp
# Login to GHCR
echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
# Pull new image
docker pull ghcr.io/${{ github.repository }}:main
# Stop old container
docker stop myapp || true
docker rm myapp || true
# Run new container
docker run -d \
--name myapp \
--restart unless-stopped \
-p 8000:8000 \
-e DATABASE_URL=${{ secrets.DATABASE_URL }} \
-e API_KEY=${{ secrets.API_KEY }} \
ghcr.io/${{ github.repository }}:main
# Cleanup old images
docker image prune -f --filter "until=24h"
# Health check
sleep 5
curl -f http://localhost:8000/health || exit 1env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
API_KEY: ${{ secrets.API_KEY }}Важно: GitHub автоматически маскирует секреты в логах. Но избегайте команд типа printenv или echo $SECRET, которые могут раскрыть значения.
helm/
├── Chart.yaml
├── values.yaml
├── values-production.yaml
└── templates/
├── deployment.yaml
├── service.yaml
├── ingress.yaml
├── configmap.yaml
└── secret.yaml
replicaCount: 3
image:
repository: ghcr.io/myorg/myapp
tag: latest
pullPolicy: Always
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
autoscaling:
enabled: true
minReplicas: 3
maxReplicas: 10
targetCPUUtilizationPercentage: 80
ingress:
enabled: true
className: nginx
hosts:
- host: api.example.com
paths:
- path: /
pathType: Prefix
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8000
initialDelaySeconds: 5
periodSeconds: 5# .github/workflows/deploy-k8s.yml
name: Deploy to Kubernetes
on:
push:
branches: [main]
tags:
- 'v*'
jobs:
deploy:
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v4
- name: Set up Helm
uses: azure/setup-helm@v3
with:
version: v3.14.0
- name: Configure kubectl
uses: azure/k8s-set-context@v3
with:
method: kubeconfig
kubeconfig: ${{ secrets.KUBE_CONFIG }}
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Deploy with Helm
run: |
helm upgrade --install myapp ./helm \
--namespace production \
--create-namespace \
--values helm/values-production.yaml \
--set image.tag=${{ github.ref_name }} \
--wait \
--timeout 5m \
--atomic
- name: Verify deployment
run: |
kubectl rollout status deployment/myapp -n production
kubectl get pods -n production -l app=myapp--wait — ждать готовности всех ресурсов--timeout 5m — таймаут ожидания--atomic — автоматический откат при неудаче--set image.tag=... — переопределение тега образаДля минимального downtime используйте blue-green стратегию:
# .github/workflows/deploy-blue-green.yml
name: Blue-Green Deploy
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy new version (green)
uses: appleboy/ssh-action@v1.0.3
with:
host: ${{ secrets.SERVER_HOST }}
username: ${{ secrets.SERVER_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
script: |
# Deploy green (new version)
docker run -d \
--name myapp-green \
--restart unless-stopped \
-p 8001:8000 \
-e DATABASE_URL=${{ secrets.DATABASE_URL }} \
ghcr.io/${{ github.repository }}:main
# Wait for health check
sleep 10
curl -f http://localhost:8001/health || exit 1
- name: Switch traffic to green
uses: appleboy/ssh-action@v1.0.3
with:
host: ${{ secrets.SERVER_HOST }}
username: ${{ secrets.SERVER_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
script: |
# Update nginx config to point to green
sed -i 's/8000/8001/g' /etc/nginx/conf.d/myapp.conf
nginx -s reload
# Stop old (blue) container
docker stop myapp-blue || true
docker rm myapp-blue || true
# Rename green to blue for next iteration
docker rename myapp-green myapp-blueOIDC устраняет необходимость хранить долгосрочные секреты:
# .github/workflows/deploy-aws-oidc.yml
name: Deploy to AWS with OIDC
on:
push:
branches: [main]
permissions:
id-token: write
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789:role/github-actions-role
aws-region: us-east-1
- name: Deploy to ECS
run: |
aws ecs update-service \
--cluster my-cluster \
--service my-service \
--force-new-deployment# app/main.py
import sentry_sdk
from sentry_sdk.integrations.fastapi import FastApiIntegration
sentry_sdk.init(
dsn=os.getenv("SENTRY_DSN"),
integrations=[FastApiIntegration()],
traces_sample_rate=1.0,
environment=os.getenv("ENVIRONMENT", "development"),
)# app/routers/health.py
from fastapi import APIRouter
from datetime import datetime
router = APIRouter()
@router.get("/health")
async def health_check():
return {"status": "healthy", "timestamp": datetime.utcnow()}
@router.get("/ready")
async def readiness_check():
# Проверка подключения к БД
try:
await database.fetch_one("SELECT 1")
return {"status": "ready"}
except Exception as e:
raise HTTPException(status_code=503, detail=str(e))Полный CI/CD пайплайн для Python API включает:
CI (Continuous Integration)
CD (Continuous Delivery/Deployment)
Безопасность
Надёжность
--atomic в HelmВопросы ещё не добавлены
Вопросы для этой подтемы ещё не добавлены.