ci: 配置 master 自动验证与服务器部署

deploy/README.md

@@ -0,0 +1,41 @@
+# Veyra 自动部署
+
+开发在 `dev` 分支进行，不直接提交或推送 `master`。Gitea 中的 `master` 保护规则禁用直接推送、强制推送和管理员绕过；通过合并请求进入 `master`。流水线只监听 `master` 的 push，合并提交也会触发；不监听 `dev` 或 PR 事件。因此合并前由本地检查把关，流水线在合并后验证，验证失败不部署。
+
+## 当前部署目标
+
+- Gitea：`https://git.sundynix.cn/Blizzard/Veyra`，使用已有全局 Runner 的 `ubuntu-latest` 标签。
+- SSH：`root@192.168.100.132`；目录：`/home/workspace/sundynix-veyra`。
+- 默认管理端与 API：`http://192.168.100.132:8080`，同一个镜像、同源服务。
+- 应用配置：服务器 `configs/.env`，权限 600；日志：服务器 `logs/`，由容器 UID 10001 写入并轮转。
+- PostgreSQL、Redis 使用已有内网服务；不额外部署数据库容器，不覆盖数据、身份或加密主密钥。
+
+仓库 Actions 已启用，部署使用 `DEPLOY_PASSWORD`、`DEPLOY_KNOWN_HOSTS` 两个仓库 Secret。服务器地址、用户和目录在工作流中明确配置；SSH 严格校验已知服务器公钥。可选 `VEYRA_ENV_FILE` 仅用于新服务器首次创建配置，已有文件永不覆盖。当前服务器已通过 SSH 安全初始化，无需设置此可选 Secret。
+
+应用环境文件必须采用无外层引号的 `KEY=VALUE` 格式。Compose 使用 `env_file.format: raw`，保留密码中的 `$`、`#` 等字符，不把应用配置作为 Compose 插值文件。需要 Docker、Compose >=2.30、bash、flock、gzip、sha256sum；当前服务器 Compose 2.36.2 满足要求。
+
+## 流水线
+
+1. 使用 Gitea job token 下载当前提交归档，与参考项目保持一致，避免 Git HTTP pack 检出问题。
+2. Docker 内检查 App 类型、12 项客户端测试、iOS/Android Hermes 导出；管理端类型检查、构建、Go embed；gofmt、go vet 与 Go race 测试。构建阶段使用已验证可达的 `https://goproxy.cn,direct` 下载 Go 依赖，保留默认 checksum 校验；运行镜像不包含此构建设置。PostgreSQL 集成测试使用独立、用后清理的 CI 容器，不访问开发库。部署脚本也有成功、校验失败、迁移失败、健康失败回滚、配置保留测试。
+3. 所有检查通过才构建 `sundynix/veyra:<完整提交 SHA>`，打包压缩并生成 SHA-256 校验文件，通过 SSH 传入服务器的独立发布目录。
+4. 服务器加锁串行发布，校验压缩包、加载镜像，执行同一可执行文件的 `-command db-create` 和 `-command db-migrate`，再启动 Compose。
+5. `/readyz` 检查 PostgreSQL 与 Redis。新版本就绪后才更新 `.current-image`；失败则恢复上一版镜像与 Compose，首次部署失败则停止应用。迁移失败不替换运行中的服务。不会自动回退数据库迁移；后续迁移应保持与上一版应用兼容，破坏性迁移需单独设计。
+
+回测 Python 引擎和管理端已包含在运行镜像中；App 导出仅用于 CI 检查，不是 TestFlight、IPA 发布或 Expo OTA 更新。
+
+目前认证与行情、交易研究仍处于开发阶段，服务器沿用本地 `APP_ENV=local` 配置以支持现有开发身份。此部署不代表生产认证、安全与正式行情授权已完成；改为生产环境前需完成正式登录流程。
+
+## 本地检查
+
+```sh
+bash deploy/check.sh                  # 需要运行中的 Docker；完成容器内全部检查
+python3 deploy/tests/test_release.py   # 无需 Docker，检查发布决策与配置保留
+npm --prefix web/admin run build
+go vet ./...
+go build -trimpath -o bin/veyra .
+```
+
+服务器排查使用 `docker compose --env-file /dev/null -f docker-compose.yml`，同时将 `VEYRA_IMAGE` 设为 `.current-image` 中的值。不要将含凭据的 `docker compose config`、环境文件或容器完整 inspect 输出到日志；配置语法检查使用 `config --quiet`。
+
+配置语义参考 [Gitea Actions](https://docs.gitea.com/usage/actions/) 与 [Docker Compose env_file](https://docs.docker.com/reference/compose-file/services/#env_file)。

deploy/bootstrap-config.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+set -euo pipefail
+root="${1:?deployment directory required}"
+[[ "$root" == /* && "$root" != / && "$root" != *[!a-zA-Z0-9_/-]* ]] || exit 2
+umask 077
+mkdir -p "$root/configs"
+tmp="$(mktemp "$root/configs/.env.XXXXXX")"
+trap 'rm -f "$tmp"' EXIT
+cat > "$tmp"
+[[ -s "$tmp" ]] || exit 1
+# Atomic create-if-absent. Existing configuration and encryption key win.
+ln "$tmp" "$root/configs/.env" 2>/dev/null || test -s "$root/configs/.env"

deploy/check.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+set -euo pipefail
+run_id="veyra-ci-$(date +%s)-${RANDOM}"
+network="$run_id"
+postgres="$run_id-postgres"
+cleanup() {
+  docker rm -f "$postgres" >/dev/null 2>&1 || true
+  docker network rm "$network" >/dev/null 2>&1 || true
+  docker image rm "veyra-mobile-check:$run_id" "veyra-test:$run_id" >/dev/null 2>&1 || true
+}
+trap cleanup EXIT
+docker build --target mobile-check -t "veyra-mobile-check:$run_id" .
+docker build --target test -t "veyra-test:$run_id" .
+docker run --rm --entrypoint python3 "veyra-test:$run_id" /src/deploy/tests/test_release.py
+docker network create "$network" >/dev/null
+docker run -d --name "$postgres" --network "$network" --network-alias postgres \
+  -e POSTGRES_USER=veyra_test -e POSTGRES_PASSWORD=test_only -e POSTGRES_DB=veyra_test postgres:17 >/dev/null
+ready=false
+for ((i=0; i<60; i++)); do
+  if docker exec "$postgres" pg_isready -U veyra_test -d veyra_test >/dev/null 2>&1; then ready=true; break; fi
+  sleep 1
+done
+if [[ "$ready" != true ]]; then echo 'CI PostgreSQL did not become ready' >&2; exit 1; fi
+docker run --rm --network "$network" \
+  -e VEYRA_TEST_DATABASE_URL='postgres://veyra_test:test_only@postgres:5432/veyra_test?sslmode=disable' \
+  "veyra-test:$run_id"

deploy/docker-compose.yml

@@ -0,0 +1,25 @@
+name: veyra
+services:
+  app:
+    image: ${VEYRA_IMAGE:?VEYRA_IMAGE is required}
+    restart: unless-stopped
+    init: true
+    env_file:
+      - path: ./configs/.env
+        format: raw
+    environment:
+      HTTP_ADDR: 0.0.0.0:8080
+      LOG_DIR: logs
+      PYTHON_EXECUTABLE: python3
+    ports:
+      - "${VEYRA_BIND_ADDR:-0.0.0.0}:${VEYRA_PORT:-8080}:8080"
+    volumes:
+      - ./logs:/app/logs
+    read_only: true
+    tmpfs:
+      - /tmp:size=64m,mode=1777
+    security_opt:
+      - no-new-privileges:true
+    cap_drop:
+      - ALL
+    stop_grace_period: 30s

deploy/release.sh

@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+# Run on the deployment host: release.sh /home/workspace/veyra <commit SHA>
+set -euo pipefail
+root="${1:?deployment directory required}"
+sha="${2:?commit SHA required}"
+[[ "$root" == /* && "$root" != / && "$root" != *[!a-zA-Z0-9_/-]* ]] || exit 2
+[[ "$sha" =~ ^[0-9a-f]{40}$ ]] || exit 2
+release="$root/releases/$sha"
+image="sundynix/veyra:$sha"
+cd "$root"
+# Serialize releases even when several master commits queue up.
+exec 9> .deploy.lock
+flock -w 600 9
+if [[ ! -s configs/.env ]]; then echo 'Missing server configs/.env (or VEYRA_ENV_FILE bootstrap secret)' >&2; exit 1; fi
+(cd "$release" && sha256sum -c image.tar.gz.sha256)
+gunzip -c "$release/image.tar.gz" | docker load
+mkdir -p logs
+docker run --rm --user 0:0 --entrypoint sh -v "$root/logs:/app/logs" "$image" -c 'chown 10001:10001 /app/logs'
+previous=''
+if [[ -s .current-image ]]; then
+  previous="$(cat .current-image)"
+  [[ "$previous" =~ ^sundynix/veyra:[0-9a-f]{40}$ ]] || exit 2
+fi
+# Keep the previous Compose file as well as the previously healthy image.
+if [[ -f docker-compose.yml ]]; then cp docker-compose.yml "$release/previous-compose.yml"; fi
+cp "$release/docker-compose.yml" docker-compose.yml
+export VEYRA_IMAGE="$image"
+# Use an empty interpolation file: application secrets are only read via raw env_file.
+compose() { docker compose --env-file /dev/null -f "$root/docker-compose.yml" "$@"; }
+if ! compose run --rm --no-deps app -command db-create || ! compose run --rm --no-deps app -command db-migrate; then
+  if [[ -f "$release/previous-compose.yml" ]]; then cp "$release/previous-compose.yml" docker-compose.yml; fi
+  echo 'Database preparation failed; existing application was not replaced' >&2
+  exit 1
+fi
+if compose up -d --no-build --wait --wait-timeout 120; then
+  printf '%s\n' "$image" > .current-image.tmp
+  mv .current-image.tmp .current-image
+  rm -f "$release/image.tar.gz"
+  echo "Veyra deployed: $sha"
+else
+  echo 'New application failed readiness; restoring previous application' >&2
+  if [[ -n "$previous" && -f "$release/previous-compose.yml" ]]; then
+    cp "$release/previous-compose.yml" docker-compose.yml
+    export VEYRA_IMAGE="$previous"
+    compose up -d --no-build --wait --wait-timeout 120
+  else
+    compose stop app
+  fi
+  # Schema migrations are never automatically undone.
+  exit 1
+fi

deploy/tests/test_release.py

@@ -0,0 +1,112 @@
+"""Verify deployment decisions without connecting to a server or Docker daemon."""
+import gzip
+import hashlib
+import os
+from pathlib import Path
+import subprocess
+import tempfile
+import unittest
+
+SCRIPTS = Path(__file__).resolve().parents[1]
+SHA = "a" * 40
+OLD = "sundynix/veyra:" + "b" * 40
+
+class ReleaseTests(unittest.TestCase):
+    def setUp(self):
+        self.temp = tempfile.TemporaryDirectory(prefix="veyra-release-")
+        self.addCleanup(self.temp.cleanup)
+        self.root = Path(self.temp.name)
+        (self.root / "configs").mkdir()
+        (self.root / "configs/.env").write_text("TEST_SETTING=unchanged\n")
+        self.release = self.root / "releases" / SHA
+        self.release.mkdir(parents=True)
+        archive = gzip.compress(b"fake image")
+        (self.release / "image.tar.gz").write_bytes(archive)
+        digest = hashlib.sha256(archive).hexdigest()
+        (self.release / "image.tar.gz.sha256").write_text(digest + "  image.tar.gz\n")
+        (self.release / "docker-compose.yml").write_text("new compose\n")
+        (self.root / "docker-compose.yml").write_text("old compose\n")
+        (self.root / ".current-image").write_text(OLD + "\n")
+        tools = self.root / "tools"
+        tools.mkdir()
+        mock = tools / "docker"
+        mock.write_text(r"""#!/usr/bin/env python3
+import os, sys
+from pathlib import Path
+args = sys.argv[1:]
+with open(os.environ['MOCK_LOG'], 'a') as log:
+    log.write(os.environ.get('VEYRA_IMAGE', '') + ' ' + ' '.join(args) + '\n')
+if args and args[0] == 'load':
+    sys.stdin.buffer.read()
+if '-command' in args and args[-1] == 'db-migrate' and os.environ.get('FAIL_MIGRATION'):
+    sys.exit(1)
+if 'up' in args and os.environ.get('FAIL_READINESS') and os.environ.get('VEYRA_IMAGE') != os.environ['OLD_IMAGE']:
+    sys.exit(1)
+""")
+        mock.chmod(0o755)
+        # macOS has no flock; emulate it for this single-process decision test.
+        if not __import__('shutil').which('flock'):
+            (tools / 'flock').write_text('#!/bin/sh\nexit 0\n')
+            (tools / 'flock').chmod(0o755)
+        if not __import__('shutil').which('sha256sum'):
+            checksum = tools / 'sha256sum'
+            checksum.write_text("""#!/usr/bin/env python3
+import hashlib, sys
+from pathlib import Path
+for line in Path(sys.argv[2]).read_text().splitlines():
+    digest, name = line.split('  ', 1)
+    if hashlib.sha256(Path(name).read_bytes()).hexdigest() != digest:
+        sys.exit(1)
+""")
+            checksum.chmod(0o755)
+        self.env = dict(os.environ, PATH=str(tools) + os.pathsep + os.environ['PATH'],
+                        MOCK_LOG=str(self.root / 'commands'), OLD_IMAGE=OLD)
+
+    def run_release(self, **flags):
+        return subprocess.run(['bash', str(SCRIPTS / 'release.sh'), str(self.root), SHA],
+                              env=dict(self.env, **flags), capture_output=True, text=True)
+
+    def test_success_promotes_exact_commit(self):
+        result = self.run_release()
+        self.assertEqual(result.returncode, 0, result.stderr)
+        self.assertEqual((self.root / '.current-image').read_text().strip(), 'sundynix/veyra:' + SHA)
+        self.assertEqual((self.root / 'configs/.env').read_text(), 'TEST_SETTING=unchanged\n')
+        self.assertFalse((self.release / 'image.tar.gz').exists())
+        calls = (self.root / 'commands').read_text()
+        self.assertLess(calls.index('db-migrate'), calls.index(' up '))
+
+    def test_failed_readiness_restores_previous_application(self):
+        self.assertNotEqual(self.run_release(FAIL_READINESS='1').returncode, 0)
+        self.assertEqual((self.root / '.current-image').read_text().strip(), OLD)
+        self.assertEqual((self.root / 'docker-compose.yml').read_text(), 'old compose\n')
+        self.assertIn(OLD + ' compose', (self.root / 'commands').read_text())
+
+    def test_failed_migration_does_not_replace_running_application(self):
+        self.assertNotEqual(self.run_release(FAIL_MIGRATION='1').returncode, 0)
+        self.assertNotIn(' up ', (self.root / 'commands').read_text())
+        self.assertEqual((self.root / 'docker-compose.yml').read_text(), 'old compose\n')
+
+    def test_invalid_archive_never_loads_image(self):
+        (self.release / 'image.tar.gz').write_bytes(b'corrupt')
+        self.assertNotEqual(self.run_release().returncode, 0)
+        self.assertFalse((self.root / 'commands').exists())
+
+    def test_missing_configuration_fails_before_image_load(self):
+        (self.root / 'configs/.env').unlink()
+        self.assertNotEqual(self.run_release().returncode, 0)
+        self.assertFalse((self.root / 'commands').exists())
+
+    def test_bootstrap_preserves_existing_configuration(self):
+        for _ in range(2):
+            result = subprocess.run(['bash', str(SCRIPTS / 'bootstrap-config.sh'), str(self.root)],
+                                    input='TEST_SETTING=new\n', capture_output=True, text=True)
+            self.assertEqual(result.returncode, 0, result.stderr)
+        self.assertEqual((self.root / 'configs/.env').read_text(), 'TEST_SETTING=unchanged\n')
+        (self.root / 'configs/.env').unlink()
+        result = subprocess.run(['bash', str(SCRIPTS / 'bootstrap-config.sh'), str(self.root)],
+                                input='TEST_SETTING=new\n', capture_output=True, text=True)
+        self.assertEqual(result.returncode, 0, result.stderr)
+        self.assertEqual((self.root / 'configs/.env').stat().st_mode & 0o777, 0o600)
+
+if __name__ == '__main__':
+    unittest.main()