commit c7a02c390551b23261b92656d91e2e1cf0ee3fe6 Author: Blizzard Date: Wed Jun 10 11:00:29 2026 +0800 feat: 初始化 sundynix-agentix 分层式 AI Agent 平台脚手架 5 层 + 1 条 NATS 零拷贝消息总线的 monorepo（Monolith First → Microservices Morph B）。 纵向主干（任务流 + Token 流回流）已真实跑通，横向各层能力为带注释的桩。 已贯通（real code）: - sundynix-shared: 共享契约 + JetStream/core NATS 真实收发(bus) + 内嵌 NATS(devnats) + e2e 测试 - sundynix-gateway: Gin 接入 + DSL 解析组装 + NATS Publish + SSE 流式输出 - sundynix-dispatcher: NATS 消费 + Eino Orchestrator 流式回流 + 熔断器 + LLM Pool 占位流式 - 链路: HTTP POST → DSL → sundynix.tasks.* → Dispatcher → Token 经 sundynix.streams. 回流 → SSE - 基础设施: docker-compose(nats/postgres/redis/neo4j/milvus) + Makefile(make demo/e2e) 待填(桩): - Eino 图编排 compose.NewGraph、LLM Pool 接 vLLM/Ollama - Gateway store 换真实 pgx/redis - sundynix-mcp-go: Bleve+Milvus+Neo4j 混合检索 / UniOffice / 外部 API - sundynix-mcp-py: gVisor 沙箱 / MinerU(PaddleOCR) / Docker 解释器 - sundynix-desktop: React Flow 画布 → DSL 导出 → SSE 展示 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..8f28b92 --- /dev/null +++ b/.gitignore @@ -0,0 +1,22 @@ +# Go +*.exe +bin/ + +# Node +node_modules/ +dist/ + +# Python +__pycache__/ +*.pyc +.venv/ + +# Wails +build/bin/ +frontend/dist/ + +# Data +data/ + +# 演示产物 +.bin/ diff --git a/.idea/.gitignore b/.idea/.gitignore new file mode 100644 index 0000000..30cf57e --- /dev/null +++ b/.idea/.gitignore @@ -0,0 +1,10 @@ +# Default ignored files +/shelf/ +/workspace.xml +# Editor-based HTTP Client requests +/httpRequests/ +# Ignored default folder with query files +/queries/ +# Datasource local storage ignored files +/dataSources/ +/dataSources.local.xml diff --git a/.idea/go.imports.xml b/.idea/go.imports.xml new file mode 100644 index 0000000..644cdf0 --- /dev/null +++ b/.idea/go.imports.xml @@ -0,0 +1,10 @@ + + + + + + \ No newline at end of file diff --git a/.idea/modules.xml b/.idea/modules.xml new file mode 100644 index 0000000..1b3d558 --- /dev/null +++ b/.idea/modules.xml @@ -0,0 +1,8 @@ + + + + + + + + \ No newline at end of file diff --git a/.idea/sundynix-agentix.iml b/.idea/sundynix-agentix.iml new file mode 100644 index 0000000..5e764c4 --- /dev/null +++ b/.idea/sundynix-agentix.iml @@ -0,0 +1,9 @@ + + + + + + + + + \ No newline at end of file diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..fee9a3a --- /dev/null +++ b/Makefile @@ -0,0 +1,36 @@ +.PHONY: infra infra-down devnats demo e2e gateway dispatcher mcp-go mcp-py desktop tidy + +infra: ## 启动基础设施 (NATS / Postgres / Redis / Milvus / Neo4j) + docker compose up -d + +infra-down: + docker compose down + +devnats: ## 启动内嵌 JetStream NATS（无 Docker 本地联调） + cd sundynix-shared && go run ./cmd/devnats + +demo: ## 一键演示 Gateway→NATS→Dispatcher 任务流（无需 Docker） + bash scripts/demo.sh + +e2e: ## 跑共享 bus 的端到端测试（内嵌 NATS） + cd sundynix-shared && go test ./bus/ -run 'TestTaskRoundTrip|TestTokenStreamRoundTrip' -v + +gateway: + cd sundynix-gateway && go run ./cmd/server + +dispatcher: + cd sundynix-dispatcher && go run ./cmd/dispatcher + +mcp-go: + cd sundynix-mcp-go && go run ./cmd/server + +mcp-py: + cd sundynix-mcp-py && python -m sundynix_mcp_py.main + +desktop: + cd sundynix-desktop && wails dev + +tidy: + cd sundynix-gateway && go mod tidy + cd sundynix-dispatcher && go mod tidy + cd sundynix-mcp-go && go mod tidy diff --git a/README.md b/README.md new file mode 100644 index 0000000..1061401 --- /dev/null +++ b/README.md @@ -0,0 +1,64 @@ +# sundynix-agentix + +分层式 AI Agent 平台 — **Monolith First → Microservices (Morph B)**。 +架构总览见 [architecture.md](architecture.md) / [architecture.png](architecture.png)。 + +## 仓库结构（Monorepo） + +| 目录 | 层 | 语言 / 技术栈 | 职责 | +|---|---|---|---| +| [`sundynix-desktop/`](sundynix-desktop) | 1 · Client | Wails + React 19 + TS + shadcn/ui | 桌面端、React Flow Agent 编排、导出 JSON DSL、LLM Wiki 面板 | +| [`sundynix-gateway/`](sundynix-gateway) | 2 · Gateway | Go · Gin | 统一接入、DSL 解析组装、计费、Guardrail、PgSQL + Redis | +| [`deploy/nats/`](deploy/nats) | 3 · Message Bus | NATS (Go) | 零拷贝骨干网：Queue + Stream | +| [`sundynix-dispatcher/`](sundynix-dispatcher) | 4 · Dispatcher | Go · Eino | 图编排、LLM Pool 调度、自动化评测、熔断降级 | +| [`sundynix-mcp-go/`](sundynix-mcp-go) | 5a · MCP Tools (I/O) | Go | MCP 网关、Wiki 混合检索(Bleve/Milvus/Neo4j)、UniOffice、外部 API | +| [`sundynix-mcp-py/`](sundynix-mcp-py) | 5b · MCP Tools (算法) | Python | MCP 网关、安全沙箱(gVisor/KataVM)、MinerU(PaddleOCR)、Docker 解释器 | +| [`sundynix-shared/`](sundynix-shared) | 共享契约 | Go | Task 数据契约、NATS subject 约定、JetStream 收发逻辑(bus)、内嵌 NATS(devnats) | + +## 核心数据流 + +1. `Gateway` 解析 DSL → **Publish** `sundynix.tasks.*` (NATS Queue) +2. `Dispatcher` 订阅任务 → `Eino` 图编排 → 调用 `LLM Pool` +3. 经 NATS 调用第 5 层 `MCP Tools` +4. 结果以零拷贝 Token Stream 经 `sundynix.streams.task_id` 回流 → SSE/WS 推给 `Client` + +## 快速开始 + +### 无 Docker — 一键验证任务流（推荐先跑这个） + +```bash +make demo # 内嵌NATS + Gateway + Dispatcher，提交一个 DSL 任务，看 Dispatcher 消费到 +make e2e # 仅跑共享 bus 的端到端测试（go test，内嵌 NATS） +``` + +`make demo` 实测输出： +``` +Gateway: POST /api/v1/tasks → task_xxx → published (seq=1) +Dispatcher: [eino] task_xxx received → streaming tokens... +SSE 客户端: event:token data:已 event:token data:编 ... event:done ← 流式闭环打通 +``` +完整链路：HTTP POST → DSL 解析 → NATS 任务队列 → Dispatcher 消费 → LLM 流式推理 +→ Token 经 `sundynix.streams.` 回流 → Gateway SSE → 客户端逐 token 收到。 + +### 完整环境（Docker） + +```bash +make infra # 启动 NATS / Postgres / Redis / Milvus / Neo4j (docker-compose) +make devnats # 或：无 Docker 时单独起内嵌 JetStream NATS +make gateway # 运行 Gateway +make dispatcher # 运行 Dispatcher +make mcp-go # 运行 Go MCP 工具服务 +make mcp-py # 运行 Python MCP 工具服务 +make desktop # 开发模式运行桌面端 (wails dev) +``` + +> Go 多模块用 `go.work` 工作区串联；`sundynix-shared` 通过各服务 go.mod 的 `replace` 指向本地。 + +## NATS Subject 约定 + +| Subject | 类型 | 说明 | +|---|---|---| +| `sundynix.tasks.*` | Queue | 分布式任务队列 | +| `sundynix.streams.` | Stream | 零拷贝 Token 字节管道 | +| `sundynix.tools.go.*` | Queue | Go MCP 工具调用 | +| `sundynix.tools.py.*` | Queue | Python MCP 工具调用 | diff --git a/architecture.html b/architecture.html new file mode 100644 index 0000000..7d74e50 --- /dev/null +++ b/architecture.html @@ -0,0 +1,286 @@ + + + + + +sundynix-agentix · 系统架构图 + + + + +

sundynix-agentix · 系统架构图

+

分层式 AI Agent 平台 — Client / Gateway / NATS 总线 / Dispatcher / MCP Tools（Monolith First → Microservices Morph B）

+ +
+ + +
+ + +
+
1. CLIENT LAYER Edge Interaction
+
+
sundynix-desktop (User Device App)
+
UI Representation LayerReact 19 + TypeScript
+
shadcn/ui + Tailwind CSSStyles / Components
+
+
React Flow CanvasAgent Orchestration
+
JSON DSL export 
+
+
LLM Wiki Management Panel
+
Wails Local Go Runtime
+
+
Strong BindingTS / Go
+
Local File System I/O
+
+
+
+ + +
+ ↕ 4. PHYSICAL COMMUNICATION
+ HTTPS / WebSocket / SSE  ·  sundynix.streams.task_id Token Stream +
+ + +
+
2. BUSINESS GATEWAY LAYER
+
+
sundynix-gateway (Gin 微服务 · 集成 Monolith First)
+
Gin 微服务 / 统一接入层
+
+
🗄️ MainDBPgSQL: Users, Billing, DSL
+
🧱 CacheDBSession / Rate Limit
+
+
+
🧱 CacheDBRedis: Session, Rate Limit
+
商业化与计费模块
+
+
+
HarnessInput / Output Guardrail
+
Task DSL Parser & Assembly
+
+
+
+ + +
+ ↓ 1. Publish Task DSL   |   4. Subscribe Stream ↑ +
+ + +
+
3. MESSAGE BUS — NATS 零拷贝骨干网
+
+
NATS Server (Go) 🗄️
+
+
NATS Queue分布式任务队列
sundynix.tasks.*
+
NATS Stream零拷贝字节管道
sundynix.streams.*
+
+
+
+ + +
↕ 4. Subscribe Stream
+ + +
+
4. AI AGENT DISPATCHER LAYER
+
+ LLM PoolvLLM / Ollama 集群 +
+
↑ EinoCore  ·  Eino 回调机制 ↓
+
+
sundynix-dispatcher (Go / Eino 集群)
+
+
Eino 图编排引擎
+
nats.go client
+
+
Harness: LLM 自动化评测
+
Harness: 熔断降级中心
+
+
+ +
+ + +
+
⟷   4. PHYSICAL COMMUNICATION   ⟷
+
+ 核心数据流（编号对应连线）
+ 1. Gateway 解析 DSL 后 Publish Task DSL → NATS Queue（sundynix.tasks.*）
+ 2. Dispatcher 通过 nats.go 订阅任务，Eino 图编排引擎驱动 LLM Pool 推理
+ 3. Dispatcher / Gateway 经 NATS 调用第 5 层 MCP 工具（Go I/O 型 + Python 算法型）
+ 4. 推理结果以 零拷贝 Token Stream 经 sundynix.streams.task_id 回流 Gateway → 经 SSE/WS 推送给 Client +
+
⟶   4. Subscribe Stream   ⟶   MCP Tools
+
+ + +
+
+
5. MICROSERVICE MCP TOOLS LAYER
+
Standalone microservices · Morph B
+ + +
+
sundynix-mcp-go (Go 微服务 · I/O 型)
+
MCP Protocol Gateway
+
LLM Wiki 搜索引擎Hybrid Search: Bleve · Milvus Go SDK · Neo4j Go Driver
+
+
🧊 MilvusVector DB
+
🔎 BleveGo Search Index
+
+
+
🕸️ Neo4jKnowledge Graph
+
🕸️ Neo4jGraph
+
+
+
📄 UniOfficeWord/Doc Rendering
+
External APIs
+
+
+ + +
+
sundynix-mcp-py (Python 微服务 · 算法型)
+
MCP Protocol Gateway
+
Harness: Secure Code SandboxgVisor / KataVM · Static Code Guard
+
MinerUMultimodal Parser: PaddleOCR
+
Docker 隔离沙箱Code Interpreter
+
+
+
+ +
+ + +
+ Legend +
Client
+
Gateway
+
NATS
+
Dispatcher
+
Go Tools
+
Python Tools
+
GinServer
+
+ + + diff --git a/architecture.md b/architecture.md new file mode 100644 index 0000000..2a25083 --- /dev/null +++ b/architecture.md @@ -0,0 +1,139 @@ +# sundynix-agentix · 系统架构图 + +分层式 AI Agent 平台 — **Monolith First → Microservices (Morph B)** 演进。 +共 **5 层 + 1 条 NATS 零拷贝消息总线**。 + +> 下方 Mermaid 图在 GitHub / VS Code(装 Mermaid 插件) / Typora / Obsidian 中可直接渲染。 + +```mermaid +flowchart TB + +%% ===================== 1. CLIENT ===================== +subgraph CLIENT["1 · CLIENT LAYER (Edge Interaction)"] + direction TB + subgraph DESKTOP["sundynix-desktop (User Device App)"] + UI["UI Representation Layer
React 19 + TypeScript"] + SHAD["shadcn/ui + Tailwind CSS
Styles / Components"] + RF["React Flow Canvas
Agent Orchestration"] + DSL["JSON DSL export"] + WIKI["LLM Wiki Management Panel"] + WAILS["Wails Local Go Runtime"] + BIND["Strong Binding TS/Go"] + FS["Local File System I/O"] + UI --> SHAD --> RF --> DSL + RF --> WIKI --> WAILS + WAILS --> BIND + WAILS --> FS + end +end + +%% ===================== 2. GATEWAY ===================== +subgraph GATEWAY["2 · BUSINESS GATEWAY LAYER"] + direction TB + subgraph GW["sundynix-gateway (Gin 微服务 · Monolith First)"] + GIN["Gin 微服务 / 统一接入层"] + MAINDB[("MainDB
PgSQL: Users, Billing, DSL")] + CACHE1[("CacheDB
Session / Rate Limit")] + CACHE2[("CacheDB
Redis: Session, Rate Limit")] + BILL["商业化与计费模块"] + GUARD["Harness: Input/Output Guardrail"] + PARSER["Task DSL Parser & Assembly"] + GIN --> MAINDB + GIN --> CACHE1 + GIN --> CACHE2 + GIN --> BILL + GIN --> GUARD + GIN --> PARSER + end +end + +%% ===================== 3. NATS ===================== +subgraph NATS["3 · MESSAGE BUS — NATS 零拷贝骨干网"] + direction LR + QUEUE["NATS Queue
分布式任务队列
sundynix.tasks.*"] + STREAM["NATS Stream
零拷贝字节管道
sundynix.streams.*"] +end + +%% ===================== 4. DISPATCHER ===================== +subgraph DISPATCHER["4 · AI AGENT DISPATCHER LAYER"] + direction TB + LLMPOOL[("LLM Pool
vLLM / Ollama 集群")] + subgraph DISP["sundynix-dispatcher (Go / Eino 集群)"] + EINO["Eino 图编排引擎"] + NATSGO["nats.go client"] + EVAL["Harness: LLM 自动化评测"] + FUSE["Harness: 熔断降级中心"] + EINO <--> NATSGO + end + LLMPOOL <-->|"EinoCore · Eino 回调机制"| EINO +end + +%% ===================== 5. MCP TOOLS ===================== +subgraph TOOLS["5 · MICROSERVICE MCP TOOLS LAYER (Morph B)"] + direction TB + subgraph MCPGO["sundynix-mcp-go (Go 微服务 · I/O 型)"] + GGW["MCP Protocol Gateway"] + SEARCH["LLM Wiki 搜索引擎
Hybrid: Bleve · Milvus Go SDK · Neo4j Go Driver"] + MILVUS[("Milvus · Vector DB")] + BLEVE["Bleve · Go Search Index"] + NEO1[("Neo4j · Knowledge Graph")] + NEO2[("Neo4j · Graph")] + UNI["UniOffice · Word/Doc Rendering"] + EXT["External APIs"] + GGW --> SEARCH --> MILVUS & BLEVE & NEO1 & NEO2 + GGW --> UNI + GGW --> EXT + end + subgraph MCPPY["sundynix-mcp-py (Python 微服务 · 算法型)"] + PGW["MCP Protocol Gateway"] + SANDBOX["Harness: Secure Code Sandbox
gVisor / KataVM · Static Code Guard"] + MINERU["MinerU · Multimodal Parser (PaddleOCR)"] + DOCKER["Docker 隔离沙箱 · Code Interpreter"] + PGW --> SANDBOX --> MINERU --> DOCKER + end +end + +%% ===================== 跨层连线 ===================== +DESKTOP <==>|"4 · PHYSICAL COMMUNICATION
HTTPS / WebSocket / SSE
sundynix.streams.task_id (Token Stream)"| GIN +PARSER ==>|"1 · Publish Task DSL"| QUEUE +STREAM ==>|"4 · Subscribe Stream"| GIN +QUEUE <==>|"4 · Subscribe Stream"| NATSGO +STREAM <==> NATSGO +NATS ==> MCPGO +NATS ==> MCPPY + +%% ===================== 配色 ===================== +classDef client fill:#fdf2e3,stroke:#e0a85a,color:#1c2b3a; +classDef gateway fill:#dcecf7,stroke:#5fa9d4,color:#1c2b3a; +classDef nats fill:#e7f4e1,stroke:#86c06a,color:#1c2b3a; +classDef dispatcher fill:#efe5f8,stroke:#a984cf,color:#1c2b3a; +classDef gotools fill:#d6e7f3,stroke:#5fa9d4,color:#1c2b3a; +classDef pytools fill:#fbf3cf,stroke:#d8bf52,color:#1c2b3a; + +class CLIENT,DESKTOP,UI,SHAD,RF,DSL,WIKI,WAILS,BIND,FS client; +class GATEWAY,GW,GIN,MAINDB,CACHE1,CACHE2,BILL,GUARD,PARSER gateway; +class NATS,QUEUE,STREAM nats; +class DISPATCHER,DISP,LLMPOOL,EINO,NATSGO,EVAL,FUSE dispatcher; +class TOOLS,MCPGO,GGW,SEARCH,MILVUS,BLEVE,NEO1,NEO2,UNI,EXT gotools; +class MCPPY,PGW,SANDBOX,MINERU,DOCKER pytools; +``` + +--- + +## 分层说明 + +| 层 | 组件 | 技术栈 / 职责 | 配色 | +|---|---|---|---| +| **1. Client** | `sundynix-desktop` | React 19 + TS、shadcn/ui + Tailwind、React Flow 画布（Agent 编排 → 导出 JSON DSL）、LLM Wiki 面板、Wails 本地 Go 运行时、TS/Go 强绑定、本地文件 I/O | 🟧 橙 | +| **2. Gateway** | `sundynix-gateway` | Gin 统一接入层；MainDB(PgSQL: Users/Billing/DSL) + CacheDB(Redis: Session/Rate Limit)；商业化计费；输入/输出 Guardrail；**Task DSL 解析与组装** | 🟦 蓝 | +| **3. 消息总线** | `NATS Server (Go)` | 零拷贝骨干网：Queue(`sundynix.tasks.*` 分布式任务队列) + Stream(`sundynix.streams.*` 零拷贝字节管道) | 🟩 绿 | +| **4. Dispatcher** | `sundynix-dispatcher` | Go/Eino 集群：Eino 图编排引擎 + nats.go client，经 EinoCore/回调驱动 LLM Pool(vLLM/Ollama)；LLM 自动化评测、熔断降级中心 | 🟪 紫 | +| **5a. MCP Go** | `sundynix-mcp-go` | Go I/O 型：MCP 协议网关、Wiki 混合检索(Bleve+Milvus+Neo4j)、UniOffice 文档渲染、外部 API | 🟦 蓝 | +| **5b. MCP Py** | `sundynix-mcp-py` | Python 算法型：MCP 协议网关、安全代码沙箱(gVisor/KataVM)、MinerU 多模态解析(PaddleOCR)、Docker 代码解释器 | 🟨 黄 | + +## 核心数据流（图中编号） + +1. **Publish Task DSL** — Gateway 解析 DSL 后发布任务到 NATS Queue (`sundynix.tasks.*`) +2. **订阅与编排** — Dispatcher 经 `nats.go` 订阅任务，Eino 图编排引擎驱动 LLM Pool 推理 +3. **调用 MCP 工具** — Dispatcher / Gateway 经 NATS 调用第 5 层 Go(I/O 型) + Python(算法型) 工具 +4. **Token Stream 回流** — 推理结果以零拷贝流经 `sundynix.streams.task_id` 回 Gateway → 经 SSE/WebSocket 推送给 Client diff --git a/architecture.png b/architecture.png new file mode 100644 index 0000000..26b789d Binary files /dev/null and b/architecture.png differ diff --git a/deploy/nats/nats-server.conf b/deploy/nats/nats-server.conf new file mode 100644 index 0000000..4992b50 --- /dev/null +++ b/deploy/nats/nats-server.conf @@ -0,0 +1,15 @@ +# NATS 零拷贝骨干网 — JetStream 开启 +port: 4222 +http_port: 8222 # 监控端点 + +jetstream { + store_dir: "/data/jetstream" + max_memory_store: 1GB + max_file_store: 10GB +} + +# Subject 命名空间约定（仅文档，实际 stream 由各服务声明式创建） +# sundynix.tasks.* 分布式任务队列 (Queue) +# sundynix.streams. 零拷贝 Token 字节管道 (Stream) +# sundynix.tools.go.* Go MCP 工具调用 +# sundynix.tools.py.* Python MCP 工具调用 diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..ffffa50 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,71 @@ +version: "3.9" + +# 基础设施 — NATS 零拷贝骨干网 + 业务存储 + 向量/图数据库 +services: + nats: + image: nats:2-alpine + command: ["-c", "/etc/nats/nats-server.conf"] + ports: ["4222:4222", "8222:8222"] + volumes: + - ./deploy/nats/nats-server.conf:/etc/nats/nats-server.conf:ro + + postgres: + image: postgres:16-alpine + environment: + POSTGRES_USER: sundynix + POSTGRES_PASSWORD: sundynix + POSTGRES_DB: sundynix + ports: ["5432:5432"] + volumes: ["pg_data:/var/lib/postgresql/data"] + + redis: + image: redis:7-alpine + ports: ["6379:6379"] + + # --- Milvus 向量数据库 (standalone 需 etcd + minio) --- + milvus-etcd: + image: quay.io/coreos/etcd:v3.5.14 + environment: + ETCD_AUTO_COMPACTION_MODE: revision + ETCD_AUTO_COMPACTION_RETENTION: "1000" + ETCD_QUOTA_BACKEND_BYTES: "4294967296" + ETCD_SNAPSHOT_COUNT: "50000" + command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd + volumes: ["milvus_etcd:/etcd"] + + milvus-minio: + image: minio/minio:RELEASE.2023-03-20T20-16-18Z + environment: + MINIO_ACCESS_KEY: minioadmin + MINIO_SECRET_KEY: minioadmin + command: minio server /minio_data + volumes: ["milvus_minio:/minio_data"] + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] + interval: 30s + timeout: 20s + retries: 3 + + milvus: + image: milvusdb/milvus:v2.4.13 + command: ["milvus", "run", "standalone"] + environment: + ETCD_ENDPOINTS: milvus-etcd:2379 + MINIO_ADDRESS: milvus-minio:9000 + ports: ["19530:19530", "9091:9091"] # 19530=gRPC, 9091=metrics/health + volumes: ["milvus_data:/var/lib/milvus"] + depends_on: [milvus-etcd, milvus-minio] + + neo4j: + image: neo4j:5-community + environment: + NEO4J_AUTH: neo4j/sundynix + ports: ["7474:7474", "7687:7687"] + volumes: ["neo4j_data:/data"] + +volumes: + pg_data: + milvus_etcd: + milvus_minio: + milvus_data: + neo4j_data: diff --git a/go.work b/go.work new file mode 100644 index 0000000..d7c43ca --- /dev/null +++ b/go.work @@ -0,0 +1,8 @@ +go 1.23 + +use ( + ./sundynix-shared + ./sundynix-gateway + ./sundynix-dispatcher + ./sundynix-mcp-go +) diff --git a/go.work.sum b/go.work.sum new file mode 100644 index 0000000..0b01978 --- /dev/null +++ b/go.work.sum @@ -0,0 +1,135 @@ +github.com/RoaringBitmap/roaring v1.9.3 h1:t4EbC5qQwnisr5PrP9nt0IRhRTb9gMUgQF4t4S2OByM= +github.com/RoaringBitmap/roaring v1.9.3/go.mod h1:6AXUsoIEzDTFFQCe1RbGA6uFONMhvejWj5rqITANK90= +github.com/bits-and-blooms/bitset v1.12.0 h1:U/q1fAF7xXRhFCrhROzIfffYnu+dlS38vCZtmFVPHmA= +github.com/bits-and-blooms/bitset v1.12.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8= +github.com/blevesearch/bleve/v2 v2.4.2 h1:NooYP1mb3c0StkiY9/xviiq2LGSaE8BQBCc/pirMx0U= +github.com/blevesearch/bleve_index_api v1.1.10 h1:PDLFhVjrjQWr6jCuU7TwlmByQVCSEURADHdCqVS9+g0= +github.com/blevesearch/bleve_index_api v1.1.10/go.mod h1:PbcwjIcRmjhGbkS/lJCpfgVSMROV6TRubGGAODaK1W8= +github.com/blevesearch/geo v0.1.20 h1:paaSpu2Ewh/tn5DKn/FB5SzvH0EWupxHEIwbCk/QPqM= +github.com/blevesearch/geo v0.1.20/go.mod h1:DVG2QjwHNMFmjo+ZgzrIq2sfCh6rIHzy9d9d0B59I6w= +github.com/blevesearch/go-faiss v1.0.20 h1:AIkdTQFWuZ5LQmKQSebgMR4RynGNw8ZseJXaan5kvtI= +github.com/blevesearch/go-faiss v1.0.20/go.mod h1:jrxHrbl42X/RnDPI+wBoZU8joxxuRwedrxqswQ3xfU8= +github.com/blevesearch/go-metrics v0.0.0-20201227073835-cf1acfcdf475 h1:kDy+zgJFJJoJYBvdfBSiZYBbdsUL0XcjHYWezpQBGPA= +github.com/blevesearch/go-metrics v0.0.0-20201227073835-cf1acfcdf475/go.mod h1:9eJDeqxJ3E7WnLebQUlPD7ZjSce7AnDb9vjGmMCbD0A= +github.com/blevesearch/go-porterstemmer v1.0.3 h1:GtmsqID0aZdCSNiY8SkuPJ12pD4jI+DdXTAn4YRcHCo= +github.com/blevesearch/go-porterstemmer v1.0.3/go.mod h1:angGc5Ht+k2xhJdZi511LtmxuEf0OVpvUUNrwmM1P7M= +github.com/blevesearch/goleveldb v1.0.1 h1:iAtV2Cu5s0GD1lwUiekkFHe2gTMCCNVj2foPclDLIFI= +github.com/blevesearch/goleveldb v1.0.1/go.mod h1:WrU8ltZbIp0wAoig/MHbrPCXSOLpe79nz5lv5nqfYrQ= +github.com/blevesearch/gtreap v0.1.1 h1:2JWigFrzDMR+42WGIN/V2p0cUvn4UP3C4Q5nmaZGW8Y= +github.com/blevesearch/gtreap v0.1.1/go.mod h1:QaQyDRAT51sotthUWAH4Sj08awFSSWzgYICSZ3w0tYk= +github.com/blevesearch/mmap-go v1.0.4 h1:OVhDhT5B/M1HNPpYPBKIEJaD0F3Si+CrEKULGCDPWmc= +github.com/blevesearch/mmap-go v1.0.4/go.mod h1:EWmEAOmdAS9z/pi/+Toxu99DnsbhG1TIxUoRmJw/pSs= +github.com/blevesearch/scorch_segment_api/v2 v2.2.15 h1:prV17iU/o+A8FiZi9MXmqbagd8I0bCqM7OKUYPbnb5Y= +github.com/blevesearch/scorch_segment_api/v2 v2.2.15/go.mod h1:db0cmP03bPNadXrCDuVkKLV6ywFSiRgPFT1YVrestBc= +github.com/blevesearch/segment v0.9.1 h1:+dThDy+Lvgj5JMxhmOVlgFfkUtZV2kw49xax4+jTfSU= +github.com/blevesearch/segment v0.9.1/go.mod h1:zN21iLm7+GnBHWTao9I+Au/7MBiL8pPFtJBJTsk6kQw= +github.com/blevesearch/snowball v0.6.1 h1:cDYjn/NCH+wwt2UdehaLpr2e4BwLIjN4V/TdLsL+B5A= +github.com/blevesearch/snowball v0.6.1/go.mod h1:ZF0IBg5vgpeoUhnMza2v0A/z8m1cWPlwhke08LpNusg= +github.com/blevesearch/snowballstem v0.9.0 h1:lMQ189YspGP6sXvZQ4WZ+MLawfV8wOmPoD/iWeNXm8s= +github.com/blevesearch/snowballstem v0.9.0/go.mod h1:PivSj3JMc8WuaFkTSRDW2SlrulNWPl4ABg1tC/hlgLs= +github.com/blevesearch/stempel v0.2.0 h1:CYzVPaScODMvgE9o+kf6D4RJ/VRomyi9uHF+PtB+Afc= +github.com/blevesearch/stempel v0.2.0/go.mod h1:wjeTHqQv+nQdbPuJ/YcvOjTInA2EIc6Ks1FoSUzSLvc= +github.com/blevesearch/upsidedown_store_api v1.0.2 h1:U53Q6YoWEARVLd1OYNc9kvhBMGZzVrdmaozG2MfoB+A= +github.com/blevesearch/upsidedown_store_api v1.0.2/go.mod h1:M01mh3Gpfy56Ps/UXHjEO/knbqyQ1Oamg8If49gRwrQ= +github.com/blevesearch/vellum v1.0.10 h1:HGPJDT2bTva12hrHepVT3rOyIKFFF4t7Gf6yMxyMIPI= +github.com/blevesearch/vellum v1.0.10/go.mod h1:ul1oT0FhSMDIExNjIxHqJoGpVrBpKCdgDQNxfqgJt7k= +github.com/blevesearch/zapx/v11 v11.3.10 h1:hvjgj9tZ9DeIqBCxKhi70TtSZYMdcFn7gDb71Xo/fvk= +github.com/blevesearch/zapx/v11 v11.3.10/go.mod h1:0+gW+FaE48fNxoVtMY5ugtNHHof/PxCqh7CnhYdnMzQ= +github.com/blevesearch/zapx/v12 v12.3.10 h1:yHfj3vXLSYmmsBleJFROXuO08mS3L1qDCdDK81jDl8s= +github.com/blevesearch/zapx/v12 v12.3.10/go.mod h1:0yeZg6JhaGxITlsS5co73aqPtM04+ycnI6D1v0mhbCs= +github.com/blevesearch/zapx/v13 v13.3.10 h1:0KY9tuxg06rXxOZHg3DwPJBjniSlqEgVpxIqMGahDE8= +github.com/blevesearch/zapx/v13 v13.3.10/go.mod h1:w2wjSDQ/WBVeEIvP0fvMJZAzDwqwIEzVPnCPrz93yAk= +github.com/blevesearch/zapx/v14 v14.3.10 h1:SG6xlsL+W6YjhX5N3aEiL/2tcWh3DO75Bnz77pSwwKU= +github.com/blevesearch/zapx/v14 v14.3.10/go.mod h1:qqyuR0u230jN1yMmE4FIAuCxmahRQEOehF78m6oTgns= +github.com/blevesearch/zapx/v15 v15.3.13 h1:6EkfaZiPlAxqXz0neniq35my6S48QI94W/wyhnpDHHQ= +github.com/blevesearch/zapx/v15 v15.3.13/go.mod h1:Turk/TNRKj9es7ZpKK95PS7f6D44Y7fAFy8F4LXQtGg= +github.com/blevesearch/zapx/v16 v16.1.5 h1:b0sMcarqNFxuXvjoXsF8WtwVahnxyhEvBSRJi/AUHjU= +github.com/blevesearch/zapx/v16 v16.1.5/go.mod h1:J4mSF39w1QELc11EWRSBFkPeZuO7r/NPKkHzDCoiaI8= +github.com/bytedance/sonic v1.12.2/go.mod h1:B8Gt/XvtZ3Fqj+iSKMypzymZxw/FVwgIGKzMzT9r/rk= +github.com/bytedance/sonic/loader v0.2.0/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU= +github.com/cockroachdb/errors v1.9.1 h1:yFVvsI0VxmRShfawbt/laCIDy/mtTqqnvoNgiy5bEV8= +github.com/cockroachdb/errors v1.9.1/go.mod h1:2sxOtL2WIc096WSZqZ5h8fa17rdDq9HZOZLBCor4mBk= +github.com/cockroachdb/logtags v0.0.0-20211118104740-dabe8e521a4f h1:6jduT9Hfc0njg5jJ1DdKCFPdMBrp/mdZfCpa5h+WM74= +github.com/cockroachdb/logtags v0.0.0-20211118104740-dabe8e521a4f/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs= +github.com/cockroachdb/redact v1.1.3 h1:AKZds10rFSIj7qADf0g46UixK8NNLwWTNdCIGS5wfSQ= +github.com/cockroachdb/redact v1.1.3/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg= +github.com/couchbase/ghistogram v0.1.0 h1:b95QcQTCzjTUocDXp/uMgSNQi8oj1tGwnJ4bODWZnps= +github.com/couchbase/ghistogram v0.1.0/go.mod h1:s1Jhy76zqfEecpNWJfWUiKZookAFaiGOEoyzgHt9i7k= +github.com/couchbase/moss v0.2.0 h1:VCYrMzFwEryyhRSeI+/b3tRBSeTpi/8gn5Kf6dxqn+o= +github.com/couchbase/moss v0.2.0/go.mod h1:9MaHIaRuy9pvLPUJxB8sh8OrLfyDczECVL37grCIubs= +github.com/creack/pty v1.1.9 h1:uDmaGzcdjhF4i/plgjmEsriH11Y0o7RKapEf/LDaM3w= +github.com/getsentry/sentry-go v0.12.0 h1:era7g0re5iY13bHSdN/xMkyV+5zZppjRVQhZrXCaEIk= +github.com/getsentry/sentry-go v0.12.0/go.mod h1:NSap0JBYWzHND8oMbyi0+XZhUalc1TBdRL1M71JZW2c= +github.com/go-faker/faker/v4 v4.1.0 h1:ffuWmpDrducIUOO0QSKSF5Q2dxAht+dhsT9FvVHhPEI= +github.com/go-faker/faker/v4 v4.1.0/go.mod h1:uuNc0PSRxF8nMgjGrrrU4Nw5cF30Jc6Kd0/FUTTYbhg= +github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= +github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/golang/geo v0.0.0-20210211234256-740aa86cb551 h1:gtexQ/VGyN+VVFRXSFiguSNcXmS6rkKT+X7FdIrTtfo= +github.com/golang/geo v0.0.0-20210211234256-740aa86cb551/go.mod h1:QZ0nwyI2jOfgRAoBvP+ab5aRr7c9x7lhGEJrKvBwjWI= +github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= +github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4= +github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/gofuzz v1.0.0 h1:A8PeW59pxE9IoFRqBp37U+mSNaQoZ46F1f0f863XSXw= +github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 h1:+9834+KizmvFV7pXQGSXQTsaWhq2GjuNUt0aUU0YBYw= +github.com/grpc-ecosystem/go-grpc-middleware v1.3.0/go.mod h1:z0ButlSOZa5vEBq9m2m2hlwIgKw+rp3sdCBRoJY+30Y= +github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= +github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/knz/go-libedit v1.10.1 h1:0pHpWtx9vcvC0xGZqEQlQdfSQs7WRlAjuPvk3fOZDCo= +github.com/kr/pty v1.1.1 h1:VkoXIwSboBpnk99O/KFauAEILuNHv5DVFKZMBN/gUgw= +github.com/milvus-io/milvus-proto/go-api/v2 v2.4.3 h1:KUSaWVePVlHMIluAXf2qmNffI1CMlGFLLiP+4iy9014= +github.com/milvus-io/milvus-proto/go-api/v2 v2.4.3/go.mod h1:1OIl0v5PQeNxIJhCvY+K55CBUOYDZevw9g9380u1Wek= +github.com/milvus-io/milvus-sdk-go/v2 v2.4.1 h1:KhqjmaJE4mSxj1a88XtkGaqgH4duGiHs1sjnvSXkwE0= +github.com/milvus-io/milvus-sdk-go/v2 v2.4.1/go.mod h1:7SJxshlnVhNLksS73tLPtHYY9DiX7lyL43Rv41HCPCw= +github.com/mschoch/smat v0.2.0 h1:8imxQsjDm8yFEAVBe7azKmKSgzSkZXDuKkSq9374khM= +github.com/mschoch/smat v0.2.0/go.mod h1:kc9mz7DoBKqDyiRL7VZN8KvXQMWeTaVnttLRXOlotKw= +github.com/neo4j/neo4j-go-driver/v5 v5.24.0 h1:7MAFoB7L6f9heQUo/tJ5EnrrpVzm9ZBHgH8ew03h6Eo= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/rogpeppe/go-internal v1.8.1 h1:geMPLpDpQOgVyCg5z5GoRwLHepNdb71NXb67XFkP+Eg= +github.com/rogpeppe/go-internal v1.8.1/go.mod h1:JeRgkft04UBgHMgCIwADu4Pn6Mtm5d4nPKWu0nJ5d+o= +github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I= +github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0= +github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= +github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY= +github.com/tidwall/gjson v1.14.4 h1:uo0p8EbA09J7RQaflQ1aBRffTR7xedD2bcIVSYxLnkM= +github.com/tidwall/gjson v1.14.4/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= +github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA= +github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM= +github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs= +github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU= +github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= +github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg= +github.com/yuin/goldmark v1.4.13 h1:fVcFKWvrslecOb/tg+Cc05dkeYx540o0FuFt3nUVDoE= +github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= +go.etcd.io/bbolt v1.3.7 h1:j+zJOnnEjF/kyHlDDgGnVL/AIqIJPq8UoB2GSNfkUfQ= +go.etcd.io/bbolt v1.3.7/go.mod h1:N9Mkw9X8x5fupy0IKsmuqVtoGDyxsaDlbk4Rd05IAQw= +go.uber.org/automaxprocs v1.5.3 h1:kWazyxZUrS3Gs4qUpbwo5kEIMGe/DAvi5Z4tl2NW4j8= +go.uber.org/automaxprocs v1.5.3/go.mod h1:eRbA25aqJrxAbsLO0xy5jVwPt7FQnRgjW+efnwa1WM0= +golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8= +golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA= +golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c= +golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44= +golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ= +golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2 h1:IRJeR9r1pYWsHKTRe/IInb7lYvbBVIqOgsX/u0mbOWY= +golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE= +golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY= +golang.org/x/term v0.24.0 h1:Mh5cbb+Zk2hqqXNO7S1iTjEphVL+jb8ZWaqh/g+JWkM= +golang.org/x/term v0.24.0/go.mod h1:lOBK/LVxemqiMij05LGJ0tzNr8xlmwBRJ81PX6wVLH8= +golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg= +golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk= +google.golang.org/genproto v0.0.0-20220503193339-ba3ae3f07e29 h1:DJUvgAPiJWeMBiT+RzBVcJGQN7bAEWS5UEoMshES9xs= +google.golang.org/genproto v0.0.0-20220503193339-ba3ae3f07e29/go.mod h1:RAyBrSAP7Fh3Nc84ghnVLDPuV51xc9agzmm4Ph6i0Q4= +google.golang.org/grpc v1.48.0 h1:rQOsyJ/8+ufEDJd/Gdsz7HG220Mh9HAhFHRGnIjda0w= +google.golang.org/grpc v1.48.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk= +google.golang.org/grpc/examples v0.0.0-20220617181431-3e7b97febc7f h1:rqzndB2lIQGivcXdTuY3Y9NBvr70X+y77woofSRluec= +google.golang.org/grpc/examples v0.0.0-20220617181431-3e7b97febc7f/go.mod h1:gxndsbNG1n4TZcHGgsYEfVGnTxqfEdfiDv6/DADXX9o= +gopkg.in/errgo.v2 v2.1.0 h1:0vLT13EuvQ0hNvakwLuFZ/jYrLp5F3kcWHXdRggjCE8= +nullprogram.com/x/optparse v1.0.0 h1:xGFgVi5ZaWOnYdac2foDT3vg0ZZC9ErXFV57mr4OHrI= +rsc.io/pdf v0.1.1 h1:k1MczvYDUvJBe93bYd7wrZLLUEcLZAuF824/I4e5Xr4= diff --git a/scripts/demo.sh b/scripts/demo.sh new file mode 100755 index 0000000..210fd50 --- /dev/null +++ b/scripts/demo.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# 无 Docker 的最小任务流演示：devnats(内嵌NATS) + gateway + dispatcher。 +# 提交一个 DSL 任务，验证 Gateway → NATS → Dispatcher 全链路。 +set -euo pipefail +cd "$(dirname "$0")/.." + +mkdir -p .bin +echo "== 编译 ==" +( cd sundynix-shared && go build -o ../.bin/devnats ./cmd/devnats ) +( cd sundynix-gateway && go build -o ../.bin/gateway ./cmd/server ) +( cd sundynix-dispatcher && go build -o ../.bin/dispatcher ./cmd/dispatcher ) + +cleanup() { kill "${GW_PID:-}" "${DISP_PID:-}" "${NATS_PID:-}" 2>/dev/null || true; } +trap cleanup EXIT + +# 若 :4222 已有 NATS（docker compose 的容器），直接复用；否则起内嵌 devnats。 +if nc -z 127.0.0.1 4222 2>/dev/null; then + echo "== 检测到已运行的 NATS(:4222)，复用之 ==" +else + echo "== 启动内嵌 devnats ==" + .bin/devnats > .bin/devnats.log 2>&1 & NATS_PID=$! + for _ in $(seq 1 30); do nc -z 127.0.0.1 4222 2>/dev/null && break || sleep 0.2; done +fi + +echo "== 启动 dispatcher / gateway ==" +.bin/dispatcher > .bin/dispatcher.log 2>&1 & DISP_PID=$! +.bin/gateway > .bin/gateway.log 2>&1 & GW_PID=$! + +for _ in $(seq 1 30); do + curl -s -o /dev/null http://127.0.0.1:8080/api/v1/billing && break || sleep 0.3 +done + +echo "== 提交 DSL 任务 ==" +RESP=$(curl -s -X POST http://127.0.0.1:8080/api/v1/tasks \ + -H 'Content-Type: application/json' \ + -d '{"nodes":[{"id":"n1","type":"agent","data":{"prompt":"hello"}}],"edges":[]}') +echo "$RESP" +TASK_ID=$(echo "$RESP" | sed -n 's/.*"task_id":"\([^"]*\)".*/\1/p') + +echo "== 订阅 SSE Token 流 (Gateway ← NATS ← Dispatcher) ==" +# 客户端在 TTFT(700ms) 内连上即可收全部 token；--max-time 超时(exit 28) 属正常，不让 set -e 中断 +curl -sN --max-time 10 "http://127.0.0.1:8080/api/v1/tasks/$TASK_ID/stream" || true +echo + +echo "== dispatcher 日志 ==" +cat .bin/dispatcher.log diff --git a/sundynix-desktop/app.go b/sundynix-desktop/app.go new file mode 100644 index 0000000..f38a00e --- /dev/null +++ b/sundynix-desktop/app.go @@ -0,0 +1,22 @@ +package main + +import "context" + +// App 通过 Wails 的 TS/Go 强绑定暴露给前端，承载本地文件 I/O 等能力。 +type App struct { + ctx context.Context +} + +func NewApp() *App { return &App{} } + +// SubmitDSL 接收 React Flow 导出的 JSON DSL，转发到 Gateway。 +func (a *App) SubmitDSL(dsl string) (string, error) { + // TODO: HTTP POST 到 sundynix-gateway /api/v1/tasks + return "task_placeholder", nil +} + +// ReadLocalFile 本地文件系统 I/O（Local File System I/O）。 +func (a *App) ReadLocalFile(path string) (string, error) { + // TODO: os.ReadFile，受权限白名单约束 + return "", nil +} diff --git a/sundynix-desktop/frontend/index.html b/sundynix-desktop/frontend/index.html new file mode 100644 index 0000000..4d2e227 --- /dev/null +++ b/sundynix-desktop/frontend/index.html @@ -0,0 +1,12 @@ + + + + + + sundynix-agentix + + +
+ + + diff --git a/sundynix-desktop/frontend/package.json b/sundynix-desktop/frontend/package.json new file mode 100644 index 0000000..edd59aa --- /dev/null +++ b/sundynix-desktop/frontend/package.json @@ -0,0 +1,26 @@ +{ + "name": "sundynix-desktop-frontend", + "private": true, + "version": "0.1.0", + "type": "module", + "scripts": { + "dev": "vite", + "build": "tsc && vite build", + "preview": "vite preview" + }, + "dependencies": { + "react": "^19.0.0", + "react-dom": "^19.0.0", + "@xyflow/react": "^12.3.0" + }, + "devDependencies": { + "@types/react": "^19.0.0", + "@types/react-dom": "^19.0.0", + "@vitejs/plugin-react": "^4.3.0", + "typescript": "^5.6.0", + "vite": "^5.4.0", + "tailwindcss": "^3.4.0", + "autoprefixer": "^10.4.0", + "postcss": "^8.4.0" + } +} diff --git a/sundynix-desktop/frontend/src/App.tsx b/sundynix-desktop/frontend/src/App.tsx new file mode 100644 index 0000000..c1e8f5c --- /dev/null +++ b/sundynix-desktop/frontend/src/App.tsx @@ -0,0 +1,16 @@ +import { AgentCanvas } from "./canvas/AgentCanvas"; +import { WikiPanel } from "./wiki/WikiPanel"; + +// UI Representation Layer —— 顶层布局：左侧编排画布 + 右侧 Wiki 面板。 +export default function App() { + return ( +
+
+ +
+ +
+ ); +} diff --git a/sundynix-desktop/frontend/src/canvas/AgentCanvas.tsx b/sundynix-desktop/frontend/src/canvas/AgentCanvas.tsx new file mode 100644 index 0000000..698f6e7 --- /dev/null +++ b/sundynix-desktop/frontend/src/canvas/AgentCanvas.tsx @@ -0,0 +1,49 @@ +import { useCallback } from "react"; +import { + ReactFlow, + Background, + Controls, + addEdge, + useNodesState, + useEdgesState, + type Connection, +} from "@xyflow/react"; +import "@xyflow/react/dist/style.css"; + +import { exportDsl } from "../lib/dsl"; + +// React Flow Canvas —— Agent 编排，可导出 JSON DSL 提交到 Gateway。 +export function AgentCanvas() { + const [nodes, , onNodesChange] = useNodesState([]); + const [edges, setEdges, onEdgesChange] = useEdgesState([]); + + const onConnect = useCallback( + (c: Connection) => setEdges((eds) => addEdge(c, eds)), + [setEdges], + ); + + const onExport = useCallback(() => { + const dsl = exportDsl(nodes, edges); // → JSON DSL export + // TODO: 经 Wails 强绑定调用 App.SubmitDSL(dsl) + console.log(dsl); + }, [nodes, edges]); + + return ( +
+ + + + + +
+ ); +} diff --git a/sundynix-desktop/frontend/src/components/.gitkeep b/sundynix-desktop/frontend/src/components/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/sundynix-desktop/frontend/src/index.css b/sundynix-desktop/frontend/src/index.css new file mode 100644 index 0000000..ad8a360 --- /dev/null +++ b/sundynix-desktop/frontend/src/index.css @@ -0,0 +1,10 @@ +@tailwind base; +@tailwind components; +@tailwind utilities; + +html, +body, +#root { + height: 100%; + margin: 0; +} diff --git a/sundynix-desktop/frontend/src/lib/dsl.ts b/sundynix-desktop/frontend/src/lib/dsl.ts new file mode 100644 index 0000000..6938dde --- /dev/null +++ b/sundynix-desktop/frontend/src/lib/dsl.ts @@ -0,0 +1,17 @@ +import type { Edge, Node } from "@xyflow/react"; + +// Task DSL —— React Flow 画布的可序列化表示，提交给 Gateway 解析组装。 +export interface TaskDsl { + version: "1"; + nodes: Array<{ id: string; type?: string; data: unknown }>; + edges: Array<{ source: string; target: string }>; +} + +// exportDsl 把画布的节点/连线导出为 JSON DSL。 +export function exportDsl(nodes: Node[], edges: Edge[]): TaskDsl { + return { + version: "1", + nodes: nodes.map((n) => ({ id: n.id, type: n.type, data: n.data })), + edges: edges.map((e) => ({ source: e.source, target: e.target })), + }; +} diff --git a/sundynix-desktop/frontend/src/main.tsx b/sundynix-desktop/frontend/src/main.tsx new file mode 100644 index 0000000..27481e0 --- /dev/null +++ b/sundynix-desktop/frontend/src/main.tsx @@ -0,0 +1,10 @@ +import React from "react"; +import ReactDOM from "react-dom/client"; +import App from "./App"; +import "./index.css"; + +ReactDOM.createRoot(document.getElementById("root")!).render( + + + , +); diff --git a/sundynix-desktop/frontend/src/wiki/WikiPanel.tsx b/sundynix-desktop/frontend/src/wiki/WikiPanel.tsx new file mode 100644 index 0000000..a3817a0 --- /dev/null +++ b/sundynix-desktop/frontend/src/wiki/WikiPanel.tsx @@ -0,0 +1,13 @@ +// LLM Wiki Management Panel —— 管理知识库条目，触发第 5 层混合检索。 +export function WikiPanel() { + return ( +
+

LLM Wiki

+ + {/* TODO: 检索结果列表 / 条目编辑 */} +
+ ); +} diff --git a/sundynix-desktop/frontend/tailwind.config.js b/sundynix-desktop/frontend/tailwind.config.js new file mode 100644 index 0000000..a440ac6 --- /dev/null +++ b/sundynix-desktop/frontend/tailwind.config.js @@ -0,0 +1,6 @@ +/** @type {import('tailwindcss').Config} */ +export default { + content: ["./index.html", "./src/**/*.{ts,tsx}"], + theme: { extend: {} }, + plugins: [], +}; diff --git a/sundynix-desktop/frontend/tsconfig.json b/sundynix-desktop/frontend/tsconfig.json new file mode 100644 index 0000000..b491768 --- /dev/null +++ b/sundynix-desktop/frontend/tsconfig.json @@ -0,0 +1,18 @@ +{ + "compilerOptions": { + "target": "ES2022", + "useDefineForClassFields": true, + "lib": ["ES2022", "DOM", "DOM.Iterable"], + "module": "ESNext", + "skipLibCheck": true, + "moduleResolution": "bundler", + "resolveJsonModule": true, + "isolatedModules": true, + "noEmit": true, + "jsx": "react-jsx", + "strict": true, + "baseUrl": ".", + "paths": { "@/*": ["./src/*"] } + }, + "include": ["src"] +} diff --git a/sundynix-desktop/frontend/vite.config.ts b/sundynix-desktop/frontend/vite.config.ts new file mode 100644 index 0000000..6da5cbf --- /dev/null +++ b/sundynix-desktop/frontend/vite.config.ts @@ -0,0 +1,10 @@ +import { defineConfig } from "vite"; +import react from "@vitejs/plugin-react"; +import path from "node:path"; + +export default defineConfig({ + plugins: [react()], + resolve: { + alias: { "@": path.resolve(__dirname, "./src") }, + }, +}); diff --git a/sundynix-desktop/go.mod b/sundynix-desktop/go.mod new file mode 100644 index 0000000..bd3a957 --- /dev/null +++ b/sundynix-desktop/go.mod @@ -0,0 +1,5 @@ +module github.com/sundynix/sundynix-desktop + +go 1.23 + +require github.com/wailsapp/wails/v2 v2.9.2 diff --git a/sundynix-desktop/go.sum b/sundynix-desktop/go.sum new file mode 100644 index 0000000..d87ea6f --- /dev/null +++ b/sundynix-desktop/go.sum @@ -0,0 +1 @@ +github.com/wailsapp/wails/v2 v2.9.2/go.mod h1:uehvlCwJSFcBq7rMCGfk4rxca67QQGsbg5Nm4m9UnBs= diff --git a/sundynix-desktop/main.go b/sundynix-desktop/main.go new file mode 100644 index 0000000..c4edaf2 --- /dev/null +++ b/sundynix-desktop/main.go @@ -0,0 +1,22 @@ +// Command sundynix-desktop —— 第 1 层客户端，Wails 本地 Go 运行时入口。 +package main + +import ( + "embed" + + "github.com/wailsapp/wails/v2/pkg/options" +) + +//go:embed all:frontend/dist +var assets embed.FS + +func main() { + app := NewApp() + _ = wails.Run(&options.App{ + Title: "sundynix-agentix", + Width: 1280, + Height: 800, + // Bind: TS/Go 强绑定 —— 把 App 的方法暴露给前端 + Bind: []any{app}, + }) +} diff --git a/sundynix-desktop/wails.json b/sundynix-desktop/wails.json new file mode 100644 index 0000000..815426f --- /dev/null +++ b/sundynix-desktop/wails.json @@ -0,0 +1,12 @@ +{ + "$schema": "https://wails.io/schemas/config.v2.json", + "name": "sundynix_desktop", + "outputfilename": "sundynix_desktop", + "frontend:install": "npm install", + "frontend:build": "npm run build", + "frontend:dev:watcher": "npm run dev", + "frontend:dev:serverUrl": "auto", + "author": { + "name": "sundynix" + } +} diff --git a/sundynix-dispatcher/cmd/dispatcher/main.go b/sundynix-dispatcher/cmd/dispatcher/main.go new file mode 100644 index 0000000..2a2184a --- /dev/null +++ b/sundynix-dispatcher/cmd/dispatcher/main.go @@ -0,0 +1,44 @@ +// Command dispatcher 启动 sundynix-dispatcher —— 第 4 层 AI Agent 调度集群。 +package main + +import ( + "context" + "log" + "os" + "os/signal" + "syscall" + + "github.com/sundynix/sundynix-dispatcher/internal/eino" + "github.com/sundynix/sundynix-dispatcher/internal/harness" + "github.com/sundynix/sundynix-dispatcher/internal/llm" + dnats "github.com/sundynix/sundynix-dispatcher/internal/nats" +) + +func main() { + natsURL := envOr("NATS_URL", "nats://localhost:4222") + + pool := llm.NewPool() // LLM Pool: vLLM / Ollama 集群 + breaker := harness.NewCircuitBreaker() // Harness: 熔断降级中心 + + sub := dnats.MustConnect(natsURL) + defer sub.Close() + + // sub 同时作为 Token 回流出口（TokenSink）。 + orch := eino.NewOrchestrator(pool, breaker, sub) + + // 监听退出信号，优雅停止消费。 + ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) + defer stop() + + log.Println("[dispatcher] consuming sundynix.tasks.* (Ctrl-C to quit)") + if err := sub.ConsumeTasks(ctx, orch.Handle); err != nil && err != context.Canceled { + log.Fatalf("[dispatcher] exit: %v", err) + } +} + +func envOr(key, def string) string { + if v := os.Getenv(key); v != "" { + return v + } + return def +} diff --git a/sundynix-dispatcher/config/config.yaml b/sundynix-dispatcher/config/config.yaml new file mode 100644 index 0000000..3710b86 --- /dev/null +++ b/sundynix-dispatcher/config/config.yaml @@ -0,0 +1,18 @@ +nats: + url: "nats://localhost:4222" + task_subject: "sundynix.tasks.*" + queue_group: "dispatchers" + stream_prefix: "sundynix.streams" + +llm_pool: + backends: + - name: "vllm-0" + base_url: "http://localhost:8000/v1" + type: "vllm" + - name: "ollama-0" + base_url: "http://localhost:11434" + type: "ollama" + +circuit_breaker: + error_threshold: 0.5 + open_timeout: "30s" diff --git a/sundynix-dispatcher/go.mod b/sundynix-dispatcher/go.mod new file mode 100644 index 0000000..7499977 --- /dev/null +++ b/sundynix-dispatcher/go.mod @@ -0,0 +1,17 @@ +module github.com/sundynix/sundynix-dispatcher + +go 1.23 + +require github.com/sundynix/sundynix-shared v0.0.0 + +require ( + github.com/klauspost/compress v1.17.9 // indirect + github.com/nats-io/nats.go v1.37.0 // indirect + github.com/nats-io/nkeys v0.4.7 // indirect + github.com/nats-io/nuid v1.0.1 // indirect + golang.org/x/crypto v0.26.0 // indirect + golang.org/x/sys v0.24.0 // indirect + golang.org/x/text v0.17.0 // indirect +) + +replace github.com/sundynix/sundynix-shared => ../sundynix-shared diff --git a/sundynix-dispatcher/go.sum b/sundynix-dispatcher/go.sum new file mode 100644 index 0000000..7483077 --- /dev/null +++ b/sundynix-dispatcher/go.sum @@ -0,0 +1,22 @@ +github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA= +github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw= +github.com/minio/highwayhash v1.0.3 h1:kbnuUMoHYyVl7szWjSxJnxw11k2U709jqFPPmIUyD6Q= +github.com/minio/highwayhash v1.0.3/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ= +github.com/nats-io/jwt/v2 v2.5.8 h1:uvdSzwWiEGWGXf+0Q+70qv6AQdvcvxrv9hPM0RiPamE= +github.com/nats-io/jwt/v2 v2.5.8/go.mod h1:ZdWS1nZa6WMZfFwwgpEaqBV8EPGVgOTDHN/wTbz0Y5A= +github.com/nats-io/nats-server/v2 v2.10.20 h1:CXDTYNHeBiAKBTAIP2gjpgbWap2GhATnTLgP8etyvEI= +github.com/nats-io/nats-server/v2 v2.10.20/go.mod h1:hgcPnoUtMfxz1qVOvLZGurVypQ+Cg6GXVXjG53iHk+M= +github.com/nats-io/nats.go v1.37.0 h1:07rauXbVnnJvv1gfIyghFEo6lUcYRY0WXc3x7x0vUxE= +github.com/nats-io/nats.go v1.37.0/go.mod h1:Ubdu4Nh9exXdSz0RVWRFBbRfrbSxOYd26oF0wkWclB8= +github.com/nats-io/nkeys v0.4.7 h1:RwNJbbIdYCoClSDNY7QVKZlyb/wfT6ugvFCiKy6vDvI= +github.com/nats-io/nkeys v0.4.7/go.mod h1:kqXRgRDPlGy7nGaEDMuYzmiJCIAAWDK0IMBtDmGD0nc= +github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw= +github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c= +golang.org/x/crypto v0.26.0 h1:RrRspgV4mU+YwB4FYnuBoKsUapNIL5cohGAmSH3azsw= +golang.org/x/crypto v0.26.0/go.mod h1:GY7jblb9wI+FOo5y8/S2oY4zWP07AkOJ4+jxCqdqn54= +golang.org/x/sys v0.24.0 h1:Twjiwq9dn6R1fQcyiK+wQyHWfaz/BJB+YIpzU/Cv3Xg= +golang.org/x/sys v0.24.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/text v0.17.0 h1:XtiM5bkSOt+ewxlOE/aE/AKEHibwj/6gvWMl9Rsh0Qc= +golang.org/x/text v0.17.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY= +golang.org/x/time v0.6.0 h1:eTDhh4ZXt5Qf0augr54TN6suAUudPcawVZeIAPU7D4U= +golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= diff --git a/sundynix-dispatcher/internal/eino/orchestrator.go b/sundynix-dispatcher/internal/eino/orchestrator.go new file mode 100644 index 0000000..9b49f29 --- /dev/null +++ b/sundynix-dispatcher/internal/eino/orchestrator.go @@ -0,0 +1,60 @@ +// Package eino 封装基于 CloudWeGo Eino 的 Agent 图编排引擎。 +package eino + +import ( + "context" + "log" + + "github.com/sundynix/sundynix-dispatcher/internal/harness" + "github.com/sundynix/sundynix-dispatcher/internal/llm" + "github.com/sundynix/sundynix-shared/contract" +) + +// TokenSink 是 Token 流回流出口（由 NATS bus 实现）。 +type TokenSink interface { + PublishToken(taskID string, token []byte) error + CompleteStream(taskID string) error +} + +// Orchestrator 将 DSL 图编译为 Eino Graph 并驱动执行。 +type Orchestrator struct { + pool *llm.Pool + breaker *harness.CircuitBreaker + sink TokenSink +} + +func NewOrchestrator(pool *llm.Pool, breaker *harness.CircuitBreaker, sink TokenSink) *Orchestrator { + return &Orchestrator{pool: pool, breaker: breaker, sink: sink} +} + +// Handle 消费一个任务：编译图 → 流式推理 → 经 sink 把 Token 回流到 sundynix.streams.。 +func (o *Orchestrator) Handle(ctx context.Context, t *contract.Task) error { + if !o.breaker.Allow() { + log.Printf("[eino] circuit open, drop task %s", t.ID) + return nil + } + log.Printf("[eino] task %s received (graph=%d bytes), streaming tokens...", t.ID, len(t.Graph)) + + // TODO: compose.NewGraph(...) 编译 DSL；此处 prompt 占位为图原文。 + // 工具节点经 NATS 调用第 5 层 MCP（sundynix.tools.go.* / sundynix.tools.py.*）。 + prompt := string(t.Graph) + + n := 0 + err := o.pool.Stream(ctx, prompt, func(tok []byte) { + if perr := o.sink.PublishToken(t.ID, tok); perr != nil { + log.Printf("[eino] publish token failed: %v", perr) + return + } + n++ + }) + if err != nil { + log.Printf("[eino] task %s stream error: %v", t.ID, err) + } + + if cerr := o.sink.CompleteStream(t.ID); cerr != nil { + log.Printf("[eino] complete stream failed: %v", cerr) + } + log.Printf("[eino] task %s done, %d tokens streamed", t.ID, n) + o.breaker.Report(err == nil) + return err +} diff --git a/sundynix-dispatcher/internal/harness/circuitbreaker.go b/sundynix-dispatcher/internal/harness/circuitbreaker.go new file mode 100644 index 0000000..0710c7a --- /dev/null +++ b/sundynix-dispatcher/internal/harness/circuitbreaker.go @@ -0,0 +1,12 @@ +package harness + +// CircuitBreaker 实现熔断降级中心：后端异常时熔断并切换降级策略。 +type CircuitBreaker struct{ /* state, counters */ } + +func NewCircuitBreaker() *CircuitBreaker { return &CircuitBreaker{} } + +// Allow 判定当前是否放行请求。 +func (c *CircuitBreaker) Allow() bool { return true } // TODO: half-open / open 状态机 + +// Report 上报一次调用结果以驱动状态机。 +func (c *CircuitBreaker) Report(success bool) {} // TODO diff --git a/sundynix-dispatcher/internal/harness/eval.go b/sundynix-dispatcher/internal/harness/eval.go new file mode 100644 index 0000000..b4b6d50 --- /dev/null +++ b/sundynix-dispatcher/internal/harness/eval.go @@ -0,0 +1,15 @@ +// Package harness 提供 LLM 自动化评测与熔断降级能力。 +package harness + +import "context" + +// Evaluator 实现 LLM 自动化评测（质量打分 / 回归对比）。 +type Evaluator struct{} + +func NewEvaluator() *Evaluator { return &Evaluator{} } + +// Score 对一次推理输出打分。 +func (e *Evaluator) Score(ctx context.Context, input, output string) (float64, error) { + // TODO: LLM-as-judge / 规则评测 + return 0, nil +} diff --git a/sundynix-dispatcher/internal/llm/pool.go b/sundynix-dispatcher/internal/llm/pool.go new file mode 100644 index 0000000..b8d6c30 --- /dev/null +++ b/sundynix-dispatcher/internal/llm/pool.go @@ -0,0 +1,62 @@ +// Package llm 抽象 LLM Pool（vLLM / Ollama 集群）的负载均衡与流式推理。 +package llm + +import ( + "context" + "strings" + "time" +) + +// Pool 维护后端 LLM 实例列表与路由策略。 +type Pool struct{ /* backends []Backend */ } + +func NewPool() *Pool { return &Pool{} } + +// 占位参数：模拟真实后端的 TTFT(首 token 延迟) 与逐 token 间隔。 +const ( + timeToFirstToken = 700 * time.Millisecond + interTokenDelay = 60 * time.Millisecond +) + +// Stream 选择一个后端进行流式推理，逐 Token 回调 onToken。 +// 当前为占位实现：把对 prompt 的确定性回复按 token 流式返回， +// 真实接入 vLLM/Ollama 时替换为后端 streaming API 即可（回调签名不变）。 +func (p *Pool) Stream(ctx context.Context, prompt string, onToken func([]byte)) error { + // TODO: 选路 (least-load / 模型亲和) → 调 vLLM/Ollama streaming API + reply := buildReply(prompt) + + select { + case <-ctx.Done(): + return ctx.Err() + case <-time.After(timeToFirstToken): // 模拟 TTFT + } + + for _, tok := range tokenize(reply) { + select { + case <-ctx.Done(): + return ctx.Err() + default: + } + onToken([]byte(tok)) + time.Sleep(interTokenDelay) + } + return nil +} + +// buildReply 占位：真实实现应由 DSL 编排出的对话上下文驱动后端生成。 +func buildReply(prompt string) string { + p := strings.TrimSpace(prompt) + if len(p) > 40 { + p = p[:40] + "…" + } + return "已编排执行该 Agent 图，输入摘要: " + p +} + +// tokenize 占位分词：按 rune 切，保证多字节中文也能逐字流式。 +func tokenize(s string) []string { + out := make([]string, 0, len(s)) + for _, r := range s { + out = append(out, string(r)) + } + return out +} diff --git a/sundynix-dispatcher/internal/nats/subscriber.go b/sundynix-dispatcher/internal/nats/subscriber.go new file mode 100644 index 0000000..ddad315 --- /dev/null +++ b/sundynix-dispatcher/internal/nats/subscriber.go @@ -0,0 +1,56 @@ +// Package nats 是调度器对共享 bus 的薄封装（消费任务 / 回写 Token）。 +package nats + +import ( + "context" + "log" + + sharedbus "github.com/sundynix/sundynix-shared/bus" + "github.com/sundynix/sundynix-shared/contract" +) + +// TaskHandler 处理单个任务。 +type TaskHandler func(ctx context.Context, t *contract.Task) error + +// Subscriber 包装共享 bus，向调度器暴露消费能力。 +type Subscriber struct { + inner *sharedbus.Bus +} + +// MustConnect 接入 NATS 并确保任务流存在（消费者声明在 Consume 时完成）。 +func MustConnect(url string) *Subscriber { + inner, err := sharedbus.Connect(url) + if err != nil { + log.Fatalf("[dispatcher/nats] connect: %v", err) + } + if err := inner.EnsureTaskStream(context.Background()); err != nil { + log.Fatalf("[dispatcher/nats] ensure stream: %v", err) + } + log.Printf("[dispatcher/nats] connected %s", url) + return &Subscriber{inner: inner} +} + +// ConsumeTasks 从 sundynix.tasks.* 持续消费任务（队列组负载均衡），阻塞至 ctx 取消。 +func (s *Subscriber) ConsumeTasks(ctx context.Context, h TaskHandler) error { + stop, err := s.inner.ConsumeTasks(ctx, func(c context.Context, t *contract.Task) error { + return h(c, t) + }) + if err != nil { + return err + } + defer stop() + <-ctx.Done() + return ctx.Err() +} + +// PublishToken / CompleteStream 让 Subscriber 满足 eino.TokenSink， +// 把推理 Token 回流到 sundynix.streams.。 +func (s *Subscriber) PublishToken(taskID string, token []byte) error { + return s.inner.PublishToken(taskID, token) +} + +func (s *Subscriber) CompleteStream(taskID string) error { + return s.inner.CompleteStream(taskID) +} + +func (s *Subscriber) Close() { s.inner.Close() } diff --git a/sundynix-gateway/cmd/server/main.go b/sundynix-gateway/cmd/server/main.go new file mode 100644 index 0000000..fb2583e --- /dev/null +++ b/sundynix-gateway/cmd/server/main.go @@ -0,0 +1,34 @@ +// Command server 启动 sundynix-gateway —— 第 2 层业务网关 / 统一接入层。 +package main + +import ( + "log" + "os" + + "github.com/sundynix/sundynix-gateway/internal/nats" + "github.com/sundynix/sundynix-gateway/internal/router" + "github.com/sundynix/sundynix-gateway/internal/store" +) + +func main() { + natsURL := envOr("NATS_URL", "nats://localhost:4222") + + db := store.MustOpenPostgres() // MainDB: Users / Billing / DSL + cache := store.MustOpenRedis() // CacheDB: Session / Rate Limit + bus := nats.MustConnect(natsURL) // 接入 NATS 零拷贝骨干网 + 声明任务流 + defer bus.Close() + + r := router.New(db, cache, bus) + addr := envOr("GATEWAY_ADDR", ":8080") + log.Printf("[gateway] listening on %s", addr) + if err := r.Run(addr); err != nil { + log.Fatalf("[gateway] exit: %v", err) + } +} + +func envOr(key, def string) string { + if v := os.Getenv(key); v != "" { + return v + } + return def +} diff --git a/sundynix-gateway/config/config.yaml b/sundynix-gateway/config/config.yaml new file mode 100644 index 0000000..7bfd832 --- /dev/null +++ b/sundynix-gateway/config/config.yaml @@ -0,0 +1,13 @@ +server: + addr: ":8080" + +postgres: # MainDB + dsn: "postgres://sundynix:sundynix@localhost:5432/sundynix?sslmode=disable" + +redis: # CacheDB + addr: "localhost:6379" + +nats: + url: "nats://localhost:4222" + task_subject: "sundynix.tasks" + stream_prefix: "sundynix.streams" diff --git a/sundynix-gateway/go.mod b/sundynix-gateway/go.mod new file mode 100644 index 0000000..7087b97 --- /dev/null +++ b/sundynix-gateway/go.mod @@ -0,0 +1,45 @@ +module github.com/sundynix/sundynix-gateway + +go 1.23 + +require ( + github.com/gin-gonic/gin v1.10.0 + github.com/sundynix/sundynix-shared v0.0.0 +) + +replace github.com/sundynix/sundynix-shared => ../sundynix-shared + +require ( + github.com/bytedance/sonic v1.11.6 // indirect + github.com/bytedance/sonic/loader v0.1.1 // indirect + github.com/cloudwego/base64x v0.1.4 // indirect + github.com/cloudwego/iasm v0.2.0 // indirect + github.com/gabriel-vasile/mimetype v1.4.3 // indirect + github.com/gin-contrib/sse v0.1.0 // indirect + github.com/go-playground/locales v0.14.1 // indirect + github.com/go-playground/universal-translator v0.18.1 // indirect + github.com/go-playground/validator/v10 v10.20.0 // indirect + github.com/goccy/go-json v0.10.2 // indirect + github.com/json-iterator/go v1.1.12 // indirect + github.com/klauspost/compress v1.17.9 // indirect + github.com/klauspost/cpuid/v2 v2.2.7 // indirect + github.com/kr/pretty v0.3.0 // indirect + github.com/leodido/go-urn v1.4.0 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/nats-io/nats.go v1.37.0 // indirect + github.com/nats-io/nkeys v0.4.7 // indirect + github.com/nats-io/nuid v1.0.1 // indirect + github.com/pelletier/go-toml/v2 v2.2.2 // indirect + github.com/twitchyliquid64/golang-asm v0.15.1 // indirect + github.com/ugorji/go/codec v1.2.12 // indirect + golang.org/x/arch v0.8.0 // indirect + golang.org/x/crypto v0.27.0 // indirect + golang.org/x/net v0.25.0 // indirect + golang.org/x/sys v0.25.0 // indirect + golang.org/x/text v0.18.0 // indirect + google.golang.org/protobuf v1.34.1 // indirect + gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect +) diff --git a/sundynix-gateway/go.sum b/sundynix-gateway/go.sum new file mode 100644 index 0000000..ae0b10e --- /dev/null +++ b/sundynix-gateway/go.sum @@ -0,0 +1,119 @@ +github.com/bytedance/sonic v1.11.6 h1:oUp34TzMlL+OY1OUWxHqsdkgC/Zfc85zGqw9siXjrc0= +github.com/bytedance/sonic v1.11.6/go.mod h1:LysEHSvpvDySVdC2f87zGWf6CIKJcAvqab1ZaiQtds4= +github.com/bytedance/sonic/loader v0.1.1 h1:c+e5Pt1k/cy5wMveRDyk2X4B9hF4g7an8N3zCYjJFNM= +github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU= +github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y= +github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w= +github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg= +github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0= +github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk= +github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE= +github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI= +github.com/gin-gonic/gin v1.10.0 h1:nTuyha1TYqgedzytsKYqna+DfLos46nTv2ygFy86HFU= +github.com/gin-gonic/gin v1.10.0/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y= +github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s= +github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4= +github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA= +github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY= +github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY= +github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY= +github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8= +github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM= +github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU= +github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I= +github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU= +github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA= +github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw= +github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= +github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM= +github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws= +github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M= +github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= +github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0= +github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ= +github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/minio/highwayhash v1.0.3 h1:kbnuUMoHYyVl7szWjSxJnxw11k2U709jqFPPmIUyD6Q= +github.com/minio/highwayhash v1.0.3/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/nats-io/jwt/v2 v2.5.8 h1:uvdSzwWiEGWGXf+0Q+70qv6AQdvcvxrv9hPM0RiPamE= +github.com/nats-io/jwt/v2 v2.5.8/go.mod h1:ZdWS1nZa6WMZfFwwgpEaqBV8EPGVgOTDHN/wTbz0Y5A= +github.com/nats-io/nats-server/v2 v2.10.20 h1:CXDTYNHeBiAKBTAIP2gjpgbWap2GhATnTLgP8etyvEI= +github.com/nats-io/nats-server/v2 v2.10.20/go.mod h1:hgcPnoUtMfxz1qVOvLZGurVypQ+Cg6GXVXjG53iHk+M= +github.com/nats-io/nats.go v1.37.0 h1:07rauXbVnnJvv1gfIyghFEo6lUcYRY0WXc3x7x0vUxE= +github.com/nats-io/nats.go v1.37.0/go.mod h1:Ubdu4Nh9exXdSz0RVWRFBbRfrbSxOYd26oF0wkWclB8= +github.com/nats-io/nkeys v0.4.7 h1:RwNJbbIdYCoClSDNY7QVKZlyb/wfT6ugvFCiKy6vDvI= +github.com/nats-io/nkeys v0.4.7/go.mod h1:kqXRgRDPlGy7nGaEDMuYzmiJCIAAWDK0IMBtDmGD0nc= +github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw= +github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c= +github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM= +github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/rogpeppe/go-internal v1.6.1 h1:/FiVV8dS/e+YqF2JvO3yXRFbBLTIuSDkuC7aBOAvL+k= +github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= +github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI= +github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08= +github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE= +github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg= +golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8= +golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc= +golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys= +golang.org/x/crypto v0.27.0 h1:GXm2NjJrPaiv/h1tb2UH8QfgC/hOf/+z0p6PT8o1w7A= +golang.org/x/crypto v0.27.0/go.mod h1:1Xngt8kV6Dvbssa53Ziq6Eqn0HqbZi5Z6R0ZpwQzt70= +golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac= +golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM= +golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.25.0 h1:r+8e+loiHxRqhXVl6ML1nO3l1+oFoWbnlu2Ehimmi34= +golang.org/x/sys v0.25.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/text v0.18.0 h1:XvMDiNzPAl0jr17s6W9lcaIhGUfUORdGCNsuLmPG224= +golang.org/x/text v0.18.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY= +golang.org/x/time v0.6.0 h1:eTDhh4ZXt5Qf0augr54TN6suAUudPcawVZeIAPU7D4U= +golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +google.golang.org/protobuf v1.34.1 h1:9ddQBjfCyZPOHPUiPxpYESBLc+T8P3E+Vo4IbKZgFWg= +google.golang.org/protobuf v1.34.1/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +nullprogram.com/x/optparse v1.0.0/go.mod h1:KdyPE+Igbe0jQUrVfMqDMeJQIJZEuyV7pjYmp6pbG50= +rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4= diff --git a/sundynix-gateway/internal/dsl/parser.go b/sundynix-gateway/internal/dsl/parser.go new file mode 100644 index 0000000..fff3606 --- /dev/null +++ b/sundynix-gateway/internal/dsl/parser.go @@ -0,0 +1,35 @@ +// Package dsl 负责把客户端导出的 JSON DSL 解析并组装为可调度的 Task。 +package dsl + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "errors" + + "github.com/sundynix/sundynix-shared/contract" +) + +// ParseAndAssemble 校验 DSL 结构并生成共享契约中的 Task。 +func ParseAndAssemble(raw json.RawMessage) (*contract.Task, error) { + if len(raw) == 0 { + return nil, errors.New("empty dsl") + } + // 轻量结构校验：至少要能解析为对象。 + var probe map[string]json.RawMessage + if err := json.Unmarshal(raw, &probe); err != nil { + return nil, errors.New("invalid dsl json: " + err.Error()) + } + // TODO: 节点拓扑校验 / 节点-工具映射 + return &contract.Task{ + ID: newID(), + Graph: raw, + Meta: map[string]any{}, + }, nil +} + +func newID() string { + var b [8]byte + _, _ = rand.Read(b[:]) + return "task_" + hex.EncodeToString(b[:]) +} diff --git a/sundynix-gateway/internal/handler/task_handler.go b/sundynix-gateway/internal/handler/task_handler.go new file mode 100644 index 0000000..a8edbdc --- /dev/null +++ b/sundynix-gateway/internal/handler/task_handler.go @@ -0,0 +1,86 @@ +// Package handler 实现网关的 HTTP 处理器。 +package handler + +import ( + "encoding/json" + "io" + "net/http" + + "github.com/gin-gonic/gin" + + "github.com/sundynix/sundynix-gateway/internal/dsl" + "github.com/sundynix/sundynix-gateway/internal/nats" + "github.com/sundynix/sundynix-gateway/internal/store" +) + +type Handler struct { + db *store.Postgres + cache *store.Redis + bus *nats.Bus +} + +func New(db *store.Postgres, cache *store.Redis, bus *nats.Bus) *Handler { + return &Handler{db: db, cache: cache, bus: bus} +} + +// SubmitTask: 解析客户端导出的 JSON DSL，组装为 Task，Publish 到 sundynix.tasks.*。 +func (h *Handler) SubmitTask(c *gin.Context) { + var raw json.RawMessage + if err := c.ShouldBindJSON(&raw); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) + return + } + task, err := dsl.ParseAndAssemble(raw) // Task DSL Parser & Assembly + if err != nil { + c.JSON(http.StatusUnprocessableEntity, gin.H{"error": err.Error()}) + return + } + if err := h.bus.PublishTask(c.Request.Context(), task); err != nil { + c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()}) + return + } + c.JSON(http.StatusAccepted, gin.H{"task_id": task.ID}) +} + +// StreamTask: 订阅 sundynix.streams.，以 SSE 把零拷贝 Token Stream 推给客户端。 +func (h *Handler) StreamTask(c *gin.Context) { + taskID := c.Param("id") + c.Writer.Header().Set("Content-Type", "text/event-stream") + c.Writer.Header().Set("Cache-Control", "no-cache") + c.Writer.Header().Set("Connection", "keep-alive") + + tokens := make(chan []byte, 256) + done := make(chan struct{}) + unsub, err := h.bus.SubscribeTokens(taskID, + func(tok []byte) { + select { + case tokens <- tok: + default: // 背压保护：客户端过慢则丢弃，避免阻塞 NATS 回调 + } + }, + func() { close(done) }, + ) + if err != nil { + c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()}) + return + } + defer func() { _ = unsub() }() + + // gin 的流式写：返回 false 即结束响应。 + c.Stream(func(w io.Writer) bool { + select { + case tok := <-tokens: + c.SSEvent("token", string(tok)) + return true + case <-done: + c.SSEvent("done", taskID) + return false + case <-c.Request.Context().Done(): + return false + } + }) +} + +func (h *Handler) Billing(c *gin.Context) { + c.JSON(http.StatusOK, gin.H{"status": "ok"}) // TODO: 商业化与计费模块 +} diff --git a/sundynix-gateway/internal/middleware/guardrail.go b/sundynix-gateway/internal/middleware/guardrail.go new file mode 100644 index 0000000..1615a83 --- /dev/null +++ b/sundynix-gateway/internal/middleware/guardrail.go @@ -0,0 +1,25 @@ +// Package middleware 提供 Guardrail 与限流等接入层中间件。 +package middleware + +import ( + "github.com/gin-gonic/gin" + + "github.com/sundynix/sundynix-gateway/internal/store" +) + +// Guardrail 实现 Harness 的输入/输出护栏（敏感词、注入、配额校验等）。 +func Guardrail() gin.HandlerFunc { + return func(c *gin.Context) { + // TODO: 输入护栏校验 + c.Next() + // TODO: 输出护栏校验 + } +} + +// RateLimit 基于 Redis 的会话级限流。 +func RateLimit(cache *store.Redis) gin.HandlerFunc { + return func(c *gin.Context) { + // TODO: 令牌桶 / 滑动窗口 + c.Next() + } +} diff --git a/sundynix-gateway/internal/nats/publisher.go b/sundynix-gateway/internal/nats/publisher.go new file mode 100644 index 0000000..d1ca662 --- /dev/null +++ b/sundynix-gateway/internal/nats/publisher.go @@ -0,0 +1,46 @@ +// Package nats 是网关对共享 bus 的薄封装（发布任务 / 订阅 Token 回流）。 +package nats + +import ( + "context" + "log" + + sharedbus "github.com/sundynix/sundynix-shared/bus" + "github.com/sundynix/sundynix-shared/contract" +) + +// Bus 包装共享 bus，向网关其余代码暴露发布能力。 +type Bus struct { + inner *sharedbus.Bus +} + +// MustConnect 接入 NATS 并确保任务流存在。 +func MustConnect(url string) *Bus { + inner, err := sharedbus.Connect(url) + if err != nil { + log.Fatalf("[nats] connect: %v", err) + } + if err := inner.EnsureTaskStream(context.Background()); err != nil { + log.Fatalf("[nats] ensure stream: %v", err) + } + log.Printf("[nats] connected %s, task stream ready", url) + return &Bus{inner: inner} +} + +// PublishTask 把组装后的 Task 发布到 sundynix.tasks.。 +func (b *Bus) PublishTask(ctx context.Context, t *contract.Task) error { + seq, err := b.inner.PublishTask(ctx, t) + if err != nil { + return err + } + log.Printf("[nats] published task %s (seq=%d)", t.ID, seq) + return nil +} + +// SubscribeTokens 订阅 sundynix.streams. 的 Token 回流， +// 每个 Token 触发 onToken，流结束触发 onDone，返回 unsub。 +func (b *Bus) SubscribeTokens(taskID string, onToken func([]byte), onDone func()) (func() error, error) { + return b.inner.SubscribeTokens(taskID, onToken, onDone) +} + +func (b *Bus) Close() { b.inner.Close() } diff --git a/sundynix-gateway/internal/router/router.go b/sundynix-gateway/internal/router/router.go new file mode 100644 index 0000000..4242f76 --- /dev/null +++ b/sundynix-gateway/internal/router/router.go @@ -0,0 +1,27 @@ +// Package router 装配 Gin 统一接入层的路由与中间件。 +package router + +import ( + "github.com/gin-gonic/gin" + + "github.com/sundynix/sundynix-gateway/internal/handler" + "github.com/sundynix/sundynix-gateway/internal/middleware" + "github.com/sundynix/sundynix-gateway/internal/nats" + "github.com/sundynix/sundynix-gateway/internal/store" +) + +// New 构建带有 Guardrail / 限流中间件的 Gin 引擎。 +func New(db *store.Postgres, cache *store.Redis, bus *nats.Bus) *gin.Engine { + r := gin.Default() + r.Use(middleware.RateLimit(cache)) + r.Use(middleware.Guardrail()) // Harness: Input/Output Guardrail + + h := handler.New(db, cache, bus) + api := r.Group("/api/v1") + { + api.POST("/tasks", h.SubmitTask) // 1. 解析 DSL 并 Publish 到 NATS + api.GET("/tasks/:id/stream", h.StreamTask) // 4. SSE/WS 回流 Token Stream + api.GET("/billing", h.Billing) + } + return r +} diff --git a/sundynix-gateway/internal/store/pgsql.go b/sundynix-gateway/internal/store/pgsql.go new file mode 100644 index 0000000..ca92fb2 --- /dev/null +++ b/sundynix-gateway/internal/store/pgsql.go @@ -0,0 +1,14 @@ +// Package store 封装 MainDB(PgSQL) 与 CacheDB(Redis) 的访问。 +package store + +import "log" + +// Postgres 持有 MainDB 连接池（Users / Billing / DSL）。 +type Postgres struct{ /* *pgxpool.Pool */ } + +// MustOpenPostgres 建立 Postgres 连接，失败即退出。 +func MustOpenPostgres() *Postgres { + // TODO: pgxpool.New(ctx, dsn) + log.Println("[store] postgres connected (stub)") + return &Postgres{} +} diff --git a/sundynix-gateway/internal/store/redis.go b/sundynix-gateway/internal/store/redis.go new file mode 100644 index 0000000..572b78e --- /dev/null +++ b/sundynix-gateway/internal/store/redis.go @@ -0,0 +1,13 @@ +package store + +import "log" + +// Redis 持有 CacheDB 连接（Session / Rate Limit）。 +type Redis struct{ /* *redis.Client */ } + +// MustOpenRedis 建立 Redis 连接，失败即退出。 +func MustOpenRedis() *Redis { + // TODO: redis.NewClient(opts) + log.Println("[store] redis connected (stub)") + return &Redis{} +} diff --git a/sundynix-mcp-go/cmd/server/main.go b/sundynix-mcp-go/cmd/server/main.go new file mode 100644 index 0000000..c0d493c --- /dev/null +++ b/sundynix-mcp-go/cmd/server/main.go @@ -0,0 +1,19 @@ +// Command server 启动 sundynix-mcp-go —— 第 5 层 Go I/O 型 MCP 工具微服务。 +package main + +import ( + "log" + + "github.com/sundynix/sundynix-mcp-go/internal/mcp" + "github.com/sundynix/sundynix-mcp-go/internal/search" +) + +func main() { + engine := search.NewHybrid() // LLM Wiki 混合检索：Bleve + Milvus + Neo4j + gw := mcp.NewGateway(engine) + + log.Println("[mcp_go] serving MCP over sundynix.tools.go.*") + if err := gw.Serve(); err != nil { + log.Fatalf("[mcp_go] exit: %v", err) + } +} diff --git a/sundynix-mcp-go/config/config.yaml b/sundynix-mcp-go/config/config.yaml new file mode 100644 index 0000000..1bec789 --- /dev/null +++ b/sundynix-mcp-go/config/config.yaml @@ -0,0 +1,15 @@ +nats: + url: "nats://localhost:4222" + tool_subject: "sundynix.tools.go.*" + +bleve: + index_path: "./data/wiki.bleve" + +milvus: + addr: "localhost:19530" + collection: "wiki" + +neo4j: + uri: "bolt://localhost:7687" + user: "neo4j" + password: "sundynix" diff --git a/sundynix-mcp-go/go.mod b/sundynix-mcp-go/go.mod new file mode 100644 index 0000000..dfa14a7 --- /dev/null +++ b/sundynix-mcp-go/go.mod @@ -0,0 +1,10 @@ +module github.com/sundynix/sundynix-mcp-go + +go 1.23 + +require ( + github.com/blevesearch/bleve/v2 v2.4.2 + github.com/milvus-io/milvus-sdk-go/v2 v2.4.1 + github.com/neo4j/neo4j-go-driver/v5 v5.24.0 + github.com/nats-io/nats.go v1.37.0 +) diff --git a/sundynix-mcp-go/go.sum b/sundynix-mcp-go/go.sum new file mode 100644 index 0000000..181e66b --- /dev/null +++ b/sundynix-mcp-go/go.sum @@ -0,0 +1,4 @@ +github.com/blevesearch/bleve/v2 v2.4.2/go.mod h1:ATNKj7Yl2oJv/lGuF4kx39bST2dveX6w0th2FFYLkc8= +github.com/nats-io/nats.go v1.37.0/go.mod h1:Ubdu4Nh9exXdSz0RVWRFBbRfrbSxOYd26oF0wkWclB8= +github.com/neo4j/neo4j-go-driver/v5 v5.24.0/go.mod h1:Vff8OwT7QpLm7L2yYr85XNWe9Rbqlbeb9asNXJTHO4k= +github.com/qdrant/go-client v1.11.0/go.mod h1:j+OVRsJIZhOSRK2toPl8tTBOhwr4AxXCz9RACzv0JB4= diff --git a/sundynix-mcp-go/internal/external/apis.go b/sundynix-mcp-go/internal/external/apis.go new file mode 100644 index 0000000..fea78e5 --- /dev/null +++ b/sundynix-mcp-go/internal/external/apis.go @@ -0,0 +1,15 @@ +// Package external 封装对第三方外部 API 的统一调用（鉴权、重试、限流）。 +package external + +import "context" + +// Client 是外部 API 的统一出口。 +type Client struct{} + +func NewClient() *Client { return &Client{} } + +// Call 调用某个外部 API。 +func (c *Client) Call(ctx context.Context, name string, args map[string]any) (any, error) { + // TODO: 路由到具体 provider，统一鉴权/重试/限流/审计 + return nil, nil +} diff --git a/sundynix-mcp-go/internal/mcp/gateway.go b/sundynix-mcp-go/internal/mcp/gateway.go new file mode 100644 index 0000000..987dbf4 --- /dev/null +++ b/sundynix-mcp-go/internal/mcp/gateway.go @@ -0,0 +1,18 @@ +// Package mcp 实现 MCP 协议网关，把工具注册到 NATS 并响应调用。 +package mcp + +import "github.com/sundynix/sundynix-mcp-go/internal/search" + +// Gateway 暴露 MCP 协议端点（stdio / HTTP / NATS）。 +type Gateway struct { + search *search.Hybrid +} + +func NewGateway(s *search.Hybrid) *Gateway { return &Gateway{search: s} } + +// Serve 监听 sundynix.tools.go.* 并按 MCP 协议分发工具调用。 +func (g *Gateway) Serve() error { + // TODO: 注册工具清单 (wiki_search / render_doc / call_external_api ...) + // 订阅 NATS，按 MCP JSON-RPC 解析并路由 + select {} +} diff --git a/sundynix-mcp-go/internal/office/unioffice.go b/sundynix-mcp-go/internal/office/unioffice.go new file mode 100644 index 0000000..013453a --- /dev/null +++ b/sundynix-mcp-go/internal/office/unioffice.go @@ -0,0 +1,15 @@ +// Package office 基于 UniOffice 提供 Word/文档渲染能力。 +package office + +import "context" + +// Renderer 把结构化数据渲染为 docx/xlsx 等文档。 +type Renderer struct{} + +func NewRenderer() *Renderer { return &Renderer{} } + +// RenderDocx 生成 Word 文档并返回字节流。 +func (r *Renderer) RenderDocx(ctx context.Context, payload map[string]any) ([]byte, error) { + // TODO: 使用 unioffice/document 构建并序列化 + return nil, nil +} diff --git a/sundynix-mcp-go/internal/search/hybrid.go b/sundynix-mcp-go/internal/search/hybrid.go new file mode 100644 index 0000000..9ef1b39 --- /dev/null +++ b/sundynix-mcp-go/internal/search/hybrid.go @@ -0,0 +1,30 @@ +// Package search 实现 LLM Wiki 混合检索引擎。 +// Hybrid Search = Bleve(全文/BM25) + Milvus(向量) + Neo4j(知识图谱) 融合排序。 +package search + +import "context" + +// Hybrid 聚合三路检索后端并做 RRF/加权融合。 +type Hybrid struct { + // bleve *bleve.Index // Go 全文检索 + // milvus client.Client // Vector DB (Milvus Go SDK) + // neo4j neo4j.DriverWithContext // Knowledge Graph +} + +func NewHybrid() *Hybrid { + // TODO: 打开 bleve 索引；连接 Milvus；连接 Neo4j + return &Hybrid{} +} + +// Result 是融合后的检索结果。 +type Result struct { + ID string + Score float64 + Text string +} + +// Query 并行查询三路后端并融合排序。 +func (h *Hybrid) Query(ctx context.Context, q string, topK int) ([]Result, error) { + // TODO: 并发 bleve.Search + milvus.Search + neo4j Cypher，做 RRF 融合 + return nil, nil +} diff --git a/sundynix-mcp-py/config/config.yaml b/sundynix-mcp-py/config/config.yaml new file mode 100644 index 0000000..de330b4 --- /dev/null +++ b/sundynix-mcp-py/config/config.yaml @@ -0,0 +1,12 @@ +nats: + url: "nats://localhost:4222" + tool_subject: "sundynix.tools.py.*" + +sandbox: + runtime: "gvisor" # gvisor | kata + mem_limit: "512m" + timeout: "30s" + +interpreter: + image: "python:3.11-slim" + network_disabled: true diff --git a/sundynix-mcp-py/pyproject.toml b/sundynix-mcp-py/pyproject.toml new file mode 100644 index 0000000..c0118bf --- /dev/null +++ b/sundynix-mcp-py/pyproject.toml @@ -0,0 +1,18 @@ +[project] +name = "sundynix-mcp-py" +version = "0.1.0" +description = "sundynix-agentix · 第 5 层 Python 算法型 MCP 工具微服务" +requires-python = ">=3.11" +dependencies = [ + "mcp>=1.2.0", # MCP 协议 + "nats-py>=2.7.0", # 接入 NATS 骨干网 + "docker>=7.1.0", # Docker 隔离沙箱 / Code Interpreter + # "magic-pdf", # MinerU 多模态解析 (PaddleOCR)，按需安装 +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.hatch.build.targets.wheel] +packages = ["src/sundynix_mcp_py"] diff --git a/sundynix-mcp-py/src/sundynix_mcp_py/__init__.py b/sundynix-mcp-py/src/sundynix_mcp_py/__init__.py new file mode 100644 index 0000000..4de7ff7 --- /dev/null +++ b/sundynix-mcp-py/src/sundynix_mcp_py/__init__.py @@ -0,0 +1,3 @@ +"""sundynix_mcp_py — 第 5 层 Python 算法型 MCP 工具微服务。""" + +__version__ = "0.1.0" diff --git a/sundynix-mcp-py/src/sundynix_mcp_py/interpreter.py b/sundynix-mcp-py/src/sundynix_mcp_py/interpreter.py new file mode 100644 index 0000000..227e393 --- /dev/null +++ b/sundynix-mcp-py/src/sundynix_mcp_py/interpreter.py @@ -0,0 +1,12 @@ +"""Docker 隔离沙箱：Code Interpreter 执行不可信代码并采集产物。""" + +from __future__ import annotations + + +class CodeInterpreter: + """Docker 隔离沙箱 · Code Interpreter。""" + + async def execute(self, code: str, *, image: str = "python:3.11-slim") -> dict: + """在一次性 Docker 容器中执行代码，返回 stdout/stderr/artifacts。""" + # TODO: docker.from_env().containers.run(..., network_disabled=True, mem_limit=...) + return {"stdout": "", "stderr": "", "artifacts": []} diff --git a/sundynix-mcp-py/src/sundynix_mcp_py/main.py b/sundynix-mcp-py/src/sundynix_mcp_py/main.py new file mode 100644 index 0000000..bd89769 --- /dev/null +++ b/sundynix-mcp-py/src/sundynix_mcp_py/main.py @@ -0,0 +1,26 @@ +"""入口：启动 MCP 协议网关，把算法型工具注册到 NATS。""" + +from __future__ import annotations + +import asyncio +import logging + +from .mcp_gateway import McpGateway + +logging.basicConfig(level=logging.INFO) +log = logging.getLogger("mcp_py") + + +async def _run() -> None: + gateway = McpGateway() + await gateway.register_tools() # secure_sandbox / parse_document / run_code + log.info("[mcp_py] serving MCP over sundynix.tools.py.*") + await gateway.serve() + + +def main() -> None: + asyncio.run(_run()) + + +if __name__ == "__main__": + main() diff --git a/sundynix-mcp-py/src/sundynix_mcp_py/mcp_gateway.py b/sundynix-mcp-py/src/sundynix_mcp_py/mcp_gateway.py new file mode 100644 index 0000000..cd6e60b --- /dev/null +++ b/sundynix-mcp-py/src/sundynix_mcp_py/mcp_gateway.py @@ -0,0 +1,24 @@ +"""MCP 协议网关：注册算法型工具并经 NATS 分发调用。""" + +from __future__ import annotations + +from .interpreter import CodeInterpreter +from .mineru import MultimodalParser +from .sandbox import SecureSandbox + + +class McpGateway: + def __init__(self) -> None: + self.sandbox = SecureSandbox() # gVisor / KataVM + Static Code Guard + self.parser = MultimodalParser() # MinerU / PaddleOCR + self.interpreter = CodeInterpreter() # Docker 隔离沙箱 + + async def register_tools(self) -> None: + # TODO: 向 MCP server 注册工具 schema + ... + + async def serve(self) -> None: + # TODO: 连接 NATS，订阅 sundynix.tools.py.*，按 MCP JSON-RPC 路由 + import asyncio + + await asyncio.Event().wait() diff --git a/sundynix-mcp-py/src/sundynix_mcp_py/mineru.py b/sundynix-mcp-py/src/sundynix_mcp_py/mineru.py new file mode 100644 index 0000000..ff99575 --- /dev/null +++ b/sundynix-mcp-py/src/sundynix_mcp_py/mineru.py @@ -0,0 +1,12 @@ +"""MinerU 多模态解析：PDF/图片 → 结构化文本（PaddleOCR）。""" + +from __future__ import annotations + + +class MultimodalParser: + """MinerU · Multimodal Parser (PaddleOCR)。""" + + async def parse(self, file_path: str) -> dict: + """解析文档，返回结构化内容（标题/段落/表格/公式）。""" + # TODO: 调 magic-pdf / PaddleOCR 流水线 + return {"path": file_path, "blocks": []} diff --git a/sundynix-mcp-py/src/sundynix_mcp_py/sandbox.py b/sundynix-mcp-py/src/sundynix_mcp_py/sandbox.py new file mode 100644 index 0000000..a897153 --- /dev/null +++ b/sundynix-mcp-py/src/sundynix_mcp_py/sandbox.py @@ -0,0 +1,19 @@ +"""安全代码沙箱：gVisor / KataVM 强隔离 + 静态代码守卫。""" + +from __future__ import annotations + + +class SecureSandbox: + """Harness: Secure Code Sandbox。""" + + def static_guard(self, code: str) -> bool: + """静态代码守卫：危险调用/导入检测，返回是否放行。""" + # TODO: AST 扫描，拦截 os/subprocess/网络等高危调用 + return True + + async def run(self, code: str, *, runtime: str = "gvisor") -> str: + """在 gVisor/KataVM 隔离环境中执行代码。""" + if not self.static_guard(code): + raise PermissionError("static code guard rejected") + # TODO: 调度到 gVisor(runsc) / Kata 容器执行并回收 + return "" diff --git a/sundynix-shared/bus/bus.go b/sundynix-shared/bus/bus.go new file mode 100644 index 0000000..9e7645a --- /dev/null +++ b/sundynix-shared/bus/bus.go @@ -0,0 +1,166 @@ +// Package bus 封装 NATS JetStream 的连接、流声明、任务发布与消费。 +// Gateway 与 Dispatcher 共用这套真实收发逻辑，e2e 测试也直接覆盖它。 +package bus + +import ( + "context" + "fmt" + "time" + + "github.com/nats-io/nats.go" + "github.com/nats-io/nats.go/jetstream" + + "github.com/sundynix/sundynix-shared/contract" +) + +// Bus 持有 NATS 连接与 JetStream 上下文。 +type Bus struct { + nc *nats.Conn + js jetstream.JetStream +} + +// Connect 接入 NATS 骨干网并初始化 JetStream，使用默认重试参数。 +func Connect(url string) (*Bus, error) { + return ConnectWithRetry(url, 30, time.Second) +} + +// ConnectWithRetry 在 NATS 暂不可用时按固定间隔重试，容忍服务先于 NATS 启动。 +func ConnectWithRetry(url string, attempts int, interval time.Duration) (*Bus, error) { + var lastErr error + for i := 0; i < attempts; i++ { + nc, err := nats.Connect(url, + nats.Timeout(5*time.Second), + nats.RetryOnFailedConnect(true), + nats.MaxReconnects(-1), + nats.ReconnectWait(interval), + ) + if err != nil { + lastErr = err + time.Sleep(interval) + continue + } + // RetryOnFailedConnect 下 Connect 可能立即返回但尚未连上，等待真正建立。 + if nc.Status() != nats.CONNECTED { + if !waitConnected(nc, 5*time.Second) { + lastErr = fmt.Errorf("nats not connected within timeout") + nc.Close() + time.Sleep(interval) + continue + } + } + js, err := jetstream.New(nc) + if err != nil { + nc.Close() + return nil, fmt.Errorf("jetstream init: %w", err) + } + return &Bus{nc: nc, js: js}, nil + } + return nil, fmt.Errorf("nats connect after %d attempts: %w", attempts, lastErr) +} + +func waitConnected(nc *nats.Conn, d time.Duration) bool { + deadline := time.Now().Add(d) + for time.Now().Before(deadline) { + if nc.Status() == nats.CONNECTED { + return true + } + time.Sleep(50 * time.Millisecond) + } + return nc.Status() == nats.CONNECTED +} + +// Close 关闭底层连接。 +func (b *Bus) Close() { + if b.nc != nil { + b.nc.Close() + } +} + +// EnsureTaskStream 幂等地创建/更新任务流，捕获 sundynix.tasks.>。 +func (b *Bus) EnsureTaskStream(ctx context.Context) error { + _, err := b.js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{ + Name: contract.StreamTasks, + Subjects: []string{contract.SubjectTasksAll}, + Storage: jetstream.FileStorage, + }) + return err +} + +// PublishTask 把任务发布到 sundynix.tasks.，返回序列号。 +func (b *Bus) PublishTask(ctx context.Context, t *contract.Task) (uint64, error) { + data, err := t.Marshal() + if err != nil { + return 0, err + } + ack, err := b.js.Publish(ctx, contract.TaskSubject(t.ID), data) + if err != nil { + return 0, fmt.Errorf("publish task: %w", err) + } + return ack.Sequence, nil +} + +// ---- Token 流回流（core NATS 零拷贝字节管道）---- + +// PublishToken 把一个推理 Token 以 core NATS 写到 sundynix.streams.。 +func (b *Bus) PublishToken(taskID string, token []byte) error { + return b.nc.Publish(contract.StreamSubject(taskID), token) +} + +// CompleteStream 发送 Token 流结束信号（空体 + 结束头）。 +func (b *Bus) CompleteStream(taskID string) error { + msg := nats.NewMsg(contract.StreamSubject(taskID)) + msg.Header.Set(contract.HeaderStreamEnd, "1") + return b.nc.PublishMsg(msg) +} + +// SubscribeTokens 订阅某 task 的 Token 流。每个 Token 触发 onToken； +// 收到结束信号后触发 onDone。返回的 unsub 用于退订。 +// 注意：core NATS 无持久化，订阅须在 Token 产生前建立（SSE 客户端先连）。 +func (b *Bus) SubscribeTokens(taskID string, onToken func([]byte), onDone func()) (unsub func() error, err error) { + sub, err := b.nc.Subscribe(contract.StreamSubject(taskID), func(m *nats.Msg) { + if m.Header.Get(contract.HeaderStreamEnd) == "1" { + onDone() + return + } + // 拷贝，避免 nats 复用底层 buffer。 + tok := make([]byte, len(m.Data)) + copy(tok, m.Data) + onToken(tok) + }) + if err != nil { + return nil, fmt.Errorf("subscribe tokens: %w", err) + } + return sub.Unsubscribe, nil +} + +// TaskHandler 处理一个消费到的任务。 +type TaskHandler func(ctx context.Context, t *contract.Task) error + +// ConsumeTasks 在持久消费者上消费任务，队列组内负载均衡。 +// 返回的 stop 函数用于优雅停止消费。 +func (b *Bus) ConsumeTasks(ctx context.Context, h TaskHandler) (stop func(), err error) { + cons, err := b.js.CreateOrUpdateConsumer(ctx, contract.StreamTasks, jetstream.ConsumerConfig{ + Durable: contract.ConsumerDurable, + AckPolicy: jetstream.AckExplicitPolicy, + FilterSubject: contract.SubjectTasksAll, + }) + if err != nil { + return nil, fmt.Errorf("create consumer: %w", err) + } + cc, err := cons.Consume(func(msg jetstream.Msg) { + t, err := contract.Unmarshal(msg.Data()) + if err != nil { + _ = msg.Term() // 脏数据，丢弃不重投 + return + } + if err := h(ctx, t); err != nil { + _ = msg.NakWithDelay(time.Second) // 处理失败，延迟重投 + return + } + _ = msg.Ack() + }) + if err != nil { + return nil, fmt.Errorf("consume: %w", err) + } + return cc.Stop, nil +} diff --git a/sundynix-shared/bus/bus_e2e_test.go b/sundynix-shared/bus/bus_e2e_test.go new file mode 100644 index 0000000..8dd1541 --- /dev/null +++ b/sundynix-shared/bus/bus_e2e_test.go @@ -0,0 +1,149 @@ +package bus_test + +import ( + "context" + "encoding/json" + "testing" + "time" + + natsserver "github.com/nats-io/nats-server/v2/server" + natstest "github.com/nats-io/nats-server/v2/test" + + "github.com/sundynix/sundynix-shared/bus" + "github.com/sundynix/sundynix-shared/contract" +) + +// startEmbeddedNATS 启动一个内嵌、开启 JetStream 的 NATS 服务器，免 Docker。 +func startEmbeddedNATS(t *testing.T) string { + t.Helper() + opts := natstest.DefaultTestOptions + opts.Port = -1 // 随机端口 + opts.JetStream = true + opts.StoreDir = t.TempDir() + srv := natstest.RunServer(&opts) + if !srv.ReadyForConnections(5 * time.Second) { + t.Fatal("embedded NATS not ready") + } + t.Cleanup(srv.Shutdown) + _ = natsserver.Server{} // 触发包引用 + return srv.ClientURL() +} + +// TestTaskRoundTrip 模拟 Gateway 发布 → NATS → Dispatcher 消费 的完整任务流。 +func TestTaskRoundTrip(t *testing.T) { + url := startEmbeddedNATS(t) + + // --- Gateway 侧：连接并声明任务流 --- + gw, err := bus.Connect(url) + if err != nil { + t.Fatalf("gateway connect: %v", err) + } + defer gw.Close() + + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + + if err := gw.EnsureTaskStream(ctx); err != nil { + t.Fatalf("ensure stream: %v", err) + } + + // --- Dispatcher 侧：连接并开始消费 --- + dp, err := bus.Connect(url) + if err != nil { + t.Fatalf("dispatcher connect: %v", err) + } + defer dp.Close() + + got := make(chan *contract.Task, 1) + stop, err := dp.ConsumeTasks(ctx, func(_ context.Context, task *contract.Task) error { + got <- task + return nil + }) + if err != nil { + t.Fatalf("consume: %v", err) + } + defer stop() + + // --- Gateway 发布一个任务 --- + want := &contract.Task{ + ID: "task_demo_001", + Graph: json.RawMessage(`{"nodes":[{"id":"n1","type":"agent"}],"edges":[]}`), + Meta: map[string]any{"user": "wt"}, + } + seq, err := gw.PublishTask(ctx, want) + if err != nil { + t.Fatalf("publish: %v", err) + } + if seq == 0 { + t.Fatal("expected non-zero stream sequence") + } + + // --- 断言 Dispatcher 收到同一个任务 --- + select { + case task := <-got: + if task.ID != want.ID { + t.Fatalf("task id = %q, want %q", task.ID, want.ID) + } + if task.Meta["user"] != "wt" { + t.Fatalf("task meta lost: %+v", task.Meta) + } + t.Logf("✓ 任务流打通：Gateway publish (seq=%d) → NATS → Dispatcher consume，task_id=%s", seq, task.ID) + case <-time.After(5 * time.Second): + t.Fatal("timeout: dispatcher 未收到任务") + } +} + +// TestTokenStreamRoundTrip 模拟 Dispatcher 回流 Token → Gateway 订阅 的流式闭环。 +func TestTokenStreamRoundTrip(t *testing.T) { + url := startEmbeddedNATS(t) + + // Gateway 侧：先订阅（core NATS 无持久化，须先连）。 + gw, err := bus.Connect(url) + if err != nil { + t.Fatalf("gateway connect: %v", err) + } + defer gw.Close() + + const taskID = "task_stream_001" + var got []string + done := make(chan struct{}) + unsub, err := gw.SubscribeTokens(taskID, + func(tok []byte) { got = append(got, string(tok)) }, + func() { close(done) }, + ) + if err != nil { + t.Fatalf("subscribe tokens: %v", err) + } + defer func() { _ = unsub() }() + + // Dispatcher 侧：逐 Token 回流后发结束信号。 + dp, err := bus.Connect(url) + if err != nil { + t.Fatalf("dispatcher connect: %v", err) + } + defer dp.Close() + + want := []string{"Hello", " ", "Agent", "!"} + for _, tok := range want { + if err := dp.PublishToken(taskID, []byte(tok)); err != nil { + t.Fatalf("publish token: %v", err) + } + } + if err := dp.CompleteStream(taskID); err != nil { + t.Fatalf("complete stream: %v", err) + } + + select { + case <-done: + joined := "" + for _, s := range got { + joined += s + } + if joined != "Hello Agent!" { + t.Fatalf("token stream = %q, want %q", joined, "Hello Agent!") + } + t.Logf("✓ Token 流闭环：Dispatcher 回流 %d 个 token → Gateway 拼回 %q", len(got), joined) + case <-time.After(5 * time.Second): + t.Fatal("timeout: 未收到流结束信号") + } +} diff --git a/sundynix-shared/cmd/devnats/main.go b/sundynix-shared/cmd/devnats/main.go new file mode 100644 index 0000000..46ed7f4 --- /dev/null +++ b/sundynix-shared/cmd/devnats/main.go @@ -0,0 +1,48 @@ +// Command devnats 启动一个内嵌、开启 JetStream 的本地 NATS 服务器， +// 用于无 Docker 环境下的本地联调（生产环境用 deploy/nats 的真实集群）。 +package main + +import ( + "log" + "os" + "os/signal" + "syscall" + + "github.com/nats-io/nats-server/v2/server" +) + +func main() { + storeDir, err := os.MkdirTemp("", "sundynix-jetstream-") + if err != nil { + log.Fatalf("[devnats] tempdir: %v", err) + } + opts := &server.Options{ + Host: "127.0.0.1", + Port: 4222, + JetStream: true, + StoreDir: storeDir, + } + ns, err := server.NewServer(opts) + if err != nil { + log.Fatalf("[devnats] new server: %v", err) + } + go ns.Start() + if !ns.ReadyForConnections(5e9) { + log.Fatal("[devnats] not ready") + } + log.Printf("[devnats] JetStream NATS ready on %s (store=%s)", ns.ClientURL(), storeDir) + + sig := make(chan os.Signal, 1) + signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM) + <-sig + log.Println("[devnats] shutting down") + shutdownQuietly(ns) + _ = os.RemoveAll(storeDir) +} + +// shutdownQuietly 容忍内嵌 server 退出时偶发的 "close of nil channel" panic。 +func shutdownQuietly(ns *server.Server) { + defer func() { _ = recover() }() + ns.Shutdown() + ns.WaitForShutdown() +} diff --git a/sundynix-shared/contract/task.go b/sundynix-shared/contract/task.go new file mode 100644 index 0000000..60c6478 --- /dev/null +++ b/sundynix-shared/contract/task.go @@ -0,0 +1,41 @@ +// Package contract 是 Gateway / Dispatcher / MCP 之间的共享契约： +// Task 数据结构与 NATS subject 命名约定。 +package contract + +import "encoding/json" + +// NATS subject / stream 约定（与 README、各服务 config 保持一致）。 +const ( + StreamTasks = "SUNDYNIX_TASKS" // JetStream stream 名 + SubjectTasks = "sundynix.tasks" // 任务发布主题前缀；实际为 sundynix.tasks. + SubjectTasksAll = "sundynix.tasks.>" // stream 捕获的通配 + SubjectStream = "sundynix.streams" // Token 回流前缀；实际 sundynix.streams. + ConsumerDurable = "dispatchers" // Dispatcher 持久消费者（队列组负载均衡） + + // HeaderStreamEnd 是 Token 流的结束信号（core NATS 消息头）。 + // 置为 "1" 的消息体为空，表示该 task 的 Token 流结束。 + HeaderStreamEnd = "X-Stream-End" +) + +// Task 是 DSL 解析组装后的可调度任务，在 NATS 上以 JSON 传输。 +type Task struct { + ID string `json:"id"` + Graph json.RawMessage `json:"graph"` // React Flow 导出的 Agent 编排图 + Meta map[string]any `json:"meta,omitempty"` +} + +// TaskSubject 返回某任务的发布主题。 +func TaskSubject(id string) string { return SubjectTasks + "." + id } + +// StreamSubject 返回某任务的 Token 回流主题。 +func StreamSubject(id string) string { return SubjectStream + "." + id } + +// Marshal / Unmarshal 便捷方法。 +func (t *Task) Marshal() ([]byte, error) { return json.Marshal(t) } +func Unmarshal(b []byte) (*Task, error) { + var t Task + if err := json.Unmarshal(b, &t); err != nil { + return nil, err + } + return &t, nil +} diff --git a/sundynix-shared/go.mod b/sundynix-shared/go.mod new file mode 100644 index 0000000..d625d30 --- /dev/null +++ b/sundynix-shared/go.mod @@ -0,0 +1,20 @@ +module github.com/sundynix/sundynix-shared + +go 1.23 + +require ( + github.com/nats-io/nats-server/v2 v2.10.20 + github.com/nats-io/nats.go v1.37.0 +) + +require ( + github.com/klauspost/compress v1.17.9 // indirect + github.com/minio/highwayhash v1.0.3 // indirect + github.com/nats-io/jwt/v2 v2.5.8 // indirect + github.com/nats-io/nkeys v0.4.7 // indirect + github.com/nats-io/nuid v1.0.1 // indirect + golang.org/x/crypto v0.26.0 // indirect + golang.org/x/sys v0.24.0 // indirect + golang.org/x/text v0.17.0 // indirect + golang.org/x/time v0.6.0 // indirect +) diff --git a/sundynix-shared/go.sum b/sundynix-shared/go.sum new file mode 100644 index 0000000..20df5e2 --- /dev/null +++ b/sundynix-shared/go.sum @@ -0,0 +1,23 @@ +github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA= +github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw= +github.com/minio/highwayhash v1.0.3 h1:kbnuUMoHYyVl7szWjSxJnxw11k2U709jqFPPmIUyD6Q= +github.com/minio/highwayhash v1.0.3/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ= +github.com/nats-io/jwt/v2 v2.5.8 h1:uvdSzwWiEGWGXf+0Q+70qv6AQdvcvxrv9hPM0RiPamE= +github.com/nats-io/jwt/v2 v2.5.8/go.mod h1:ZdWS1nZa6WMZfFwwgpEaqBV8EPGVgOTDHN/wTbz0Y5A= +github.com/nats-io/nats-server/v2 v2.10.20 h1:CXDTYNHeBiAKBTAIP2gjpgbWap2GhATnTLgP8etyvEI= +github.com/nats-io/nats-server/v2 v2.10.20/go.mod h1:hgcPnoUtMfxz1qVOvLZGurVypQ+Cg6GXVXjG53iHk+M= +github.com/nats-io/nats.go v1.37.0 h1:07rauXbVnnJvv1gfIyghFEo6lUcYRY0WXc3x7x0vUxE= +github.com/nats-io/nats.go v1.37.0/go.mod h1:Ubdu4Nh9exXdSz0RVWRFBbRfrbSxOYd26oF0wkWclB8= +github.com/nats-io/nkeys v0.4.7 h1:RwNJbbIdYCoClSDNY7QVKZlyb/wfT6ugvFCiKy6vDvI= +github.com/nats-io/nkeys v0.4.7/go.mod h1:kqXRgRDPlGy7nGaEDMuYzmiJCIAAWDK0IMBtDmGD0nc= +github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw= +github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c= +golang.org/x/crypto v0.26.0 h1:RrRspgV4mU+YwB4FYnuBoKsUapNIL5cohGAmSH3azsw= +golang.org/x/crypto v0.26.0/go.mod h1:GY7jblb9wI+FOo5y8/S2oY4zWP07AkOJ4+jxCqdqn54= +golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.24.0 h1:Twjiwq9dn6R1fQcyiK+wQyHWfaz/BJB+YIpzU/Cv3Xg= +golang.org/x/sys v0.24.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/text v0.17.0 h1:XtiM5bkSOt+ewxlOE/aE/AKEHibwj/6gvWMl9Rsh0Qc= +golang.org/x/text v0.17.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY= +golang.org/x/time v0.6.0 h1:eTDhh4ZXt5Qf0augr54TN6suAUudPcawVZeIAPU7D4U= +golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=