feat(voice): 桌面端 JARVIS 语音坞——麦克风上行 + TTS 播放队列(Phase 1 收口)

右下角悬浮麦克风:点按说话→转写→提交任务→跳运行页→朗读回答,接入既有运行·观测流。

- lib/voice.ts: VoiceClient——一条 WS 承载上行 PCM16k/下行转写/下行 TTS PCM24k;
  ScriptProcessorNode 采麦(48k→16k 降采样+Float32→Int16);AudioBufferSourceNode
  排队调度做无缝连续朗读;start/end/barge_in/bye 控制;ready/transcript/task/speaking/tts_end
- shell/VoiceDock.tsx: 悬浮麦克风 UI(聆听/思考/朗读态 + 转写气泡 + 打断),懒建客户端(用户手势启 AudioContext)
- App.tsx: onVoiceTask 把语音 task 挂回 attachRun(后端已提交,前端只 attach);挂载 VoiceDock

真识别/合成需部署联调(真连火山+admin录入语音配置);本地过 tsc/build/68 测试。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Blizzard
2026-07-22 09:29:47 +08:00
parent 6fe0a58f1b
commit 9458bf21f3
3 changed files with 375 additions and 0 deletions
+18
View File
@@ -16,6 +16,7 @@ import { Home } from "./views/Home";
import { CommandPalette, type Command } from "./components/CommandPalette";
import { UpdateBanner } from "./components/UpdateBanner";
import { Login } from "./views/Login";
import { VoiceDock } from "./shell/VoiceDock";
import { submitTask, generateReport, streamTokens, streamExec, taskStatus, listRuns, authMe, logout, tenantCurrent, myTenants, switchTenant, spaceCurrent, mySpaces, switchSpace, createSpace, type Identity, type AuthUser, type TenantCtx, type MyTenant, type SpaceCtx, type MySpace } from "./lib/api";
import type { TaskDsl } from "./lib/dsl";
import { emptyRun, type RunState } from "./lib/run";
@@ -283,6 +284,22 @@ export default function App() {
[identity, attachRun],
);
// 语音任务:后端已在 ASR 转写后自行提交(走 SubmitTask 同一关卡),前端只需挂回它的运行流。
// 与 onRun 的区别:不再 submit,直接 attach(task 已在后端跑)——跳「运行·观测」看轨迹/输出。
const onVoiceTask = useCallback(
(taskId: string) => {
closeRef.current?.();
execCloseRef.current?.();
stopPoll();
const t0 = Date.now();
setRun({ phase: "streaming", taskId, output: "", events: [{ t: 0, label: `语音任务 ${taskId}` }], exec: [] });
setFocusRun(null);
setView("runs");
attachRun(taskId, t0, "语音任务已跑完");
},
[attachRun],
);
// 恢复在途待审任务:登录后若存在 waiting 任务且当前无 live run,挂回它 → 全局审批条重现,
// 用户刷新页面/重开 app 也能继续批准(HITL 持久化中断后审批可跨重启、可等数小时)。
const restoredRef = useRef(false);
@@ -360,6 +377,7 @@ export default function App() {
selfUserId={user?.id ?? ""}
/>
<InviteMembers open={inviteOpen} onClose={() => setInviteOpen(false)} tenantName={tenant?.tenant?.name ?? ""} />
<VoiceDock onTask={onVoiceTask} />
</div>
</ToastProvider>
);