Merge remote-tracking branch 'origin/main' into website

update version
Merge pull request #5786 from ConnectAI-E/feature/realtime-chat
2025-08-31 03:09:04 +08:00 · 2024-11-11 13:26:21 +08:00 · 2024-11-11 13:26:08 +08:00 · 2024-11-11 13:19:26 +08:00 · 2024-11-11 13:15:09 +08:00 · 2024-11-11 13:13:09 +08:00
76 changed files with 2895 additions and 844 deletions
--- a/README.md
+++ b/README.md
@@ -31,7 +31,7 @@ One-Click to get a well-designed cross-platform ChatGPT web UI, with GPT3, GPT4
 [MacOS-image]: https://img.shields.io/badge/-MacOS-black?logo=apple
 [Linux-image]: https://img.shields.io/badge/-Linux-333?logo=ubuntu

-[<img src="https://vercel.com/button" alt="Deploy on Zeabur" height="30">](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FChatGPTNextWeb%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=nextchat&repository-name=NextChat) [<img src="https://zeabur.com/button.svg" alt="Deploy on Zeabur" height="30">](https://zeabur.com/templates/ZBUEFA)  [<img src="https://gitpod.io/button/open-in-gitpod.svg" alt="Open in Gitpod" height="30">](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)
+[<img src="https://vercel.com/button" alt="Deploy on Vercel" height="30">](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FChatGPTNextWeb%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=nextchat&repository-name=NextChat) [<img src="https://zeabur.com/button.svg" alt="Deploy on Zeabur" height="30">](https://zeabur.com/templates/ZBUEFA)  [<img src="https://gitpod.io/button/open-in-gitpod.svg" alt="Open in Gitpod" height="30">](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web) [<img src="https://img.shields.io/badge/BT_Deploy-Install-20a53a" alt="BT Deply Install" height="30">](https://www.bt.cn/new/download.html) [<img src="https://svgshare.com/i/1AVg.svg" alt="Deploy to Alibaba Cloud" height="30">](https://computenest.aliyun.com/market/service-f1c9b75e59814dc49d52)

 [<img src="https://github.com/user-attachments/assets/903482d4-3e87-4134-9af1-f2588fa90659" height="60" width="288" >](https://monica.im/?utm=nxcrp)

@@ -96,10 +96,11 @@ For enterprise inquiries, please contact: **business@nextchat.dev**
 - [x] Artifacts: Easily preview, copy and share generated content/webpages through a separate window [#5092](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/pull/5092)
 - [x] Plugins: support network search, calculator, any other apis etc. [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
  - [x] network search, calculator, any other apis etc. [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
+- [x] Supports Realtime Chat [#5672](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5672)
 - [ ] local knowledge base

 ## What's New
-
+- 🚀 v2.15.8 Now supports Realtime Chat [#5672](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5672)
 - 🚀 v2.15.4 The Application supports using Tauri fetch LLM API, MORE SECURITY! [#5379](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5379)
 - 🚀 v2.15.0 Now supports Plugins! Read this: [NextChat-Awesome-Plugins](https://github.com/ChatGPTNextWeb/NextChat-Awesome-Plugins)
 - 🚀 v2.14.0 Now supports  Artifacts & SD 
@@ -134,10 +135,11 @@ For enterprise inquiries, please contact: **business@nextchat.dev**
 - [x] Artifacts: 通过独立窗口，轻松预览、复制和分享生成的内容/可交互网页 [#5092](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/pull/5092)
 - [x] 插件机制，支持`联网搜索`、`计算器`、调用其他平台 api [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
   - [x] 支持联网搜索、计算器、调用其他平台 api [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
+ - [x] 支持 Realtime Chat [#5672](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5672)
 - [ ] 本地知识库

 ## 最新动态
-
+- 🚀 v2.15.8 现在支持Realtime Chat [#5672](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5672)
 - 🚀 v2.15.4 客户端支持Tauri本地直接调用大模型API，更安全！[#5379](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5379)
 - 🚀 v2.15.0 现在支持插件功能了！了解更多：[NextChat-Awesome-Plugins](https://github.com/ChatGPTNextWeb/NextChat-Awesome-Plugins)
 - 🚀 v2.14.0 现在支持 Artifacts & SD 了。
@@ -301,6 +303,14 @@ iflytek Api Key.

 iflytek Api Secret.

+### `CHATGLM_API_KEY` (optional)
+
+ChatGLM Api Key.
+
+### `CHATGLM_URL` (optional)
+
+ChatGLM Api Url.
+
 ### `HIDE_USER_API_KEY` (optional)

 > Default: Empty
@@ -397,6 +407,9 @@ yarn dev

 > [简体中文 > 如何部署到私人服务器](./README_CN.md#部署)

+### BT Install
+> [简体中文 > 如何通过宝塔一键部署](./docs/bt-cn.md)
+
 ### Docker (Recommended)

 ```shell
--- a/README_CN.md
+++ b/README_CN.md
@@ -184,6 +184,13 @@ ByteDance Api Url.

 讯飞星火Api Secret.

+### `CHATGLM_API_KEY` (可选)
+
+ChatGLM Api Key.
+
+### `CHATGLM_URL` (可选)
+
+ChatGLM Api Url.


 ### `HIDE_USER_API_KEY` （可选）
@@ -264,6 +271,9 @@ BASE_URL=https://b.nextweb.fun/api/proxy

 ## 部署

+### 宝塔面板部署
+> [简体中文 > 如何通过宝塔一键部署](./docs/bt-cn.md)
+
 ### 容器部署 （推荐）

 > Docker 版本需要在 20 及其以上，否则会提示找不到镜像。
--- a/app/api/[provider]/[...path]/route.ts
+++ b/app/api/[provider]/[...path]/route.ts
@@ -11,6 +11,7 @@ import { handle as moonshotHandler } from "../../moonshot";
 import { handle as stabilityHandler } from "../../stability";
 import { handle as iflytekHandler } from "../../iflytek";
 import { handle as xaiHandler } from "../../xai";
+import { handle as chatglmHandler } from "../../glm";
 import { handle as proxyHandler } from "../../proxy";

 async function handle(
@@ -41,6 +42,8 @@ async function handle(
      return iflytekHandler(req, { params });
    case ApiPath.XAI:
      return xaiHandler(req, { params });
+    case ApiPath.ChatGLM:
+      return chatglmHandler(req, { params });
    case ApiPath.OpenAI:
      return openaiHandler(req, { params });
    default:
--- a/app/api/auth.ts
+++ b/app/api/auth.ts
@@ -95,6 +95,9 @@ export function auth(req: NextRequest, modelProvider: ModelProvider) {
      case ModelProvider.XAI:
        systemApiKey = serverConfig.xaiApiKey;
        break;
+      case ModelProvider.ChatGLM:
+        systemApiKey = serverConfig.chatglmApiKey;
+        break;
      case ModelProvider.GPT:
      default:
        if (req.nextUrl.pathname.includes("azure/deployments")) {
--- a/app/api/common.ts
+++ b/app/api/common.ts
@@ -1,8 +1,8 @@
 import { NextRequest, NextResponse } from "next/server";
 import { getServerSideConfig } from "../config/server";
 import { OPENAI_BASE_URL, ServiceProvider } from "../constant";
-import { isModelAvailableInServer } from "../utils/model";
 import { cloudflareAIGatewayUrl } from "../utils/cloudflare";
+import { getModelProvider, isModelAvailableInServer } from "../utils/model";

 const serverConfig = getServerSideConfig();

@@ -71,7 +71,7 @@ export async function requestOpenai(req: NextRequest) {
        .filter((v) => !!v && !v.startsWith("-") && v.includes(modelName))
        .forEach((m) => {
          const [fullName, displayName] = m.split("=");
-          const [_, providerName] = fullName.split("@");
+          const [_, providerName] = getModelProvider(fullName);
          if (providerName === "azure" && !displayName) {
            const [_, deployId] = (serverConfig?.azureUrl ?? "").split(
              "deployments/",
--- a/app/api/glm.ts
+++ b/app/api/glm.ts
@@ -0,0 +1,129 @@
+import { getServerSideConfig } from "@/app/config/server";
+import {
+  CHATGLM_BASE_URL,
+  ApiPath,
+  ModelProvider,
+  ServiceProvider,
+} from "@/app/constant";
+import { prettyObject } from "@/app/utils/format";
+import { NextRequest, NextResponse } from "next/server";
+import { auth } from "@/app/api/auth";
+import { isModelAvailableInServer } from "@/app/utils/model";
+
+const serverConfig = getServerSideConfig();
+
+export async function handle(
+  req: NextRequest,
+  { params }: { params: { path: string[] } },
+) {
+  console.log("[GLM Route] params ", params);
+
+  if (req.method === "OPTIONS") {
+    return NextResponse.json({ body: "OK" }, { status: 200 });
+  }
+
+  const authResult = auth(req, ModelProvider.ChatGLM);
+  if (authResult.error) {
+    return NextResponse.json(authResult, {
+      status: 401,
+    });
+  }
+
+  try {
+    const response = await request(req);
+    return response;
+  } catch (e) {
+    console.error("[GLM] ", e);
+    return NextResponse.json(prettyObject(e));
+  }
+}
+
+async function request(req: NextRequest) {
+  const controller = new AbortController();
+
+  // alibaba use base url or just remove the path
+  let path = `${req.nextUrl.pathname}`.replaceAll(ApiPath.ChatGLM, "");
+
+  let baseUrl = serverConfig.chatglmUrl || CHATGLM_BASE_URL;
+
+  if (!baseUrl.startsWith("http")) {
+    baseUrl = `https://${baseUrl}`;
+  }
+
+  if (baseUrl.endsWith("/")) {
+    baseUrl = baseUrl.slice(0, -1);
+  }
+
+  console.log("[Proxy] ", path);
+  console.log("[Base Url]", baseUrl);
+
+  const timeoutId = setTimeout(
+    () => {
+      controller.abort();
+    },
+    10 * 60 * 1000,
+  );
+
+  const fetchUrl = `${baseUrl}${path}`;
+  console.log("[Fetch Url] ", fetchUrl);
+  const fetchOptions: RequestInit = {
+    headers: {
+      "Content-Type": "application/json",
+      Authorization: req.headers.get("Authorization") ?? "",
+    },
+    method: req.method,
+    body: req.body,
+    redirect: "manual",
+    // @ts-ignore
+    duplex: "half",
+    signal: controller.signal,
+  };
+
+  // #1815 try to refuse some request to some models
+  if (serverConfig.customModels && req.body) {
+    try {
+      const clonedBody = await req.text();
+      fetchOptions.body = clonedBody;
+
+      const jsonBody = JSON.parse(clonedBody) as { model?: string };
+
+      // not undefined and is false
+      if (
+        isModelAvailableInServer(
+          serverConfig.customModels,
+          jsonBody?.model as string,
+          ServiceProvider.ChatGLM as string,
+        )
+      ) {
+        return NextResponse.json(
+          {
+            error: true,
+            message: `you are not allowed to use ${jsonBody?.model} model`,
+          },
+          {
+            status: 403,
+          },
+        );
+      }
+    } catch (e) {
+      console.error(`[GLM] filter`, e);
+    }
+  }
+  try {
+    const res = await fetch(fetchUrl, fetchOptions);
+
+    // to prevent browser prompt for credentials
+    const newHeaders = new Headers(res.headers);
+    newHeaders.delete("www-authenticate");
+    // to disable nginx buffering
+    newHeaders.set("X-Accel-Buffering", "no");
+
+    return new Response(res.body, {
+      status: res.status,
+      statusText: res.statusText,
+      headers: newHeaders,
+    });
+  } finally {
+    clearTimeout(timeoutId);
+  }
+}
--- a/app/api/proxy.ts
+++ b/app/api/proxy.ts
@@ -1,4 +1,5 @@
 import { NextRequest, NextResponse } from "next/server";
+import { getServerSideConfig } from "@/app/config/server";

 export async function handle(
  req: NextRequest,
@@ -9,6 +10,7 @@ export async function handle(
  if (req.method === "OPTIONS") {
    return NextResponse.json({ body: "OK" }, { status: 200 });
  }
+  const serverConfig = getServerSideConfig();

  // remove path params from searchParams
  req.nextUrl.searchParams.delete("path");
@@ -31,6 +33,18 @@ export async function handle(
      return true;
    }),
  );
+  // if dalle3 use openai api key
+    const baseUrl = req.headers.get("x-base-url");
+    if (baseUrl?.includes("api.openai.com")) {
+      if (!serverConfig.apiKey) {
+        return NextResponse.json(
+          { error: "OpenAI API key not configured" },
+          { status: 500 },
+        );
+      }
+      headers.set("Authorization", `Bearer ${serverConfig.apiKey}`);
+    }
+
  const controller = new AbortController();
  const fetchOptions: RequestInit = {
    headers,
--- a/app/client/api.ts
+++ b/app/client/api.ts
@@ -21,6 +21,7 @@ import { HunyuanApi } from "./platforms/tencent";
 import { MoonshotApi } from "./platforms/moonshot";
 import { SparkApi } from "./platforms/iflytek";
 import { XAIApi } from "./platforms/xai";
+import { ChatGLMApi } from "./platforms/glm";

 export const ROLES = ["system", "user", "assistant"] as const;
 export type MessageRole = (typeof ROLES)[number];
@@ -69,7 +70,7 @@ export interface ChatOptions {
  config: LLMConfig;

  onUpdate?: (message: string, chunk: string) => void;
-  onFinish: (message: string) => void;
+  onFinish: (message: string, responseRes: Response) => void;
  onError?: (err: Error) => void;
  onController?: (controller: AbortController) => void;
  onBeforeTool?: (tool: ChatMessageTool) => void;
@@ -156,6 +157,9 @@ export class ClientApi {
      case ModelProvider.XAI:
        this.llm = new XAIApi();
        break;
+      case ModelProvider.ChatGLM:
+        this.llm = new ChatGLMApi();
+        break;
      default:
        this.llm = new ChatGPTApi();
    }
@@ -244,6 +248,7 @@ export function getHeaders(ignoreHeaders: boolean = false) {
    const isMoonshot = modelConfig.providerName === ServiceProvider.Moonshot;
    const isIflytek = modelConfig.providerName === ServiceProvider.Iflytek;
    const isXAI = modelConfig.providerName === ServiceProvider.XAI;
+    const isChatGLM = modelConfig.providerName === ServiceProvider.ChatGLM;
    const isEnabledAccessControl = accessStore.enabledAccessControl();
    const apiKey = isGoogle
      ? accessStore.googleApiKey
@@ -259,6 +264,8 @@ export function getHeaders(ignoreHeaders: boolean = false) {
      ? accessStore.moonshotApiKey
      : isXAI
      ? accessStore.xaiApiKey
+      : isChatGLM
+      ? accessStore.chatglmApiKey
      : isIflytek
      ? accessStore.iflytekApiKey && accessStore.iflytekApiSecret
        ? accessStore.iflytekApiKey + ":" + accessStore.iflytekApiSecret
@@ -274,6 +281,7 @@ export function getHeaders(ignoreHeaders: boolean = false) {
      isMoonshot,
      isIflytek,
      isXAI,
+      isChatGLM,
      apiKey,
      isEnabledAccessControl,
    };
@@ -338,6 +346,8 @@ export function getClientApi(provider: ServiceProvider): ClientApi {
      return new ClientApi(ModelProvider.Iflytek);
    case ServiceProvider.XAI:
      return new ClientApi(ModelProvider.XAI);
+    case ServiceProvider.ChatGLM:
+      return new ClientApi(ModelProvider.ChatGLM);
    default:
      return new ClientApi(ModelProvider.GPT);
  }
--- a/app/client/platforms/alibaba.ts
+++ b/app/client/platforms/alibaba.ts
@@ -143,6 +143,7 @@ export class QwenApi implements LLMApi {
        let responseText = "";
        let remainText = "";
        let finished = false;
+        let responseRes: Response;

        // animate response to make it looks smooth
        function animateResponseText() {
@@ -172,7 +173,7 @@ export class QwenApi implements LLMApi {
        const finish = () => {
          if (!finished) {
            finished = true;
-            options.onFinish(responseText + remainText);
+            options.onFinish(responseText + remainText, responseRes);
          }
        };

@@ -188,6 +189,7 @@ export class QwenApi implements LLMApi {
              "[Alibaba] request response content type: ",
              contentType,
            );
+            responseRes = res;

            if (contentType?.startsWith("text/plain")) {
              responseText = await res.clone().text();
@@ -254,7 +256,7 @@ export class QwenApi implements LLMApi {

        const resJson = await res.json();
        const message = this.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/client/platforms/anthropic.ts
+++ b/app/client/platforms/anthropic.ts
@@ -317,13 +317,14 @@ export class ClaudeApi implements LLMApi {
      };

      try {
-        controller.signal.onabort = () => options.onFinish("");
+        controller.signal.onabort = () =>
+          options.onFinish("", new Response(null, { status: 400 }));

        const res = await fetch(path, payload);
        const resJson = await res.json();

        const message = this.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      } catch (e) {
        console.error("failed to chat", e);
        options.onError?.(e as Error);
--- a/app/client/platforms/baidu.ts
+++ b/app/client/platforms/baidu.ts
@@ -162,6 +162,7 @@ export class ErnieApi implements LLMApi {
        let responseText = "";
        let remainText = "";
        let finished = false;
+        let responseRes: Response;

        // animate response to make it looks smooth
        function animateResponseText() {
@@ -191,7 +192,7 @@ export class ErnieApi implements LLMApi {
        const finish = () => {
          if (!finished) {
            finished = true;
-            options.onFinish(responseText + remainText);
+            options.onFinish(responseText + remainText, responseRes);
          }
        };

@@ -204,7 +205,7 @@ export class ErnieApi implements LLMApi {
            clearTimeout(requestTimeoutId);
            const contentType = res.headers.get("content-type");
            console.log("[Baidu] request response content type: ", contentType);
-
+            responseRes = res;
            if (contentType?.startsWith("text/plain")) {
              responseText = await res.clone().text();
              return finish();
@@ -267,7 +268,7 @@ export class ErnieApi implements LLMApi {

        const resJson = await res.json();
        const message = resJson?.result;
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/client/platforms/bytedance.ts
+++ b/app/client/platforms/bytedance.ts
@@ -130,6 +130,7 @@ export class DoubaoApi implements LLMApi {
        let responseText = "";
        let remainText = "";
        let finished = false;
+        let responseRes: Response;

        // animate response to make it looks smooth
        function animateResponseText() {
@@ -159,7 +160,7 @@ export class DoubaoApi implements LLMApi {
        const finish = () => {
          if (!finished) {
            finished = true;
-            options.onFinish(responseText + remainText);
+            options.onFinish(responseText + remainText, responseRes);
          }
        };

@@ -175,7 +176,7 @@ export class DoubaoApi implements LLMApi {
              "[ByteDance] request response content type: ",
              contentType,
            );
-
+            responseRes = res;
            if (contentType?.startsWith("text/plain")) {
              responseText = await res.clone().text();
              return finish();
@@ -241,7 +242,7 @@ export class DoubaoApi implements LLMApi {

        const resJson = await res.json();
        const message = this.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/client/platforms/glm.ts
+++ b/app/client/platforms/glm.ts
@@ -0,0 +1,197 @@
+"use client";
+import {
+  ApiPath,
+  CHATGLM_BASE_URL,
+  ChatGLM,
+  REQUEST_TIMEOUT_MS,
+} from "@/app/constant";
+import {
+  useAccessStore,
+  useAppConfig,
+  useChatStore,
+  ChatMessageTool,
+  usePluginStore,
+} from "@/app/store";
+import { stream } from "@/app/utils/chat";
+import {
+  ChatOptions,
+  getHeaders,
+  LLMApi,
+  LLMModel,
+  SpeechOptions,
+} from "../api";
+import { getClientConfig } from "@/app/config/client";
+import { getMessageTextContent } from "@/app/utils";
+import { RequestPayload } from "./openai";
+import { fetch } from "@/app/utils/stream";
+
+export class ChatGLMApi implements LLMApi {
+  private disableListModels = true;
+
+  path(path: string): string {
+    const accessStore = useAccessStore.getState();
+
+    let baseUrl = "";
+
+    if (accessStore.useCustomConfig) {
+      baseUrl = accessStore.chatglmUrl;
+    }
+
+    if (baseUrl.length === 0) {
+      const isApp = !!getClientConfig()?.isApp;
+      const apiPath = ApiPath.ChatGLM;
+      baseUrl = isApp ? CHATGLM_BASE_URL : apiPath;
+    }
+
+    if (baseUrl.endsWith("/")) {
+      baseUrl = baseUrl.slice(0, baseUrl.length - 1);
+    }
+    if (!baseUrl.startsWith("http") && !baseUrl.startsWith(ApiPath.ChatGLM)) {
+      baseUrl = "https://" + baseUrl;
+    }
+
+    console.log("[Proxy Endpoint] ", baseUrl, path);
+
+    return [baseUrl, path].join("/");
+  }
+
+  extractMessage(res: any) {
+    return res.choices?.at(0)?.message?.content ?? "";
+  }
+
+  speech(options: SpeechOptions): Promise<ArrayBuffer> {
+    throw new Error("Method not implemented.");
+  }
+
+  async chat(options: ChatOptions) {
+    const messages: ChatOptions["messages"] = [];
+    for (const v of options.messages) {
+      const content = getMessageTextContent(v);
+      messages.push({ role: v.role, content });
+    }
+
+    const modelConfig = {
+      ...useAppConfig.getState().modelConfig,
+      ...useChatStore.getState().currentSession().mask.modelConfig,
+      ...{
+        model: options.config.model,
+        providerName: options.config.providerName,
+      },
+    };
+
+    const requestPayload: RequestPayload = {
+      messages,
+      stream: options.config.stream,
+      model: modelConfig.model,
+      temperature: modelConfig.temperature,
+      presence_penalty: modelConfig.presence_penalty,
+      frequency_penalty: modelConfig.frequency_penalty,
+      top_p: modelConfig.top_p,
+    };
+
+    console.log("[Request] glm payload: ", requestPayload);
+
+    const shouldStream = !!options.config.stream;
+    const controller = new AbortController();
+    options.onController?.(controller);
+
+    try {
+      const chatPath = this.path(ChatGLM.ChatPath);
+      const chatPayload = {
+        method: "POST",
+        body: JSON.stringify(requestPayload),
+        signal: controller.signal,
+        headers: getHeaders(),
+      };
+
+      // make a fetch request
+      const requestTimeoutId = setTimeout(
+        () => controller.abort(),
+        REQUEST_TIMEOUT_MS,
+      );
+
+      if (shouldStream) {
+        const [tools, funcs] = usePluginStore
+          .getState()
+          .getAsTools(
+            useChatStore.getState().currentSession().mask?.plugin || [],
+          );
+        return stream(
+          chatPath,
+          requestPayload,
+          getHeaders(),
+          tools as any,
+          funcs,
+          controller,
+          // parseSSE
+          (text: string, runTools: ChatMessageTool[]) => {
+            // console.log("parseSSE", text, runTools);
+            const json = JSON.parse(text);
+            const choices = json.choices as Array<{
+              delta: {
+                content: string;
+                tool_calls: ChatMessageTool[];
+              };
+            }>;
+            const tool_calls = choices[0]?.delta?.tool_calls;
+            if (tool_calls?.length > 0) {
+              const index = tool_calls[0]?.index;
+              const id = tool_calls[0]?.id;
+              const args = tool_calls[0]?.function?.arguments;
+              if (id) {
+                runTools.push({
+                  id,
+                  type: tool_calls[0]?.type,
+                  function: {
+                    name: tool_calls[0]?.function?.name as string,
+                    arguments: args,
+                  },
+                });
+              } else {
+                // @ts-ignore
+                runTools[index]["function"]["arguments"] += args;
+              }
+            }
+            return choices[0]?.delta?.content;
+          },
+          // processToolMessage, include tool_calls message and tool call results
+          (
+            requestPayload: RequestPayload,
+            toolCallMessage: any,
+            toolCallResult: any[],
+          ) => {
+            // @ts-ignore
+            requestPayload?.messages?.splice(
+              // @ts-ignore
+              requestPayload?.messages?.length,
+              0,
+              toolCallMessage,
+              ...toolCallResult,
+            );
+          },
+          options,
+        );
+      } else {
+        const res = await fetch(chatPath, chatPayload);
+        clearTimeout(requestTimeoutId);
+
+        const resJson = await res.json();
+        const message = this.extractMessage(resJson);
+        options.onFinish(message, res);
+      }
+    } catch (e) {
+      console.log("[Request] failed to make a chat request", e);
+      options.onError?.(e as Error);
+    }
+  }
+  async usage() {
+    return {
+      used: 0,
+      total: 0,
+    };
+  }
+
+  async models(): Promise<LLMModel[]> {
+    return [];
+  }
+}
--- a/app/client/platforms/google.ts
+++ b/app/client/platforms/google.ts
@@ -192,7 +192,10 @@ export class GeminiProApi implements LLMApi {
          requestPayload,
          getHeaders(),
          // @ts-ignore
-          [{ functionDeclarations: tools.map((tool) => tool.function) }],
+          tools.length > 0
+            ? // @ts-ignore
+              [{ functionDeclarations: tools.map((tool) => tool.function) }]
+            : [],
          funcs,
          controller,
          // parseSSE
@@ -271,7 +274,7 @@ export class GeminiProApi implements LLMApi {
          );
        }
        const message = apiClient.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/client/platforms/iflytek.ts
+++ b/app/client/platforms/iflytek.ts
@@ -117,6 +117,7 @@ export class SparkApi implements LLMApi {
        let responseText = "";
        let remainText = "";
        let finished = false;
+        let responseRes: Response;

        // Animate response text to make it look smooth
        function animateResponseText() {
@@ -143,7 +144,7 @@ export class SparkApi implements LLMApi {
        const finish = () => {
          if (!finished) {
            finished = true;
-            options.onFinish(responseText + remainText);
+            options.onFinish(responseText + remainText, responseRes);
          }
        };

@@ -156,7 +157,7 @@ export class SparkApi implements LLMApi {
            clearTimeout(requestTimeoutId);
            const contentType = res.headers.get("content-type");
            console.log("[Spark] request response content type: ", contentType);
-
+            responseRes = res;
            if (contentType?.startsWith("text/plain")) {
              responseText = await res.clone().text();
              return finish();
@@ -231,7 +232,7 @@ export class SparkApi implements LLMApi {

        const resJson = await res.json();
        const message = this.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/client/platforms/moonshot.ts
+++ b/app/client/platforms/moonshot.ts
@@ -180,7 +180,7 @@ export class MoonshotApi implements LLMApi {

        const resJson = await res.json();
        const message = this.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/client/platforms/openai.ts
+++ b/app/client/platforms/openai.ts
@@ -65,6 +65,7 @@ export interface RequestPayload {
  frequency_penalty: number;
  top_p: number;
  max_tokens?: number;
+  max_completion_tokens?: number;
 }

 export interface DalleRequestPayload {
@@ -233,6 +234,11 @@ export class ChatGPTApi implements LLMApi {
        // Please do not ask me why not send max_tokens, no reason, this param is just shit, I dont want to explain anymore.
      };

+      // O1 使用 max_completion_tokens 控制token数 (https://platform.openai.com/docs/guides/reasoning#controlling-costs)
+      if (isO1) {
+        requestPayload["max_completion_tokens"] = modelConfig.max_tokens;
+      }
+
      // add max_tokens to vision model
      if (visionModel) {
        requestPayload["max_tokens"] = Math.max(modelConfig.max_tokens, 4000);
@@ -361,7 +367,7 @@ export class ChatGPTApi implements LLMApi {

        const resJson = await res.json();
        const message = await this.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/client/platforms/tencent.ts
+++ b/app/client/platforms/tencent.ts
@@ -142,6 +142,7 @@ export class HunyuanApi implements LLMApi {
        let responseText = "";
        let remainText = "";
        let finished = false;
+        let responseRes: Response;

        // animate response to make it looks smooth
        function animateResponseText() {
@@ -171,7 +172,7 @@ export class HunyuanApi implements LLMApi {
        const finish = () => {
          if (!finished) {
            finished = true;
-            options.onFinish(responseText + remainText);
+            options.onFinish(responseText + remainText, responseRes);
          }
        };

@@ -187,7 +188,7 @@ export class HunyuanApi implements LLMApi {
              "[Tencent] request response content type: ",
              contentType,
            );
-
+            responseRes = res;
            if (contentType?.startsWith("text/plain")) {
              responseText = await res.clone().text();
              return finish();
@@ -253,7 +254,7 @@ export class HunyuanApi implements LLMApi {

        const resJson = await res.json();
        const message = this.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/client/platforms/xai.ts
+++ b/app/client/platforms/xai.ts
@@ -173,7 +173,7 @@ export class XAIApi implements LLMApi {

        const resJson = await res.json();
        const message = this.extractMessage(resJson);
-        options.onFinish(message);
+        options.onFinish(message, res);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat request", e);
--- a/app/components/auth.tsx
+++ b/app/components/auth.tsx
@@ -18,6 +18,8 @@ import {
  trackSettingsPageGuideToCPaymentClick,
  trackAuthorizationPageButtonToCPaymentClick,
 } from "../utils/auth-settings-events";
+import clsx from "clsx";
+
 const storage = safeLocalStorage();

 export function AuthPage() {
@@ -54,7 +56,7 @@ export function AuthPage() {
          onClick={() => navigate(Path.Home)}
        ></IconButton>
      </div>
-      <div className={`no-dark ${styles["auth-logo"]}`}>
+      <div className={clsx("no-dark", styles["auth-logo"])}>
        <BotIcon />
      </div>

@@ -163,7 +165,7 @@ function TopBanner() {
      onMouseEnter={handleMouseEnter}
      onMouseLeave={handleMouseLeave}
    >
-      <div className={`${styles["top-banner-inner"]} no-dark`}>
+      <div className={clsx(styles["top-banner-inner"], "no-dark")}>
        <Logo className={styles["top-banner-logo"]}></Logo>
        <span>
          {Locale.Auth.TopTips}
--- a/app/components/button.tsx
+++ b/app/components/button.tsx
@@ -2,6 +2,7 @@ import * as React from "react";

 import styles from "./button.module.scss";
 import { CSSProperties } from "react";
+import clsx from "clsx";

 export type ButtonType = "primary" | "danger" | null;

@@ -22,12 +23,16 @@ export function IconButton(props: {
 }) {
  return (
    <button
-      className={
-        styles["icon-button"] +
-        ` ${props.bordered && styles.border} ${props.shadow && styles.shadow} ${
-          props.className ?? ""
-        } clickable ${styles[props.type ?? ""]}`
-      }
+      className={clsx(
+        "clickable",
+        styles["icon-button"],
+        {
+          [styles.border]: props.bordered,
+          [styles.shadow]: props.shadow,
+        },
+        styles[props.type ?? ""],
+        props.className,
+      )}
      onClick={props.onClick}
      title={props.title}
      disabled={props.disabled}
@@ -40,10 +45,9 @@ export function IconButton(props: {
      {props.icon && (
        <div
          aria-label={props.text || props.title}
-          className={
-            styles["icon-button-icon"] +
-            ` ${props.type === "primary" && "no-dark"}`
-          }
+          className={clsx(styles["icon-button-icon"], {
+            "no-dark": props.type === "primary",
+          })}
        >
          {props.icon}
        </div>
--- a/app/components/chat-list.tsx
+++ b/app/components/chat-list.tsx
@@ -18,6 +18,7 @@ import { Mask } from "../store/mask";
 import { useRef, useEffect } from "react";
 import { showConfirm } from "./ui-lib";
 import { useMobileScreen } from "../utils";
+import clsx from "clsx";

 export function ChatItem(props: {
  onClick?: () => void;
@@ -45,11 +46,11 @@ export function ChatItem(props: {
    <Draggable draggableId={`${props.id}`} index={props.index}>
      {(provided) => (
        <div
-          className={`${styles["chat-item"]} ${
-            props.selected &&
-            (currentPath === Path.Chat || currentPath === Path.Home) &&
-            styles["chat-item-selected"]
-          }`}
+          className={clsx(styles["chat-item"], {
+            [styles["chat-item-selected"]]:
+              props.selected &&
+              (currentPath === Path.Chat || currentPath === Path.Home),
+          })}
          onClick={props.onClick}
          ref={(ele) => {
            draggableRef.current = ele;
@@ -63,7 +64,7 @@ export function ChatItem(props: {
        >
          {props.narrow ? (
            <div className={styles["chat-item-narrow"]}>
-              <div className={styles["chat-item-avatar"] + " no-dark"}>
+              <div className={clsx(styles["chat-item-avatar"], "no-dark")}>
                <MaskAvatar
                  avatar={props.mask.avatar}
                  model={props.mask.modelConfig.model}
--- a/app/components/chat.module.scss
+++ b/app/components/chat.module.scss
@@ -45,6 +45,14 @@
 .chat-input-actions {
  display: flex;
  flex-wrap: wrap;
+  justify-content: space-between;
+  gap: 5px;
+
+  &-end {
+    display: flex;
+    margin-left: auto;
+    gap: 5px;
+  }

  .chat-input-action {
    display: inline-flex;
@@ -62,10 +70,6 @@
    width: var(--icon-width);
    overflow: hidden;

-    &:not(:last-child) {
-      margin-right: 5px;
-    }
-
    .text {
      white-space: nowrap;
      padding-left: 5px;
@@ -231,10 +235,12 @@

  animation: slide-in ease 0.3s;

-  $linear: linear-gradient(to right,
-      rgba(0, 0, 0, 0),
-      rgba(0, 0, 0, 1),
-      rgba(0, 0, 0, 0));
+  $linear: linear-gradient(
+    to right,
+    rgba(0, 0, 0, 0),
+    rgba(0, 0, 0, 1),
+    rgba(0, 0, 0, 0)
+  );
  mask-image: $linear;

  @mixin show {
@@ -373,7 +379,7 @@
  }
 }

-.chat-message-user>.chat-message-container {
+.chat-message-user > .chat-message-container {
  align-items: flex-end;
 }

@@ -443,6 +449,25 @@
  transition: all ease 0.3s;
 }

+.chat-message-audio {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  border-radius: 10px;
+  background-color: rgba(0, 0, 0, 0.05);
+  border: var(--border-in-light);
+  position: relative;
+  transition: all ease 0.3s;
+  margin-top: 10px;
+  font-size: 14px;
+  user-select: text;
+  word-break: break-word;
+  box-sizing: border-box;
+  audio {
+    height: 30px; /* 调整高度 */
+  }
+}
+
 .chat-message-item-image {
  width: 100%;
  margin-top: 10px;
@@ -471,23 +496,27 @@
  border: rgba($color: #888, $alpha: 0.2) 1px solid;
 }

-
@media only screen and (max-width: 600px) {
-  $calc-image-width: calc(100vw/3*2/var(--image-count));
+  $calc-image-width: calc(100vw / 3 * 2 / var(--image-count));

  .chat-message-item-image-multi {
    width: $calc-image-width;
    height: $calc-image-width;
  }
-  
+
  .chat-message-item-image {
-    max-width: calc(100vw/3*2);
+    max-width: calc(100vw / 3 * 2);
  }
 }

@media screen and (min-width: 600px) {
-  $max-image-width: calc(calc(1200px - var(--sidebar-width))/3*2/var(--image-count));
-  $image-width: calc(calc(var(--window-width) - var(--sidebar-width))/3*2/var(--image-count));
+  $max-image-width: calc(
+    calc(1200px - var(--sidebar-width)) / 3 * 2 / var(--image-count)
+  );
+  $image-width: calc(
+    calc(var(--window-width) - var(--sidebar-width)) / 3 * 2 /
+      var(--image-count)
+  );

  .chat-message-item-image-multi {
    width: $image-width;
@@ -497,7 +526,7 @@
  }

  .chat-message-item-image {
-    max-width: calc(calc(1200px - var(--sidebar-width))/3*2);
+    max-width: calc(calc(1200px - var(--sidebar-width)) / 3 * 2);
  }
 }

@@ -515,7 +544,7 @@
  z-index: 1;
 }

-.chat-message-user>.chat-message-container>.chat-message-item {
+.chat-message-user > .chat-message-container > .chat-message-item {
  background-color: var(--second);

  &:hover {
@@ -626,7 +655,8 @@
  min-height: 68px;
 }

-.chat-input:focus {}
+.chat-input:focus {
+}

 .chat-input-send {
  background-color: var(--primary);
@@ -693,4 +723,31 @@
 .shortcut-key span {
  font-size: 12px;
  color: var(--black);
-}
+}
+
+.chat-main {
+  display: flex;
+  height: 100%;
+  width: 100%;
+  position: relative;
+  overflow: hidden;
+  .chat-body-container {
+    height: 100%;
+    display: flex;
+    flex-direction: column;
+    flex: 1;
+    width: 100%;
+  }
+  .chat-side-panel {
+    position: absolute;
+    inset: 0;
+    background: var(--white);
+    overflow: hidden;
+    z-index: 10;
+    transform: translateX(100%);
+    transition: all ease 0.3s;
+    &-show {
+      transform: translateX(0);
+    }
+  }
+}
--- a/app/components/chat.tsx
+++ b/app/components/chat.tsx
--- a/app/components/exporter.tsx
+++ b/app/components/exporter.tsx
@@ -40,6 +40,7 @@ import { EXPORT_MESSAGE_CLASS_NAME } from "../constant";
 import { getClientConfig } from "../config/client";
 import { type ClientApi, getClientApi } from "../client/api";
 import { getMessageTextContent } from "../utils";
+import clsx from "clsx";

 const Markdown = dynamic(async () => (await import("./markdown")).Markdown, {
  loading: () => <LoadingIcon />,
@@ -118,9 +119,10 @@ function Steps<
          return (
            <div
              key={i}
-              className={`${styles["step"]} ${
-                styles[i <= props.index ? "step-finished" : ""]
-              } ${i === props.index && styles["step-current"]} clickable`}
+              className={clsx("clickable", styles["step"], {
+                [styles["step-finished"]]: i <= props.index,
+                [styles["step-current"]]: i === props.index,
+              })}
              onClick={() => {
                props.onStepChange?.(i);
              }}
@@ -525,11 +527,11 @@ export function ImagePreviewer(props: {
        messages={props.messages}
      />
      <div
-        className={`${styles["preview-body"]} ${styles["default-theme"]}`}
+        className={clsx(styles["preview-body"], styles["default-theme"])}
        ref={previewRef}
      >
        <div className={styles["chat-info"]}>
-          <div className={styles["logo"] + " no-dark"}>
+          <div className={clsx(styles["logo"], "no-dark")}>
            <NextImage
              src={ChatGptIcon.src}
              alt="logo"
@@ -570,7 +572,7 @@ export function ImagePreviewer(props: {
        {props.messages.map((m, i) => {
          return (
            <div
-              className={styles["message"] + " " + styles["message-" + m.role]}
+              className={clsx(styles["message"], styles["message-" + m.role])}
              key={i}
            >
              <div className={styles["avatar"]}>
--- a/app/components/home.tsx
+++ b/app/components/home.tsx
@@ -3,7 +3,6 @@
 require("../polyfill");

 import { useState, useEffect } from "react";
-
 import styles from "./home.module.scss";

 import BotIcon from "../icons/bot.svg";
@@ -29,10 +28,11 @@ import { AuthPage } from "./auth";
 import { getClientConfig } from "../config/client";
 import { type ClientApi, getClientApi } from "../client/api";
 import { useAccessStore } from "../store";
+import clsx from "clsx";

 export function Loading(props: { noLogo?: boolean }) {
  return (
-    <div className={styles["loading-content"] + " no-dark"}>
+    <div className={clsx("no-dark", styles["loading-content"])}>
      {!props.noLogo && <BotIcon />}
      <LoadingIcon />
    </div>
@@ -179,7 +179,11 @@ function Screen() {
    if (isSdNew) return <Sd />;
    return (
      <>
-        <SideBar className={isHome ? styles["sidebar-show"] : ""} />
+        <SideBar
+          className={clsx({
+            [styles["sidebar-show"]]: isHome,
+          })}
+        />
        <WindowContent>
          <Routes>
            <Route path={Path.Home} element={<Chat />} />
@@ -197,9 +201,10 @@ function Screen() {

  return (
    <div
-      className={`${styles.container} ${
-        shouldTightBorder ? styles["tight-container"] : styles.container
-      } ${getLang() === "ar" ? styles["rtl-screen"] : ""}`}
+      className={clsx(styles.container, {
+        [styles["tight-container"]]: shouldTightBorder,
+        [styles["rtl-screen"]]: getLang() === "ar",
+      })}
    >
      {renderContent()}
    </div>
--- a/app/components/input-range.tsx
+++ b/app/components/input-range.tsx
@@ -1,5 +1,6 @@
 import * as React from "react";
 import styles from "./input-range.module.scss";
+import clsx from "clsx";

 interface InputRangeProps {
  onChange: React.ChangeEventHandler<HTMLInputElement>;
@@ -23,7 +24,7 @@ export function InputRange({
  aria,
 }: InputRangeProps) {
  return (
-    <div className={styles["input-range"] + ` ${className ?? ""}`}>
+    <div className={clsx(styles["input-range"], className)}>
      {title || value}
      <input
        aria-label={aria}
--- a/app/components/markdown.tsx
+++ b/app/components/markdown.tsx
@@ -23,6 +23,7 @@ import { useChatStore } from "../store";
 import { IconButton } from "./button";

 import { useAppConfig } from "../store/config";
+import clsx from "clsx";

 export function Mermaid(props: { code: string }) {
  const ref = useRef<HTMLDivElement>(null);
@@ -57,7 +58,7 @@ export function Mermaid(props: { code: string }) {

  return (
    <div
-      className="no-dark mermaid"
+      className={clsx("no-dark", "mermaid")}
      style={{
        cursor: "pointer",
        overflow: "auto",
@@ -193,7 +194,12 @@ function CustomCode(props: { children: any; className?: string }) {
  const renderShowMoreButton = () => {
    if (showToggle && enableCodeFold && collapsed) {
      return (
-        <div className={`show-hide-button ${collapsed ? "collapsed" : "expanded"}`}>
+        <div
+          className={clsx("show-hide-button", {
+            collapsed,
+            expanded: !collapsed,
+          })}
+        >
          <button onClick={toggleCollapsed}>{Locale.NewChat.More}</button>
        </div>
      );
@@ -203,7 +209,7 @@ function CustomCode(props: { children: any; className?: string }) {
  return (
    <>
      <code
-        className={props?.className}
+        className={clsx(props?.className)}
        ref={ref}
        style={{
          maxHeight: enableCodeFold && collapsed ? "400px" : "none",
--- a/app/components/mask.tsx
+++ b/app/components/mask.tsx
@@ -55,6 +55,7 @@ import {
  OnDragEndResponder,
 } from "@hello-pangea/dnd";
 import { getMessageTextContent } from "../utils";
+import clsx from "clsx";

 // drag and drop helper function
 function reorder<T>(list: T[], startIndex: number, endIndex: number): T[] {
@@ -588,7 +589,7 @@ export function MaskPage() {
                  </div>
                  <div className={styles["mask-title"]}>
                    <div className={styles["mask-name"]}>{m.name}</div>
-                    <div className={styles["mask-info"] + " one-line"}>
+                    <div className={clsx(styles["mask-info"], "one-line")}>
                      {`${Locale.Mask.Item.Info(m.context.length)} / ${
                        ALL_LANG_OPTIONS[m.lang]
                      } / ${m.modelConfig.model}`}
--- a/app/components/message-selector.tsx
+++ b/app/components/message-selector.tsx
@@ -8,6 +8,7 @@ import Locale from "../locales";

 import styles from "./message-selector.module.scss";
 import { getMessageTextContent } from "../utils";
+import clsx from "clsx";

 function useShiftRange() {
  const [startIndex, setStartIndex] = useState<number>();
@@ -71,6 +72,7 @@ export function MessageSelector(props: {
  defaultSelectAll?: boolean;
  onSelected?: (messages: ChatMessage[]) => void;
 }) {
+  const LATEST_COUNT = 4;
  const chatStore = useChatStore();
  const session = chatStore.currentSession();
  const isValid = (m: ChatMessage) => m.content && !m.isError && !m.streaming;
@@ -141,15 +143,13 @@ export function MessageSelector(props: {
    // eslint-disable-next-line react-hooks/exhaustive-deps
  }, [startIndex, endIndex]);

-  const LATEST_COUNT = 4;
-
  return (
    <div className={styles["message-selector"]}>
      <div className={styles["message-filter"]}>
        <input
          type="text"
          placeholder={Locale.Select.Search}
-          className={styles["filter-item"] + " " + styles["search-bar"]}
+          className={clsx(styles["filter-item"], styles["search-bar"])}
          value={searchInput}
          onInput={(e) => {
            setSearchInput(e.currentTarget.value);
@@ -196,9 +196,9 @@ export function MessageSelector(props: {

          return (
            <div
-              className={`${styles["message"]} ${
-                props.selection.has(m.id!) && styles["message-selected"]
-              }`}
+              className={clsx(styles["message"], {
+                [styles["message-selected"]]: props.selection.has(m.id!),
+              })}
              key={i}
              onClick={() => {
                props.updateSelection((selection) => {
@@ -221,7 +221,7 @@ export function MessageSelector(props: {
                <div className={styles["date"]}>
                  {new Date(m.date).toLocaleString()}
                </div>
-                <div className={`${styles["content"]} one-line`}>
+                <div className={clsx(styles["content"], "one-line")}>
                  {getMessageTextContent(m)}
                </div>
              </div>
--- a/app/components/model-config.tsx
+++ b/app/components/model-config.tsx
@@ -7,6 +7,7 @@ import { ListItem, Select } from "./ui-lib";
 import { useAllModels } from "../utils/hooks";
 import { groupBy } from "lodash-es";
 import styles from "./model-config.module.scss";
+import { getModelProvider } from "../utils/model";

 export function ModelConfigList(props: {
  modelConfig: ModelConfig;
@@ -28,7 +29,9 @@ export function ModelConfigList(props: {
          value={value}
          align="left"
          onChange={(e) => {
-            const [model, providerName] = e.currentTarget.value.split("@");
+            const [model, providerName] = getModelProvider(
+              e.currentTarget.value,
+            );
            props.updateConfig((config) => {
              config.model = ModalConfigValidator.model(model);
              config.providerName = providerName as ServiceProvider;
@@ -247,7 +250,9 @@ export function ModelConfigList(props: {
          aria-label={Locale.Settings.CompressModel.Title}
          value={compressModelValue}
          onChange={(e) => {
-            const [model, providerName] = e.currentTarget.value.split("@");
+            const [model, providerName] = getModelProvider(
+              e.currentTarget.value,
+            );
            props.updateConfig((config) => {
              config.compressModel = ModalConfigValidator.model(model);
              config.compressProviderName = providerName as ServiceProvider;
--- a/app/components/new-chat.tsx
+++ b/app/components/new-chat.tsx
@@ -16,6 +16,7 @@ import { MaskAvatar } from "./mask";
 import { useCommand } from "../command";
 import { showConfirm } from "./ui-lib";
 import { BUILTIN_MASK_STORE } from "../masks";
+import clsx from "clsx";

 function MaskItem(props: { mask: Mask; onClick?: () => void }) {
  return (
@@ -24,7 +25,9 @@ function MaskItem(props: { mask: Mask; onClick?: () => void }) {
        avatar={props.mask.avatar}
        model={props.mask.modelConfig.model}
      />
-      <div className={styles["mask-name"] + " one-line"}>{props.mask.name}</div>
+      <div className={clsx(styles["mask-name"], "one-line")}>
+        {props.mask.name}
+      </div>
    </div>
  );
 }
--- a/app/components/plugin.tsx
+++ b/app/components/plugin.tsx
@@ -28,6 +28,7 @@ import {
 import Locale from "../locales";
 import { useNavigate } from "react-router-dom";
 import { useState } from "react";
+import clsx from "clsx";

 export function PluginPage() {
  const navigate = useNavigate();
@@ -199,7 +200,7 @@ export function PluginPage() {
                    <div className={styles["mask-name"]}>
                      {m.title}@<small>{m.version}</small>
                    </div>
-                    <div className={styles["mask-info"] + " one-line"}>
+                    <div className={clsx(styles["mask-info"], "one-line")}>
                      {Locale.Plugin.Item.Info(
                        FunctionToolService.add(m).length,
                      )}
@@ -335,7 +336,10 @@ export function PluginPage() {
              <ListItem
                subTitle={
                  <div
-                    className={`markdown-body ${pluginStyles["plugin-content"]}`}
+                    className={clsx(
+                      "markdown-body",
+                      pluginStyles["plugin-content"],
+                    )}
                    dir="auto"
                  >
                    <pre>
--- a/app/components/realtime-chat/index.ts
+++ b/app/components/realtime-chat/index.ts
@@ -0,0 +1 @@
+export * from "./realtime-chat";
--- a/app/components/realtime-chat/realtime-chat.module.scss
+++ b/app/components/realtime-chat/realtime-chat.module.scss
@@ -0,0 +1,74 @@
+.realtime-chat {
+  width: 100%;
+  justify-content: center;
+  align-items: center;
+  position: relative;
+  display: flex;
+  flex-direction: column;
+  height: 100%;
+  padding: 20px;
+  box-sizing: border-box;
+  .circle-mic {
+    width: 150px;
+    height: 150px;
+    border-radius: 50%;
+    background: linear-gradient(to bottom right, #a0d8ef, #f0f8ff);
+    display: flex;
+    justify-content: center;
+    align-items: center;
+  }
+  .icon-center {
+    font-size: 24px;
+  }
+
+  .bottom-icons {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    width: 100%;
+    position: absolute;
+    bottom: 20px;
+    box-sizing: border-box;
+    padding: 0 20px;
+  }
+
+  .icon-left,
+  .icon-right {
+    width: 46px;
+    height: 46px;
+    font-size: 36px;
+    background: var(--second);
+    border-radius: 50%;
+    padding: 2px;
+    display: flex;
+    justify-content: center;
+    align-items: center;
+    cursor: pointer;
+    &:hover {
+      opacity: 0.8;
+    }
+  }
+
+  &.mobile {
+    display: none;
+  }
+}
+
+.pulse {
+  animation: pulse 1.5s infinite;
+}
+
+@keyframes pulse {
+  0% {
+    transform: scale(1);
+    opacity: 0.7;
+  }
+  50% {
+    transform: scale(1.1);
+    opacity: 1;
+  }
+  100% {
+    transform: scale(1);
+    opacity: 0.7;
+  }
+}
--- a/app/components/realtime-chat/realtime-chat.tsx
+++ b/app/components/realtime-chat/realtime-chat.tsx
@@ -0,0 +1,359 @@
+import VoiceIcon from "@/app/icons/voice.svg";
+import VoiceOffIcon from "@/app/icons/voice-off.svg";
+import PowerIcon from "@/app/icons/power.svg";
+
+import styles from "./realtime-chat.module.scss";
+import clsx from "clsx";
+
+import { useState, useRef, useEffect } from "react";
+
+import { useChatStore, createMessage, useAppConfig } from "@/app/store";
+
+import { IconButton } from "@/app/components/button";
+
+import {
+  Modality,
+  RTClient,
+  RTInputAudioItem,
+  RTResponse,
+  TurnDetection,
+} from "rt-client";
+import { AudioHandler } from "@/app/lib/audio";
+import { uploadImage } from "@/app/utils/chat";
+import { VoicePrint } from "@/app/components/voice-print";
+
+interface RealtimeChatProps {
+  onClose?: () => void;
+  onStartVoice?: () => void;
+  onPausedVoice?: () => void;
+}
+
+export function RealtimeChat({
+  onClose,
+  onStartVoice,
+  onPausedVoice,
+}: RealtimeChatProps) {
+  const chatStore = useChatStore();
+  const session = chatStore.currentSession();
+  const config = useAppConfig();
+  const [status, setStatus] = useState("");
+  const [isRecording, setIsRecording] = useState(false);
+  const [isConnected, setIsConnected] = useState(false);
+  const [isConnecting, setIsConnecting] = useState(false);
+  const [modality, setModality] = useState("audio");
+  const [useVAD, setUseVAD] = useState(true);
+  const [frequencies, setFrequencies] = useState<Uint8Array | undefined>();
+
+  const clientRef = useRef<RTClient | null>(null);
+  const audioHandlerRef = useRef<AudioHandler | null>(null);
+  const initRef = useRef(false);
+
+  const temperature = config.realtimeConfig.temperature;
+  const apiKey = config.realtimeConfig.apiKey;
+  const model = config.realtimeConfig.model;
+  const azure = config.realtimeConfig.provider === "Azure";
+  const azureEndpoint = config.realtimeConfig.azure.endpoint;
+  const azureDeployment = config.realtimeConfig.azure.deployment;
+  const voice = config.realtimeConfig.voice;
+
+  const handleConnect = async () => {
+    if (isConnecting) return;
+    if (!isConnected) {
+      try {
+        setIsConnecting(true);
+        clientRef.current = azure
+          ? new RTClient(
+              new URL(azureEndpoint),
+              { key: apiKey },
+              { deployment: azureDeployment },
+            )
+          : new RTClient({ key: apiKey }, { model });
+        const modalities: Modality[] =
+          modality === "audio" ? ["text", "audio"] : ["text"];
+        const turnDetection: TurnDetection = useVAD
+          ? { type: "server_vad" }
+          : null;
+        await clientRef.current.configure({
+          instructions: "",
+          voice,
+          input_audio_transcription: { model: "whisper-1" },
+          turn_detection: turnDetection,
+          tools: [],
+          temperature,
+          modalities,
+        });
+        startResponseListener();
+
+        setIsConnected(true);
+        // TODO
+        // try {
+        //   const recentMessages = chatStore.getMessagesWithMemory();
+        //   for (const message of recentMessages) {
+        //     const { role, content } = message;
+        //     if (typeof content === "string") {
+        //       await clientRef.current.sendItem({
+        //         type: "message",
+        //         role: role as any,
+        //         content: [
+        //           {
+        //             type: (role === "assistant" ? "text" : "input_text") as any,
+        //             text: content as string,
+        //           },
+        //         ],
+        //       });
+        //     }
+        //   }
+        //   // await clientRef.current.generateResponse();
+        // } catch (error) {
+        //   console.error("Set message failed:", error);
+        // }
+      } catch (error) {
+        console.error("Connection failed:", error);
+        setStatus("Connection failed");
+      } finally {
+        setIsConnecting(false);
+      }
+    } else {
+      await disconnect();
+    }
+  };
+
+  const disconnect = async () => {
+    if (clientRef.current) {
+      try {
+        await clientRef.current.close();
+        clientRef.current = null;
+        setIsConnected(false);
+      } catch (error) {
+        console.error("Disconnect failed:", error);
+      }
+    }
+  };
+
+  const startResponseListener = async () => {
+    if (!clientRef.current) return;
+
+    try {
+      for await (const serverEvent of clientRef.current.events()) {
+        if (serverEvent.type === "response") {
+          await handleResponse(serverEvent);
+        } else if (serverEvent.type === "input_audio") {
+          await handleInputAudio(serverEvent);
+        }
+      }
+    } catch (error) {
+      if (clientRef.current) {
+        console.error("Response iteration error:", error);
+      }
+    }
+  };
+
+  const handleResponse = async (response: RTResponse) => {
+    for await (const item of response) {
+      if (item.type === "message" && item.role === "assistant") {
+        const botMessage = createMessage({
+          role: item.role,
+          content: "",
+        });
+        // add bot message first
+        chatStore.updateTargetSession(session, (session) => {
+          session.messages = session.messages.concat([botMessage]);
+        });
+        let hasAudio = false;
+        for await (const content of item) {
+          if (content.type === "text") {
+            for await (const text of content.textChunks()) {
+              botMessage.content += text;
+            }
+          } else if (content.type === "audio") {
+            const textTask = async () => {
+              for await (const text of content.transcriptChunks()) {
+                botMessage.content += text;
+              }
+            };
+            const audioTask = async () => {
+              audioHandlerRef.current?.startStreamingPlayback();
+              for await (const audio of content.audioChunks()) {
+                hasAudio = true;
+                audioHandlerRef.current?.playChunk(audio);
+              }
+            };
+            await Promise.all([textTask(), audioTask()]);
+          }
+          // update message.content
+          chatStore.updateTargetSession(session, (session) => {
+            session.messages = session.messages.concat();
+          });
+        }
+        if (hasAudio) {
+          // upload audio get audio_url
+          const blob = audioHandlerRef.current?.savePlayFile();
+          uploadImage(blob!).then((audio_url) => {
+            botMessage.audio_url = audio_url;
+            // update text and audio_url
+            chatStore.updateTargetSession(session, (session) => {
+              session.messages = session.messages.concat();
+            });
+          });
+        }
+      }
+    }
+  };
+
+  const handleInputAudio = async (item: RTInputAudioItem) => {
+    await item.waitForCompletion();
+    if (item.transcription) {
+      const userMessage = createMessage({
+        role: "user",
+        content: item.transcription,
+      });
+      chatStore.updateTargetSession(session, (session) => {
+        session.messages = session.messages.concat([userMessage]);
+      });
+      // save input audio_url, and update session
+      const { audioStartMillis, audioEndMillis } = item;
+      // upload audio get audio_url
+      const blob = audioHandlerRef.current?.saveRecordFile(
+        audioStartMillis,
+        audioEndMillis,
+      );
+      uploadImage(blob!).then((audio_url) => {
+        userMessage.audio_url = audio_url;
+        chatStore.updateTargetSession(session, (session) => {
+          session.messages = session.messages.concat();
+        });
+      });
+    }
+    // stop streaming play after get input audio.
+    audioHandlerRef.current?.stopStreamingPlayback();
+  };
+
+  const toggleRecording = async () => {
+    if (!isRecording && clientRef.current) {
+      try {
+        if (!audioHandlerRef.current) {
+          audioHandlerRef.current = new AudioHandler();
+          await audioHandlerRef.current.initialize();
+        }
+        await audioHandlerRef.current.startRecording(async (chunk) => {
+          await clientRef.current?.sendAudio(chunk);
+        });
+        setIsRecording(true);
+      } catch (error) {
+        console.error("Failed to start recording:", error);
+      }
+    } else if (audioHandlerRef.current) {
+      try {
+        audioHandlerRef.current.stopRecording();
+        if (!useVAD) {
+          const inputAudio = await clientRef.current?.commitAudio();
+          await handleInputAudio(inputAudio!);
+          await clientRef.current?.generateResponse();
+        }
+        setIsRecording(false);
+      } catch (error) {
+        console.error("Failed to stop recording:", error);
+      }
+    }
+  };
+
+  useEffect(() => {
+    // 防止重复初始化
+    if (initRef.current) return;
+    initRef.current = true;
+
+    const initAudioHandler = async () => {
+      const handler = new AudioHandler();
+      await handler.initialize();
+      audioHandlerRef.current = handler;
+      await handleConnect();
+      await toggleRecording();
+    };
+
+    initAudioHandler().catch((error) => {
+      setStatus(error);
+      console.error(error);
+    });
+
+    return () => {
+      if (isRecording) {
+        toggleRecording();
+      }
+      audioHandlerRef.current?.close().catch(console.error);
+      disconnect();
+    };
+  }, []);
+
+  useEffect(() => {
+    let animationFrameId: number;
+
+    if (isConnected && isRecording) {
+      const animationFrame = () => {
+        if (audioHandlerRef.current) {
+          const freqData = audioHandlerRef.current.getByteFrequencyData();
+          setFrequencies(freqData);
+        }
+        animationFrameId = requestAnimationFrame(animationFrame);
+      };
+
+      animationFrameId = requestAnimationFrame(animationFrame);
+    } else {
+      setFrequencies(undefined);
+    }
+
+    return () => {
+      if (animationFrameId) {
+        cancelAnimationFrame(animationFrameId);
+      }
+    };
+  }, [isConnected, isRecording]);
+
+  // update session params
+  useEffect(() => {
+    clientRef.current?.configure({ voice });
+  }, [voice]);
+  useEffect(() => {
+    clientRef.current?.configure({ temperature });
+  }, [temperature]);
+
+  const handleClose = async () => {
+    onClose?.();
+    if (isRecording) {
+      await toggleRecording();
+    }
+    disconnect().catch(console.error);
+  };
+
+  return (
+    <div className={styles["realtime-chat"]}>
+      <div
+        className={clsx(styles["circle-mic"], {
+          [styles["pulse"]]: isRecording,
+        })}
+      >
+        <VoicePrint frequencies={frequencies} isActive={isRecording} />
+      </div>
+
+      <div className={styles["bottom-icons"]}>
+        <div>
+          <IconButton
+            icon={isRecording ? <VoiceIcon /> : <VoiceOffIcon />}
+            onClick={toggleRecording}
+            disabled={!isConnected}
+            shadow
+            bordered
+          />
+        </div>
+        <div className={styles["icon-center"]}>{status}</div>
+        <div>
+          <IconButton
+            icon={<PowerIcon />}
+            onClick={handleClose}
+            shadow
+            bordered
+          />
+        </div>
+      </div>
+    </div>
+  );
+}
--- a/app/components/realtime-chat/realtime-config.tsx
+++ b/app/components/realtime-chat/realtime-config.tsx
@@ -0,0 +1,173 @@
+import { RealtimeConfig } from "@/app/store";
+
+import Locale from "@/app/locales";
+import { ListItem, Select, PasswordInput } from "@/app/components/ui-lib";
+
+import { InputRange } from "@/app/components/input-range";
+import { Voice } from "rt-client";
+import { ServiceProvider } from "@/app/constant";
+
+const providers = [ServiceProvider.OpenAI, ServiceProvider.Azure];
+
+const models = ["gpt-4o-realtime-preview-2024-10-01"];
+
+const voice = ["alloy", "shimmer", "echo"];
+
+export function RealtimeConfigList(props: {
+  realtimeConfig: RealtimeConfig;
+  updateConfig: (updater: (config: RealtimeConfig) => void) => void;
+}) {
+  const azureConfigComponent = props.realtimeConfig.provider ===
+    ServiceProvider.Azure && (
+    <>
+      <ListItem
+        title={Locale.Settings.Realtime.Azure.Endpoint.Title}
+        subTitle={Locale.Settings.Realtime.Azure.Endpoint.SubTitle}
+      >
+        <input
+          value={props.realtimeConfig?.azure?.endpoint}
+          type="text"
+          placeholder={Locale.Settings.Realtime.Azure.Endpoint.Title}
+          onChange={(e) => {
+            props.updateConfig(
+              (config) => (config.azure.endpoint = e.currentTarget.value),
+            );
+          }}
+        />
+      </ListItem>
+      <ListItem
+        title={Locale.Settings.Realtime.Azure.Deployment.Title}
+        subTitle={Locale.Settings.Realtime.Azure.Deployment.SubTitle}
+      >
+        <input
+          value={props.realtimeConfig?.azure?.deployment}
+          type="text"
+          placeholder={Locale.Settings.Realtime.Azure.Deployment.Title}
+          onChange={(e) => {
+            props.updateConfig(
+              (config) => (config.azure.deployment = e.currentTarget.value),
+            );
+          }}
+        />
+      </ListItem>
+    </>
+  );
+
+  return (
+    <>
+      <ListItem
+        title={Locale.Settings.Realtime.Enable.Title}
+        subTitle={Locale.Settings.Realtime.Enable.SubTitle}
+      >
+        <input
+          type="checkbox"
+          checked={props.realtimeConfig.enable}
+          onChange={(e) =>
+            props.updateConfig(
+              (config) => (config.enable = e.currentTarget.checked),
+            )
+          }
+        ></input>
+      </ListItem>
+
+      {props.realtimeConfig.enable && (
+        <>
+          <ListItem
+            title={Locale.Settings.Realtime.Provider.Title}
+            subTitle={Locale.Settings.Realtime.Provider.SubTitle}
+          >
+            <Select
+              aria-label={Locale.Settings.Realtime.Provider.Title}
+              value={props.realtimeConfig.provider}
+              onChange={(e) => {
+                props.updateConfig(
+                  (config) =>
+                    (config.provider = e.target.value as ServiceProvider),
+                );
+              }}
+            >
+              {providers.map((v, i) => (
+                <option value={v} key={i}>
+                  {v}
+                </option>
+              ))}
+            </Select>
+          </ListItem>
+          <ListItem
+            title={Locale.Settings.Realtime.Model.Title}
+            subTitle={Locale.Settings.Realtime.Model.SubTitle}
+          >
+            <Select
+              aria-label={Locale.Settings.Realtime.Model.Title}
+              value={props.realtimeConfig.model}
+              onChange={(e) => {
+                props.updateConfig((config) => (config.model = e.target.value));
+              }}
+            >
+              {models.map((v, i) => (
+                <option value={v} key={i}>
+                  {v}
+                </option>
+              ))}
+            </Select>
+          </ListItem>
+          <ListItem
+            title={Locale.Settings.Realtime.ApiKey.Title}
+            subTitle={Locale.Settings.Realtime.ApiKey.SubTitle}
+          >
+            <PasswordInput
+              aria={Locale.Settings.ShowPassword}
+              aria-label={Locale.Settings.Realtime.ApiKey.Title}
+              value={props.realtimeConfig.apiKey}
+              type="text"
+              placeholder={Locale.Settings.Realtime.ApiKey.Placeholder}
+              onChange={(e) => {
+                props.updateConfig(
+                  (config) => (config.apiKey = e.currentTarget.value),
+                );
+              }}
+            />
+          </ListItem>
+          {azureConfigComponent}
+          <ListItem
+            title={Locale.Settings.TTS.Voice.Title}
+            subTitle={Locale.Settings.TTS.Voice.SubTitle}
+          >
+            <Select
+              value={props.realtimeConfig.voice}
+              onChange={(e) => {
+                props.updateConfig(
+                  (config) => (config.voice = e.currentTarget.value as Voice),
+                );
+              }}
+            >
+              {voice.map((v, i) => (
+                <option value={v} key={i}>
+                  {v}
+                </option>
+              ))}
+            </Select>
+          </ListItem>
+          <ListItem
+            title={Locale.Settings.Realtime.Temperature.Title}
+            subTitle={Locale.Settings.Realtime.Temperature.SubTitle}
+          >
+            <InputRange
+              aria={Locale.Settings.Temperature.Title}
+              value={props.realtimeConfig?.temperature?.toFixed(1)}
+              min="0.6"
+              max="1"
+              step="0.1"
+              onChange={(e) => {
+                props.updateConfig(
+                  (config) =>
+                    (config.temperature = e.currentTarget.valueAsNumber),
+                );
+              }}
+            ></InputRange>
+          </ListItem>
+        </>
+      )}
+    </>
+  );
+}
--- a/app/components/sd/sd-panel.tsx
+++ b/app/components/sd/sd-panel.tsx
@@ -4,6 +4,7 @@ import { Select } from "@/app/components/ui-lib";
 import { IconButton } from "@/app/components/button";
 import Locale from "@/app/locales";
 import { useSdStore } from "@/app/store/sd";
+import clsx from "clsx";

 export const params = [
  {
@@ -136,7 +137,7 @@ export function ControlParamItem(props: {
  className?: string;
 }) {
  return (
-    <div className={styles["ctrl-param-item"] + ` ${props.className || ""}`}>
+    <div className={clsx(styles["ctrl-param-item"], props.className)}>
      <div className={styles["ctrl-param-item-header"]}>
        <div className={styles["ctrl-param-item-title"]}>
          <div>
--- a/app/components/sd/sd.tsx
+++ b/app/components/sd/sd.tsx
@@ -36,6 +36,7 @@ import { removeImage } from "@/app/utils/chat";
 import { SideBar } from "./sd-sidebar";
 import { WindowContent } from "@/app/components/home";
 import { params } from "./sd-panel";
+import clsx from "clsx";

 function getSdTaskStatus(item: any) {
  let s: string;
@@ -104,7 +105,7 @@ export function Sd() {

  return (
    <>
-      <SideBar className={isSd ? homeStyles["sidebar-show"] : ""} />
+      <SideBar className={clsx({ [homeStyles["sidebar-show"]]: isSd })} />
      <WindowContent>
        <div className={chatStyles.chat} key={"1"}>
          <div className="window-header" data-tauri-drag-region>
@@ -121,7 +122,10 @@ export function Sd() {
              </div>
            )}
            <div
-              className={`window-header-title ${chatStyles["chat-body-title"]}`}
+              className={clsx(
+                "window-header-title",
+                chatStyles["chat-body-title"],
+              )}
            >
              <div className={`window-header-main-title`}>Stability AI</div>
              <div className="window-header-sub-title">
--- a/app/components/settings.tsx
+++ b/app/components/settings.tsx
@@ -72,6 +72,7 @@ import {
  Stability,
  Iflytek,
  SAAS_CHAT_URL,
+  ChatGLM,
 } from "../constant";
 import { Prompt, SearchService, usePromptStore } from "../store/prompt";
 import { ErrorBoundary } from "./error";
@@ -84,6 +85,7 @@ import { nanoid } from "nanoid";
 import { useMaskStore } from "../store/mask";
 import { ProviderType } from "../utils/cloud";
 import { TTSConfigList } from "./tts-config";
+import { RealtimeConfigList } from "./realtime-chat/realtime-config";

 function EditPromptModal(props: { id: string; onClose: () => void }) {
  const promptStore = usePromptStore();
@@ -1234,6 +1236,47 @@ export function Settings() {
    </>
  );

+  const chatglmConfigComponent = accessStore.provider ===
+    ServiceProvider.ChatGLM && (
+    <>
+      <ListItem
+        title={Locale.Settings.Access.ChatGLM.Endpoint.Title}
+        subTitle={
+          Locale.Settings.Access.ChatGLM.Endpoint.SubTitle +
+          ChatGLM.ExampleEndpoint
+        }
+      >
+        <input
+          aria-label={Locale.Settings.Access.ChatGLM.Endpoint.Title}
+          type="text"
+          value={accessStore.chatglmUrl}
+          placeholder={ChatGLM.ExampleEndpoint}
+          onChange={(e) =>
+            accessStore.update(
+              (access) => (access.chatglmUrl = e.currentTarget.value),
+            )
+          }
+        ></input>
+      </ListItem>
+      <ListItem
+        title={Locale.Settings.Access.ChatGLM.ApiKey.Title}
+        subTitle={Locale.Settings.Access.ChatGLM.ApiKey.SubTitle}
+      >
+        <PasswordInput
+          aria-label={Locale.Settings.Access.ChatGLM.ApiKey.Title}
+          value={accessStore.chatglmApiKey}
+          type="text"
+          placeholder={Locale.Settings.Access.ChatGLM.ApiKey.Placeholder}
+          onChange={(e) => {
+            accessStore.update(
+              (access) => (access.chatglmApiKey = e.currentTarget.value),
+            );
+          }}
+        />
+      </ListItem>
+    </>
+  );
+
  const stabilityConfigComponent = accessStore.provider ===
    ServiceProvider.Stability && (
    <>
@@ -1693,6 +1736,7 @@ export function Settings() {
                  {stabilityConfigComponent}
                  {lflytekConfigComponent}
                  {XAIConfigComponent}
+                  {chatglmConfigComponent}
                </>
              )}
            </>
@@ -1756,7 +1800,18 @@ export function Settings() {
        {shouldShowPromptModal && (
          <UserPromptModal onClose={() => setShowPromptModal(false)} />
        )}
-
+        <List>
+          <RealtimeConfigList
+            realtimeConfig={config.realtimeConfig}
+            updateConfig={(updater) => {
+              const realtimeConfig = { ...config.realtimeConfig };
+              updater(realtimeConfig);
+              config.update(
+                (config) => (config.realtimeConfig = realtimeConfig),
+              );
+            }}
+          />
+        </List>
        <List>
          <TTSConfigList
            ttsConfig={config.ttsConfig}
--- a/app/components/sidebar.tsx
+++ b/app/components/sidebar.tsx
@@ -30,6 +30,7 @@ import { Link, useNavigate } from "react-router-dom";
 import { isIOS, useMobileScreen } from "../utils";
 import dynamic from "next/dynamic";
 import { showConfirm, Selector } from "./ui-lib";
+import clsx from "clsx";

 const ChatList = dynamic(async () => (await import("./chat-list")).ChatList, {
  loading: () => null,
@@ -141,9 +142,9 @@ export function SideBarContainer(props: {
  const { children, className, onDragStart, shouldNarrow } = props;
  return (
    <div
-      className={`${styles.sidebar} ${className} ${
-        shouldNarrow && styles["narrow-sidebar"]
-      }`}
+      className={clsx(styles.sidebar, className, {
+        [styles["narrow-sidebar"]]: shouldNarrow,
+      })}
      style={{
        // #3016 disable transition on ios mobile screen
        transition: isMobileScreen && isIOSMobile ? "none" : undefined,
@@ -171,9 +172,9 @@ export function SideBarHeader(props: {
  return (
    <Fragment>
      <div
-        className={`${styles["sidebar-header"]} ${
-          shouldNarrow ? styles["sidebar-header-narrow"] : ""
-        }`}
+        className={clsx(styles["sidebar-header"], {
+          [styles["sidebar-header-narrow"]]: shouldNarrow,
+        })}
        data-tauri-drag-region
      >
        <div className={styles["sidebar-title-container"]}>
@@ -182,7 +183,7 @@ export function SideBarHeader(props: {
          </div>
          <div className={styles["sidebar-sub-title"]}>{subTitle}</div>
        </div>
-        <div className={styles["sidebar-logo"] + " no-dark"}>{logo}</div>
+        <div className={clsx(styles["sidebar-logo"], "no-dark")}>{logo}</div>
      </div>
      {children}
    </Fragment>
@@ -286,7 +287,7 @@ export function SideBar(props: { className?: string }) {
      <SideBarTail
        primaryAction={
          <>
-            <div className={styles["sidebar-action"] + " " + styles.mobile}>
+            <div className={clsx(styles["sidebar-action"], styles.mobile)}>
              <IconButton
                icon={<DeleteIcon />}
                onClick={async () => {
--- a/app/components/ui-lib.tsx
+++ b/app/components/ui-lib.tsx
@@ -23,6 +23,7 @@ import React, {
  useRef,
 } from "react";
 import { IconButton } from "./button";
+import clsx from "clsx";

 export function Popover(props: {
  children: JSX.Element;
@@ -45,7 +46,7 @@ export function Popover(props: {

 export function Card(props: { children: JSX.Element[]; className?: string }) {
  return (
-    <div className={styles.card + " " + props.className}>{props.children}</div>
+    <div className={clsx(styles.card, props.className)}>{props.children}</div>
  );
 }

@@ -60,11 +61,13 @@ export function ListItem(props: {
 }) {
  return (
    <div
-      className={
-        styles["list-item"] +
-        ` ${props.vertical ? styles["vertical"] : ""} ` +
-        ` ${props.className || ""}`
-      }
+      className={clsx(
+        styles["list-item"],
+        {
+          [styles["vertical"]]: props.vertical,
+        },
+        props.className,
+      )}
      onClick={props.onClick}
    >
      <div className={styles["list-header"]}>
@@ -135,9 +138,9 @@ export function Modal(props: ModalProps) {

  return (
    <div
-      className={
-        styles["modal-container"] + ` ${isMax && styles["modal-container-max"]}`
-      }
+      className={clsx(styles["modal-container"], {
+        [styles["modal-container-max"]]: isMax,
+      })}
    >
      <div className={styles["modal-header"]}>
        <div className={styles["modal-title"]}>{props.title}</div>
@@ -260,7 +263,7 @@ export function Input(props: InputProps) {
  return (
    <textarea
      {...props}
-      className={`${styles["input"]} ${props.className}`}
+      className={clsx(styles["input"], props.className)}
    ></textarea>
  );
 }
@@ -301,9 +304,13 @@ export function Select(
  const { className, children, align, ...otherProps } = props;
  return (
    <div
-      className={`${styles["select-with-icon"]} ${
-        align === "left" ? styles["left-align-option"] : ""
-      } ${className}`}
+      className={clsx(
+        styles["select-with-icon"],
+        {
+          [styles["left-align-option"]]: align === "left",
+        },
+        className,
+      )}
    >
      <select className={styles["select-with-icon-select"]} {...otherProps}>
        {children}
@@ -509,9 +516,9 @@ export function Selector<T>(props: {
            const selected = selectedValues.includes(item.value);
            return (
              <ListItem
-                className={`${styles["selector-item"]} ${
-                  item.disable && styles["selector-item-disabled"]
-                }`}
+                className={clsx(styles["selector-item"], {
+                  [styles["selector-item-disabled"]]: item.disable,
+                })}
                key={i}
                title={item.title}
                subTitle={item.subTitle}
--- a/app/components/voice-print/index.ts
+++ b/app/components/voice-print/index.ts
@@ -0,0 +1 @@
+export * from "./voice-print";
--- a/app/components/voice-print/voice-print.module.scss
+++ b/app/components/voice-print/voice-print.module.scss
@@ -0,0 +1,11 @@
+.voice-print {
+  width: 100%;
+  height: 60px;
+  margin: 20px 0;
+
+  canvas {
+    width: 100%;
+    height: 100%;
+    filter: brightness(1.2); // 增加整体亮度
+  }
+}
--- a/app/components/voice-print/voice-print.tsx
+++ b/app/components/voice-print/voice-print.tsx
@@ -0,0 +1,180 @@
+import { useEffect, useRef, useCallback } from "react";
+import styles from "./voice-print.module.scss";
+
+interface VoicePrintProps {
+  frequencies?: Uint8Array;
+  isActive?: boolean;
+}
+
+export function VoicePrint({ frequencies, isActive }: VoicePrintProps) {
+  // Canvas引用，用于获取绘图上下文
+  const canvasRef = useRef<HTMLCanvasElement>(null);
+  // 存储历史频率数据，用于平滑处理
+  const historyRef = useRef<number[][]>([]);
+  // 控制保留的历史数据帧数，影响平滑度
+  const historyLengthRef = useRef(10);
+  // 存储动画帧ID，用于清理
+  const animationFrameRef = useRef<number>();
+
+  /**
+   * 更新频率历史数据
+   * 使用FIFO队列维护固定长度的历史记录
+   */
+  const updateHistory = useCallback((freqArray: number[]) => {
+    historyRef.current.push(freqArray);
+    if (historyRef.current.length > historyLengthRef.current) {
+      historyRef.current.shift();
+    }
+  }, []);
+
+  useEffect(() => {
+    const canvas = canvasRef.current;
+    if (!canvas) return;
+
+    const ctx = canvas.getContext("2d");
+    if (!ctx) return;
+
+    /**
+     * 处理高DPI屏幕显示
+     * 根据设备像素比例调整canvas实际渲染分辨率
+     */
+    const dpr = window.devicePixelRatio || 1;
+    canvas.width = canvas.offsetWidth * dpr;
+    canvas.height = canvas.offsetHeight * dpr;
+    ctx.scale(dpr, dpr);
+
+    /**
+     * 主要绘制函数
+     * 使用requestAnimationFrame实现平滑动画
+     * 包含以下步骤：
+     * 1. 清空画布
+     * 2. 更新历史数据
+     * 3. 计算波形点
+     * 4. 绘制上下对称的声纹
+     */
+    const draw = () => {
+      // 清空画布
+      ctx.clearRect(0, 0, canvas.width, canvas.height);
+
+      if (!frequencies || !isActive) {
+        historyRef.current = [];
+        return;
+      }
+
+      const freqArray = Array.from(frequencies);
+      updateHistory(freqArray);
+
+      // 绘制声纹
+      const points: [number, number][] = [];
+      const centerY = canvas.height / 2;
+      const width = canvas.width;
+      const sliceWidth = width / (frequencies.length - 1);
+
+      // 绘制主波形
+      ctx.beginPath();
+      ctx.moveTo(0, centerY);
+
+      /**
+       * 声纹绘制算法：
+       * 1. 使用历史数据平均值实现平滑过渡
+       * 2. 通过正弦函数添加自然波动
+       * 3. 使用贝塞尔曲线连接点，使曲线更平滑
+       * 4. 绘制对称部分形成完整声纹
+       */
+      for (let i = 0; i < frequencies.length; i++) {
+        const x = i * sliceWidth;
+        let avgFrequency = frequencies[i];
+
+        /**
+         * 波形平滑处理：
+         * 1. 收集历史数据中对应位置的频率值
+         * 2. 计算当前值与历史值的加权平均
+         * 3. 根据平均值计算实际显示高度
+         */
+        if (historyRef.current.length > 0) {
+          const historicalValues = historyRef.current.map((h) => h[i] || 0);
+          avgFrequency =
+            (avgFrequency + historicalValues.reduce((a, b) => a + b, 0)) /
+            (historyRef.current.length + 1);
+        }
+
+        /**
+         * 波形变换：
+         * 1. 归一化频率值到0-1范围
+         * 2. 添加时间相关的正弦变换
+         * 3. 使用贝塞尔曲线平滑连接点
+         */
+        const normalized = avgFrequency / 255.0;
+        const height = normalized * (canvas.height / 2);
+        const y = centerY + height * Math.sin(i * 0.2 + Date.now() * 0.002);
+
+        points.push([x, y]);
+
+        if (i === 0) {
+          ctx.moveTo(x, y);
+        } else {
+          // 使用贝塞尔曲线使波形更平滑
+          const prevPoint = points[i - 1];
+          const midX = (prevPoint[0] + x) / 2;
+          ctx.quadraticCurveTo(
+            prevPoint[0],
+            prevPoint[1],
+            midX,
+            (prevPoint[1] + y) / 2,
+          );
+        }
+      }
+
+      // 绘制对称的下半部分
+      for (let i = points.length - 1; i >= 0; i--) {
+        const [x, y] = points[i];
+        const symmetricY = centerY - (y - centerY);
+        if (i === points.length - 1) {
+          ctx.lineTo(x, symmetricY);
+        } else {
+          const nextPoint = points[i + 1];
+          const midX = (nextPoint[0] + x) / 2;
+          ctx.quadraticCurveTo(
+            nextPoint[0],
+            centerY - (nextPoint[1] - centerY),
+            midX,
+            centerY - ((nextPoint[1] + y) / 2 - centerY),
+          );
+        }
+      }
+
+      ctx.closePath();
+
+      /**
+       * 渐变效果：
+       * 从左到右应用三色渐变，带透明度
+       * 使用蓝色系配色提升视觉效果
+       */
+      const gradient = ctx.createLinearGradient(0, 0, canvas.width, 0);
+      gradient.addColorStop(0, "rgba(100, 180, 255, 0.95)");
+      gradient.addColorStop(0.5, "rgba(140, 200, 255, 0.9)");
+      gradient.addColorStop(1, "rgba(180, 220, 255, 0.95)");
+
+      ctx.fillStyle = gradient;
+      ctx.fill();
+
+      animationFrameRef.current = requestAnimationFrame(draw);
+    };
+
+    // 启动动画循环
+    draw();
+
+    // 清理函数：在组件卸载时取消动画
+    return () => {
+      if (animationFrameRef.current) {
+        cancelAnimationFrame(animationFrameRef.current);
+      }
+    };
+  }, [frequencies, isActive, updateHistory]);
+
+  return (
+    <div className={styles["voice-print"]}>
+      <canvas ref={canvasRef} />
+    </div>
+  );
+}
--- a/app/config/server.ts
+++ b/app/config/server.ts
@@ -75,6 +75,10 @@ declare global {
      XAI_URL?: string;
      XAI_API_KEY?: string;

+      // chatglm only
+      CHATGLM_URL?: string;
+      CHATGLM_API_KEY?: string;
+
      // custom template for preprocessing user input
      DEFAULT_INPUT_TEMPLATE?: string;
    }
@@ -151,6 +155,7 @@ export const getServerSideConfig = () => {
  const isMoonshot = !!process.env.MOONSHOT_API_KEY;
  const isIflytek = !!process.env.IFLYTEK_API_KEY;
  const isXAI = !!process.env.XAI_API_KEY;
+  const isChatGLM = !!process.env.CHATGLM_API_KEY;
  // const apiKeyEnvVar = process.env.OPENAI_API_KEY ?? "";
  // const apiKeys = apiKeyEnvVar.split(",").map((v) => v.trim());
  // const randomIndex = Math.floor(Math.random() * apiKeys.length);
@@ -217,6 +222,10 @@ export const getServerSideConfig = () => {
    xaiUrl: process.env.XAI_URL,
    xaiApiKey: getApiKey(process.env.XAI_API_KEY),

+    isChatGLM,
+    chatglmUrl: process.env.CHATGLM_URL,
+    chatglmApiKey: getApiKey(process.env.CHATGLM_API_KEY),
+
    cloudflareAccountId: process.env.CLOUDFLARE_ACCOUNT_ID,
    cloudflareKVNamespaceId: process.env.CLOUDFLARE_KV_NAMESPACE_ID,
    cloudflareKVApiKey: getApiKey(process.env.CLOUDFLARE_KV_API_KEY),
--- a/app/constant.ts
+++ b/app/constant.ts
@@ -30,6 +30,8 @@ export const IFLYTEK_BASE_URL = "https://spark-api-open.xf-yun.com";

 export const XAI_BASE_URL = "https://api.x.ai";

+export const CHATGLM_BASE_URL = "https://open.bigmodel.cn";
+
 export const CACHE_URL_PREFIX = "/api/cache";
 export const UPLOAD_URL = `${CACHE_URL_PREFIX}/upload`;

@@ -62,6 +64,7 @@ export enum ApiPath {
  Stability = "/api/stability",
  Artifacts = "/api/artifacts",
  XAI = "/api/xai",
+  ChatGLM = "/api/chatglm",
 }

 export enum SlotID {
@@ -115,6 +118,7 @@ export enum ServiceProvider {
  Stability = "Stability",
  Iflytek = "Iflytek",
  XAI = "XAI",
+  ChatGLM = "ChatGLM",
 }

 // Google API safety settings, see https://ai.google.dev/gemini-api/docs/safety-settings
@@ -138,6 +142,7 @@ export enum ModelProvider {
  Moonshot = "Moonshot",
  Iflytek = "Iflytek",
  XAI = "XAI",
+  ChatGLM = "ChatGLM",
 }

 export const Stability = {
@@ -225,6 +230,11 @@ export const XAI = {
  ChatPath: "v1/chat/completions",
 };

+export const ChatGLM = {
+  ExampleEndpoint: CHATGLM_BASE_URL,
+  ChatPath: "api/paas/v4/chat/completions",
+};
+
 export const DEFAULT_INPUT_TEMPLATE = `{{input}}`; // input / time / model / lang
 // export const DEFAULT_SYSTEM_TEMPLATE = `
 // You are ChatGPT, a large language model trained by {{ServiceProvider}}.
@@ -317,8 +327,13 @@ const anthropicModels = [
  "claude-2.1",
  "claude-3-sonnet-20240229",
  "claude-3-opus-20240229",
+  "claude-3-opus-latest",
  "claude-3-haiku-20240307",
+  "claude-3-5-haiku-20241022",
+  "claude-3-5-haiku-latest",
  "claude-3-5-sonnet-20240620",
+  "claude-3-5-sonnet-20241022",
+  "claude-3-5-sonnet-latest",
 ];

 const baiduModels = [
@@ -376,6 +391,17 @@ const iflytekModels = [

 const xAIModes = ["grok-beta"];

+const chatglmModels = [
+  "glm-4-plus",
+  "glm-4-0520",
+  "glm-4",
+  "glm-4-air",
+  "glm-4-airx",
+  "glm-4-long",
+  "glm-4-flashx",
+  "glm-4-flash",
+];
+
 let seq = 1000; // 内置的模型序号生成器从1000开始
 export const DEFAULT_MODELS = [
  ...openaiModels.map((name) => ({
@@ -499,6 +525,17 @@ export const DEFAULT_MODELS = [
      sorted: 11,
    },
  })),
+  ...chatglmModels.map((name) => ({
+    name,
+    available: true,
+    sorted: seq++,
+    provider: {
+      id: "chatglm",
+      providerName: "ChatGLM",
+      providerType: "chatglm",
+      sorted: 12,
+    },
+  })),
 ] as const;

 export const CHAT_PAGE_SIZE = 15;
--- a/app/icons/headphone.svg
+++ b/app/icons/headphone.svg
@@ -0,0 +1,11 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="16" height="16" viewBox="0 0 48 48" fill="none" xmlns="http://www.w3.org/2000/svg">
+    <path d="M4 28C4 26.8954 4.89543 26 6 26H10V38H6C4.89543 38 4 37.1046 4 36V28Z" fill="none" />
+    <path d="M38 26H42C43.1046 26 44 26.8954 44 28V36C44 37.1046 43.1046 38 42 38H38V26Z"
+        fill="none" />
+    <path
+        d="M10 36V24C10 16.268 16.268 10 24 10C31.732 10 38 16.268 38 24V36M10 26H6C4.89543 26 4 26.8954 4 28V36C4 37.1046 4.89543 38 6 38H10V26ZM38 26H42C43.1046 26 44 26.8954 44 28V36C44 37.1046 43.1046 38 42 38H38V26Z"
+        stroke="#333" stroke-width="4" stroke-linecap="round" stroke-linejoin="round" />
+    <path d="M16 32H20L22 26L26 38L28 32H32" stroke="#333" stroke-width="4" stroke-linecap="round"
+        stroke-linejoin="round" />
+</svg>
--- a/app/icons/power.svg
+++ b/app/icons/power.svg
@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="24" height="24" viewBox="0 0 48 48" fill="none" xmlns="http://www.w3.org/2000/svg">
+    <path
+        d="M14.5 8C13.8406 8.37652 13.2062 8.79103 12.6 9.24051C11.5625 10.0097 10.6074 10.8814 9.75 11.8402C6.79377 15.1463 5 19.4891 5 24.2455C5 34.6033 13.5066 43 24 43C34.4934 43 43 34.6033 43 24.2455C43 19.4891 41.2062 15.1463 38.25 11.8402C37.3926 10.8814 36.4375 10.0097 35.4 9.24051C34.7938 8.79103 34.1594 8.37652 33.5 8"
+        stroke="#333" stroke-width="4" stroke-linecap="round" stroke-linejoin="round" />
+    <path d="M24 4V24" stroke="#333" stroke-width="4" stroke-linecap="round" stroke-linejoin="round" />
+</svg>
--- a/app/icons/voice-off.svg
+++ b/app/icons/voice-off.svg
@@ -0,0 +1,13 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="24" height="24" viewBox="0 0 48 48" fill="none" xmlns="http://www.w3.org/2000/svg">
+    <path
+        d="M31 24V11C31 7.13401 27.866 4 24 4C20.134 4 17 7.13401 17 11V24C17 27.866 20.134 31 24 31C27.866 31 31 27.866 31 24Z"
+        stroke="#d0021b" stroke-width="4" stroke-linejoin="round" />
+    <path
+        d="M9 23C9 31.2843 15.7157 38 24 38C25.7532 38 27.4361 37.6992 29 37.1465M39 23C39 25.1333 38.5547 27.1626 37.7519 29"
+        stroke="#d0021b" stroke-width="4" stroke-linecap="round" stroke-linejoin="round" />
+    <path d="M24 38V44" stroke="#d0021b" stroke-width="4" stroke-linecap="round"
+        stroke-linejoin="round" />
+    <path d="M42 42L6 6" stroke="#d0021b" stroke-width="4" stroke-linecap="round"
+        stroke-linejoin="round" />
+</svg>
--- a/app/icons/voice.svg
+++ b/app/icons/voice.svg
@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="24" height="24" viewBox="0 0 48 48" fill="none" xmlns="http://www.w3.org/2000/svg">
+    <rect x="17" y="4" width="14" height="27" rx="7" fill="none" stroke="#333" stroke-width="4"
+        stroke-linejoin="round" />
+    <path d="M9 23C9 31.2843 15.7157 38 24 38C32.2843 38 39 31.2843 39 23" stroke="#333"
+        stroke-width="4" stroke-linecap="round" stroke-linejoin="round" />
+    <path d="M24 38V44" stroke="#333" stroke-width="4" stroke-linecap="round"
+        stroke-linejoin="round" />
+</svg>
--- a/app/lib/audio.ts
+++ b/app/lib/audio.ts
@@ -0,0 +1,200 @@
+export class AudioHandler {
+  private context: AudioContext;
+  private mergeNode: ChannelMergerNode;
+  private analyserData: Uint8Array;
+  public analyser: AnalyserNode;
+  private workletNode: AudioWorkletNode | null = null;
+  private stream: MediaStream | null = null;
+  private source: MediaStreamAudioSourceNode | null = null;
+  private recordBuffer: Int16Array[] = [];
+  private readonly sampleRate = 24000;
+
+  private nextPlayTime: number = 0;
+  private isPlaying: boolean = false;
+  private playbackQueue: AudioBufferSourceNode[] = [];
+  private playBuffer: Int16Array[] = [];
+
+  constructor() {
+    this.context = new AudioContext({ sampleRate: this.sampleRate });
+    // using ChannelMergerNode to get merged audio data, and then get analyser data.
+    this.mergeNode = new ChannelMergerNode(this.context, { numberOfInputs: 2 });
+    this.analyser = new AnalyserNode(this.context, { fftSize: 256 });
+    this.analyserData = new Uint8Array(this.analyser.frequencyBinCount);
+    this.mergeNode.connect(this.analyser);
+  }
+
+  getByteFrequencyData() {
+    this.analyser.getByteFrequencyData(this.analyserData);
+    return this.analyserData;
+  }
+
+  async initialize() {
+    await this.context.audioWorklet.addModule("/audio-processor.js");
+  }
+
+  async startRecording(onChunk: (chunk: Uint8Array) => void) {
+    try {
+      if (!this.workletNode) {
+        await this.initialize();
+      }
+
+      this.stream = await navigator.mediaDevices.getUserMedia({
+        audio: {
+          channelCount: 1,
+          sampleRate: this.sampleRate,
+          echoCancellation: true,
+          noiseSuppression: true,
+        },
+      });
+
+      await this.context.resume();
+      this.source = this.context.createMediaStreamSource(this.stream);
+      this.workletNode = new AudioWorkletNode(
+        this.context,
+        "audio-recorder-processor",
+      );
+
+      this.workletNode.port.onmessage = (event) => {
+        if (event.data.eventType === "audio") {
+          const float32Data = event.data.audioData;
+          const int16Data = new Int16Array(float32Data.length);
+
+          for (let i = 0; i < float32Data.length; i++) {
+            const s = Math.max(-1, Math.min(1, float32Data[i]));
+            int16Data[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
+          }
+
+          const uint8Data = new Uint8Array(int16Data.buffer);
+          onChunk(uint8Data);
+          // save recordBuffer
+          // @ts-ignore
+          this.recordBuffer.push.apply(this.recordBuffer, int16Data);
+        }
+      };
+
+      this.source.connect(this.workletNode);
+      this.source.connect(this.mergeNode, 0, 0);
+      this.workletNode.connect(this.context.destination);
+
+      this.workletNode.port.postMessage({ command: "START_RECORDING" });
+    } catch (error) {
+      console.error("Error starting recording:", error);
+      throw error;
+    }
+  }
+
+  stopRecording() {
+    if (!this.workletNode || !this.source || !this.stream) {
+      throw new Error("Recording not started");
+    }
+
+    this.workletNode.port.postMessage({ command: "STOP_RECORDING" });
+
+    this.workletNode.disconnect();
+    this.source.disconnect();
+    this.stream.getTracks().forEach((track) => track.stop());
+  }
+  startStreamingPlayback() {
+    this.isPlaying = true;
+    this.nextPlayTime = this.context.currentTime;
+  }
+
+  stopStreamingPlayback() {
+    this.isPlaying = false;
+    this.playbackQueue.forEach((source) => source.stop());
+    this.playbackQueue = [];
+    this.playBuffer = [];
+  }
+
+  playChunk(chunk: Uint8Array) {
+    if (!this.isPlaying) return;
+
+    const int16Data = new Int16Array(chunk.buffer);
+    // @ts-ignore
+    this.playBuffer.push.apply(this.playBuffer, int16Data); // save playBuffer
+
+    const float32Data = new Float32Array(int16Data.length);
+    for (let i = 0; i < int16Data.length; i++) {
+      float32Data[i] = int16Data[i] / (int16Data[i] < 0 ? 0x8000 : 0x7fff);
+    }
+
+    const audioBuffer = this.context.createBuffer(
+      1,
+      float32Data.length,
+      this.sampleRate,
+    );
+    audioBuffer.getChannelData(0).set(float32Data);
+
+    const source = this.context.createBufferSource();
+    source.buffer = audioBuffer;
+    source.connect(this.context.destination);
+    source.connect(this.mergeNode, 0, 1);
+
+    const chunkDuration = audioBuffer.length / this.sampleRate;
+
+    source.start(this.nextPlayTime);
+
+    this.playbackQueue.push(source);
+    source.onended = () => {
+      const index = this.playbackQueue.indexOf(source);
+      if (index > -1) {
+        this.playbackQueue.splice(index, 1);
+      }
+    };
+
+    this.nextPlayTime += chunkDuration;
+
+    if (this.nextPlayTime < this.context.currentTime) {
+      this.nextPlayTime = this.context.currentTime;
+    }
+  }
+  _saveData(data: Int16Array, bytesPerSample = 16): Blob {
+    const headerLength = 44;
+    const numberOfChannels = 1;
+    const byteLength = data.buffer.byteLength;
+    const header = new Uint8Array(headerLength);
+    const view = new DataView(header.buffer);
+    view.setUint32(0, 1380533830, false); // RIFF identifier 'RIFF'
+    view.setUint32(4, 36 + byteLength, true); // file length minus RIFF identifier length and file description length
+    view.setUint32(8, 1463899717, false); // RIFF type 'WAVE'
+    view.setUint32(12, 1718449184, false); // format chunk identifier 'fmt '
+    view.setUint32(16, 16, true); // format chunk length
+    view.setUint16(20, 1, true); // sample format (raw)
+    view.setUint16(22, numberOfChannels, true); // channel count
+    view.setUint32(24, this.sampleRate, true); // sample rate
+    view.setUint32(28, this.sampleRate * 4, true); // byte rate (sample rate * block align)
+    view.setUint16(32, numberOfChannels * 2, true); // block align (channel count * bytes per sample)
+    view.setUint16(34, bytesPerSample, true); // bits per sample
+    view.setUint32(36, 1684108385, false); // data chunk identifier 'data'
+    view.setUint32(40, byteLength, true); // data chunk length
+
+    // using data.buffer, so no need to setUint16 to view.
+    return new Blob([view, data.buffer], { type: "audio/mpeg" });
+  }
+  savePlayFile() {
+    // @ts-ignore
+    return this._saveData(new Int16Array(this.playBuffer));
+  }
+  saveRecordFile(
+    audioStartMillis: number | undefined,
+    audioEndMillis: number | undefined,
+  ) {
+    const startIndex = audioStartMillis
+      ? Math.floor((audioStartMillis * this.sampleRate) / 1000)
+      : 0;
+    const endIndex = audioEndMillis
+      ? Math.floor((audioEndMillis * this.sampleRate) / 1000)
+      : this.recordBuffer.length;
+    return this._saveData(
+      // @ts-ignore
+      new Int16Array(this.recordBuffer.slice(startIndex, endIndex)),
+    );
+  }
+  async close() {
+    this.recordBuffer = [];
+    this.workletNode?.disconnect();
+    this.source?.disconnect();
+    this.stream?.getTracks().forEach((track) => track.stop());
+    await this.context.close();
+  }
+}
--- a/app/locales/cn.ts
+++ b/app/locales/cn.ts
@@ -473,6 +473,17 @@ const cn = {
          SubTitle: "样例：",
        },
      },
+      ChatGLM: {
+        ApiKey: {
+          Title: "接口密钥",
+          SubTitle: "使用自定义 ChatGLM API Key",
+          Placeholder: "ChatGLM API Key",
+        },
+        Endpoint: {
+          Title: "接口地址",
+          SubTitle: "样例：",
+        },
+      },
      Stability: {
        ApiKey: {
          Title: "接口密钥",
@@ -551,6 +562,39 @@ const cn = {
        SubTitle: "生成语音的速度",
      },
    },
+    Realtime: {
+      Enable: {
+        Title: "实时聊天",
+        SubTitle: "开启实时聊天功能",
+      },
+      Provider: {
+        Title: "模型服务商",
+        SubTitle: "切换不同的服务商",
+      },
+      Model: {
+        Title: "模型",
+        SubTitle: "选择一个模型",
+      },
+      ApiKey: {
+        Title: "API Key",
+        SubTitle: "API Key",
+        Placeholder: "API Key",
+      },
+      Azure: {
+        Endpoint: {
+          Title: "接口地址",
+          SubTitle: "接口地址",
+        },
+        Deployment: {
+          Title: "部署名称",
+          SubTitle: "部署名称",
+        },
+      },
+      Temperature: {
+        Title: "随机性 (temperature)",
+        SubTitle: "值越大，回复越随机",
+      },
+    },
  },
  Store: {
    DefaultTopic: "新的聊天",
--- a/app/locales/en.ts
+++ b/app/locales/en.ts
@@ -457,6 +457,17 @@ const en: LocaleType = {
          SubTitle: "Example: ",
        },
      },
+      ChatGLM: {
+        ApiKey: {
+          Title: "ChatGLM API Key",
+          SubTitle: "Use a custom ChatGLM API Key",
+          Placeholder: "ChatGLM API Key",
+        },
+        Endpoint: {
+          Title: "Endpoint Address",
+          SubTitle: "Example: ",
+        },
+      },
      Stability: {
        ApiKey: {
          Title: "Stability API Key",
@@ -559,6 +570,39 @@ const en: LocaleType = {
      },
      Engine: "TTS Engine",
    },
+    Realtime: {
+      Enable: {
+        Title: "Realtime Chat",
+        SubTitle: "Enable realtime chat feature",
+      },
+      Provider: {
+        Title: "Model Provider",
+        SubTitle: "Switch between different providers",
+      },
+      Model: {
+        Title: "Model",
+        SubTitle: "Select a model",
+      },
+      ApiKey: {
+        Title: "API Key",
+        SubTitle: "API Key",
+        Placeholder: "API Key",
+      },
+      Azure: {
+        Endpoint: {
+          Title: "Endpoint",
+          SubTitle: "Endpoint",
+        },
+        Deployment: {
+          Title: "Deployment Name",
+          SubTitle: "Deployment Name",
+        },
+      },
+      Temperature: {
+        Title: "Randomness (temperature)",
+        SubTitle: "Higher values result in more random responses",
+      },
+    },
  },
  Store: {
    DefaultTopic: "New Conversation",
--- a/app/store/access.ts
+++ b/app/store/access.ts
@@ -14,12 +14,14 @@ import {
  STABILITY_BASE_URL,
  IFLYTEK_BASE_URL,
  XAI_BASE_URL,
+  CHATGLM_BASE_URL,
 } from "../constant";
 import { getHeaders } from "../client/api";
 import { getClientConfig } from "../config/client";
 import { createPersistStore } from "../utils/store";
 import { ensure } from "../utils/clone";
 import { DEFAULT_CONFIG } from "./config";
+import { getModelProvider } from "../utils/model";

 let fetchState = 0; // 0 not fetch, 1 fetching, 2 done

@@ -47,6 +49,8 @@ const DEFAULT_IFLYTEK_URL = isApp ? IFLYTEK_BASE_URL : ApiPath.Iflytek;

 const DEFAULT_XAI_URL = isApp ? XAI_BASE_URL : ApiPath.XAI;

+const DEFAULT_CHATGLM_URL = isApp ? CHATGLM_BASE_URL : ApiPath.ChatGLM;
+
 const DEFAULT_ACCESS_STATE = {
  accessCode: "",
  useCustomConfig: false,
@@ -108,6 +112,10 @@ const DEFAULT_ACCESS_STATE = {
  xaiUrl: DEFAULT_XAI_URL,
  xaiApiKey: "",

+  // chatglm
+  chatglmUrl: DEFAULT_CHATGLM_URL,
+  chatglmApiKey: "",
+
  // server config
  needCode: true,
  hideUserApiKey: false,
@@ -180,6 +188,10 @@ export const useAccessStore = createPersistStore(
      return ensure(get(), ["xaiApiKey"]);
    },

+    isValidChatGLM() {
+      return ensure(get(), ["chatglmApiKey"]);
+    },
+
    isAuthorized() {
      this.fetch();

@@ -196,6 +208,7 @@ export const useAccessStore = createPersistStore(
        this.isValidMoonshot() ||
        this.isValidIflytek() ||
        this.isValidXAI() ||
+        this.isValidChatGLM() ||
        !this.enabledAccessControl() ||
        (this.enabledAccessControl() && ensure(get(), ["accessCode"]))
      );
@@ -214,9 +227,9 @@ export const useAccessStore = createPersistStore(
        .then((res) => {
          const defaultModel = res.defaultModel ?? "";
          if (defaultModel !== "") {
-            const [model, providerName] = defaultModel.split("@");
+            const [model, providerName] = getModelProvider(defaultModel);
            DEFAULT_CONFIG.modelConfig.model = model;
-            DEFAULT_CONFIG.modelConfig.providerName = providerName;
+            DEFAULT_CONFIG.modelConfig.providerName = providerName as any;
          }

          return res;
--- a/app/store/chat.ts
+++ b/app/store/chat.ts
@@ -52,6 +52,7 @@ export type ChatMessage = RequestMessage & {
  id: string;
  model?: ModelType;
  tools?: ChatMessageTool[];
+  audio_url?: string;
 };

 export function createMessage(override: Partial<ChatMessage>): ChatMessage {
@@ -352,13 +353,13 @@ export const useChatStore = createPersistStore(
        return session;
      },

-      onNewMessage(message: ChatMessage) {
-        get().updateCurrentSession((session) => {
+      onNewMessage(message: ChatMessage, targetSession: ChatSession) {
+        get().updateTargetSession(targetSession, (session) => {
          session.messages = session.messages.concat();
          session.lastUpdate = Date.now();
        });
-        get().updateStat(message);
-        get().summarizeSession();
+        get().updateStat(message, targetSession);
+        get().summarizeSession(false, targetSession);
      },

      async onUserInput(content: string, attachImages?: string[]) {
@@ -396,10 +397,10 @@ export const useChatStore = createPersistStore(
        // get recent messages
        const recentMessages = get().getMessagesWithMemory();
        const sendMessages = recentMessages.concat(userMessage);
-        const messageIndex = get().currentSession().messages.length + 1;
+        const messageIndex = session.messages.length + 1;

        // save user's and bot's message
-        get().updateCurrentSession((session) => {
+        get().updateTargetSession(session, (session) => {
          const savedUserMessage = {
            ...userMessage,
            content: mContent,
@@ -420,7 +421,7 @@ export const useChatStore = createPersistStore(
            if (message) {
              botMessage.content = message;
            }
-            get().updateCurrentSession((session) => {
+            get().updateTargetSession(session, (session) => {
              session.messages = session.messages.concat();
            });
          },
@@ -428,13 +429,14 @@ export const useChatStore = createPersistStore(
            botMessage.streaming = false;
            if (message) {
              botMessage.content = message;
-              get().onNewMessage(botMessage);
+              botMessage.date = new Date().toLocaleString();
+              get().onNewMessage(botMessage, session);
            }
            ChatControllerPool.remove(session.id, botMessage.id);
          },
          onBeforeTool(tool: ChatMessageTool) {
            (botMessage.tools = botMessage?.tools || []).push(tool);
-            get().updateCurrentSession((session) => {
+            get().updateTargetSession(session, (session) => {
              session.messages = session.messages.concat();
            });
          },
@@ -444,7 +446,7 @@ export const useChatStore = createPersistStore(
                tools[i] = { ...tool };
              }
            });
-            get().updateCurrentSession((session) => {
+            get().updateTargetSession(session, (session) => {
              session.messages = session.messages.concat();
            });
          },
@@ -459,7 +461,7 @@ export const useChatStore = createPersistStore(
            botMessage.streaming = false;
            userMessage.isError = !isAborted;
            botMessage.isError = !isAborted;
-            get().updateCurrentSession((session) => {
+            get().updateTargetSession(session, (session) => {
              session.messages = session.messages.concat();
            });
            ChatControllerPool.remove(
@@ -591,16 +593,19 @@ export const useChatStore = createPersistStore(
        set(() => ({ sessions }));
      },

-      resetSession() {
-        get().updateCurrentSession((session) => {
+      resetSession(session: ChatSession) {
+        get().updateTargetSession(session, (session) => {
          session.messages = [];
          session.memoryPrompt = "";
        });
      },

-      summarizeSession(refreshTitle: boolean = false) {
+      summarizeSession(
+        refreshTitle: boolean = false,
+        targetSession: ChatSession,
+      ) {
        const config = useAppConfig.getState();
-        const session = get().currentSession();
+        const session = targetSession;
        const modelConfig = session.mask.modelConfig;
        // skip summarize when using dalle3?
        if (isDalle3(modelConfig.model)) {
@@ -649,13 +654,15 @@ export const useChatStore = createPersistStore(
              stream: false,
              providerName,
            },
-            onFinish(message) {
-              if (!isValidMessage(message)) return;
-              get().updateCurrentSession(
-                (session) =>
-                  (session.topic =
-                    message.length > 0 ? trimTopic(message) : DEFAULT_TOPIC),
-              );
+            onFinish(message, responseRes) {
+              if (responseRes?.status === 200) {
+                get().updateTargetSession(
+                  session,
+                  (session) =>
+                    (session.topic =
+                      message.length > 0 ? trimTopic(message) : DEFAULT_TOPIC),
+                );
+              }
            },
          });
        }
@@ -669,7 +676,7 @@ export const useChatStore = createPersistStore(

        const historyMsgLength = countMessages(toBeSummarizedMsgs);

-        if (historyMsgLength > modelConfig?.max_tokens ?? 4000) {
+        if (historyMsgLength > (modelConfig?.max_tokens || 4000)) {
          const n = toBeSummarizedMsgs.length;
          toBeSummarizedMsgs = toBeSummarizedMsgs.slice(
            Math.max(0, n - modelConfig.historyMessageCount),
@@ -715,38 +722,38 @@ export const useChatStore = createPersistStore(
            onUpdate(message) {
              session.memoryPrompt = message;
            },
-            onFinish(message) {
-              console.log("[Memory] ", message);
-              get().updateCurrentSession((session) => {
-                session.lastSummarizeIndex = lastSummarizeIndex;
-                session.memoryPrompt = message; // Update the memory prompt for stored it in local storage
-              });
+            onFinish(message, responseRes) {
+              if (responseRes?.status === 200) {
+                console.log("[Memory] ", message);
+                get().updateTargetSession(session, (session) => {
+                  session.lastSummarizeIndex = lastSummarizeIndex;
+                  session.memoryPrompt = message; // Update the memory prompt for stored it in local storage
+                });
+              }
            },
            onError(err) {
              console.error("[Summarize] ", err);
            },
          });
        }
-
-        function isValidMessage(message: any): boolean {
-          return typeof message === "string" && !message.startsWith("```json");
-        }
      },

-      updateStat(message: ChatMessage) {
-        get().updateCurrentSession((session) => {
+      updateStat(message: ChatMessage, session: ChatSession) {
+        get().updateTargetSession(session, (session) => {
          session.stat.charCount += message.content.length;
          // TODO: should update chat count and word count
        });
      },
-
-      updateCurrentSession(updater: (session: ChatSession) => void) {
+      updateTargetSession(
+        targetSession: ChatSession,
+        updater: (session: ChatSession) => void,
+      ) {
        const sessions = get().sessions;
-        const index = get().currentSessionIndex;
+        const index = sessions.findIndex((s) => s.id === targetSession.id);
+        if (index < 0) return;
        updater(sessions[index]);
        set(() => ({ sessions }));
      },
-
      async clearAllData() {
        await indexedDBStorage.clear();
        localStorage.clear();
--- a/app/store/config.ts
+++ b/app/store/config.ts
@@ -15,6 +15,7 @@ import {
  ServiceProvider,
 } from "../constant";
 import { createPersistStore } from "../utils/store";
+import type { Voice } from "rt-client";

 export type ModelType = (typeof DEFAULT_MODELS)[number]["name"];
 export type TTSModelType = (typeof DEFAULT_TTS_MODELS)[number];
@@ -90,12 +91,26 @@ export const DEFAULT_CONFIG = {
    voice: DEFAULT_TTS_VOICE,
    speed: 1.0,
  },
+
+  realtimeConfig: {
+    enable: false,
+    provider: "OpenAI" as ServiceProvider,
+    model: "gpt-4o-realtime-preview-2024-10-01",
+    apiKey: "",
+    azure: {
+      endpoint: "",
+      deployment: "",
+    },
+    temperature: 0.9,
+    voice: "alloy" as Voice,
+  },
 };

 export type ChatConfig = typeof DEFAULT_CONFIG;

 export type ModelConfig = ChatConfig["modelConfig"];
 export type TTSConfig = ChatConfig["ttsConfig"];
+export type RealtimeConfig = ChatConfig["realtimeConfig"];

 export function limitNumber(
  x: number,
--- a/app/utils.ts
+++ b/app/utils.ts
@@ -254,6 +254,7 @@ export function getMessageImages(message: RequestMessage): string[] {
 export function isVisionModel(model: string) {
  // Note: This is a better way using the TypeScript feature instead of `&&` or `||` (ts v5.5.0-dev.20240314 I've been using)

+  const excludeKeywords = ["claude-3-5-haiku-20241022"];
  const visionKeywords = [
    "vision",
    "claude-3",
@@ -266,7 +267,10 @@ export function isVisionModel(model: string) {
    model.includes("gpt-4-turbo") && !model.includes("preview");

  return (
-    visionKeywords.some((keyword) => model.includes(keyword)) || isGpt4Turbo
+    !excludeKeywords.some((keyword) => model.includes(keyword)) &&
+    (visionKeywords.some((keyword) => model.includes(keyword)) ||
+      isGpt4Turbo ||
+      isDalle3(model))
  );
 }

@@ -278,7 +282,8 @@ export function showPlugins(provider: ServiceProvider, model: string) {
  if (
    provider == ServiceProvider.OpenAI ||
    provider == ServiceProvider.Azure ||
-    provider == ServiceProvider.Moonshot
+    provider == ServiceProvider.Moonshot ||
+    provider == ServiceProvider.ChatGLM
  ) {
    return true;
  }
--- a/app/utils/chat.ts
+++ b/app/utils/chat.ts
@@ -138,7 +138,7 @@ export function uploadImage(file: Blob): Promise<string> {
  })
    .then((res) => res.json())
    .then((res) => {
-      console.log("res", res);
+      // console.log("res", res);
      if (res?.code == 0 && res?.data) {
        return res?.data;
      }
@@ -174,6 +174,7 @@ export function stream(
  let finished = false;
  let running = false;
  let runTools: any[] = [];
+  let responseRes: Response;

  // animate response to make it looks smooth
  function animateResponseText() {
@@ -272,7 +273,7 @@ export function stream(
      }
      console.debug("[ChatAPI] end");
      finished = true;
-      options.onFinish(responseText + remainText);
+      options.onFinish(responseText + remainText, responseRes); // 将res传递给onFinish
    }
  };

@@ -304,6 +305,7 @@ export function stream(
        clearTimeout(requestTimeoutId);
        const contentType = res.headers.get("content-type");
        console.log("[Request] response content type: ", contentType);
+        responseRes = res;

        if (contentType?.startsWith("text/plain")) {
          responseText = await res.clone().text();
--- a/app/utils/model.ts
+++ b/app/utils/model.ts
@@ -37,6 +37,17 @@ const sortModelTable = (models: ReturnType<typeof collectModels>) =>
    }
  });

+/**
+ * get model name and provider from a formatted string,
+ * e.g. `gpt-4@OpenAi` or `claude-3-5-sonnet@20240620@Google`
+ * @param modelWithProvider model name with provider separated by last `@` char,
+ * @returns [model, provider] tuple, if no `@` char found, provider is undefined
+ */
+export function getModelProvider(modelWithProvider: string): [string, string?] {
+  const [model, provider] = modelWithProvider.split(/@(?!.*@)/);
+  return [model, provider];
+}
+
 export function collectModelTable(
  models: readonly LLMModel[],
  customModels: string,
@@ -79,10 +90,10 @@ export function collectModelTable(
        );
      } else {
        // 1. find model by name, and set available value
-        const [customModelName, customProviderName] = name.split("@");
+        const [customModelName, customProviderName] = getModelProvider(name);
        let count = 0;
        for (const fullName in modelTable) {
-          const [modelName, providerName] = fullName.split("@");
+          const [modelName, providerName] = getModelProvider(fullName);
          if (
            customModelName == modelName &&
            (customProviderName === undefined ||
@@ -102,7 +113,7 @@ export function collectModelTable(
        }
        // 2. if model not exists, create new model with available value
        if (count === 0) {
-          let [customModelName, customProviderName] = name.split("@");
+          let [customModelName, customProviderName] = getModelProvider(name);
          const provider = customProvider(
            customProviderName || customModelName,
          );
@@ -139,7 +150,7 @@ export function collectModelTableWithDefaultModel(
      for (const key of Object.keys(modelTable)) {
        if (
          modelTable[key].available &&
-          key.split("@").shift() == defaultModel
+          getModelProvider(key)[0] == defaultModel
        ) {
          modelTable[key].isDefault = true;
          break;
--- a/app/utils/stream.ts
+++ b/app/utils/stream.ts
@@ -19,7 +19,7 @@ type StreamResponse = {
  headers: Record<string, string>;
 };

-export function fetch(url: string, options?: RequestInit): Promise<any> {
+export function fetch(url: string, options?: RequestInit): Promise<Response> {
  if (window.__TAURI__) {
    const {
      signal,
@@ -100,7 +100,8 @@ export function fetch(url: string, options?: RequestInit): Promise<any> {
      })
      .catch((e) => {
        console.error("stream error", e);
-        throw e;
+        // throw e;
+        return new Response("", { status: 599 });
      });
  }
  return window.fetch(url, options);
--- a/docs/bt-cn.md
+++ b/docs/bt-cn.md
@@ -0,0 +1,29 @@
+# 宝塔面板 的部署说明
+
+## 拥有自己的宝塔
+当你需要通过 宝塔面板 部署本项目之前，需要在服务器上先安装好 宝塔面板工具。 接下来的 部署流程 都建立在已有宝塔面板的前提下。宝塔安装请参考 ([宝塔官网](https://www.bt.cn/new/download.html))
+
+> 注意：本项目需要宝塔面板版本 9.2.0 及以上
+
+## 一键安装
+![bt-install-1](./images/bt/bt-install-1.jpeg)
+1. 在 宝塔面板 -> Docker -> 应用商店 页面，搜索 ChatGPT-Next-Web 找到本项目的docker应用；
+2. 点击 安装 开始部署本项目
+
+![bt-install-2](./images/bt/bt-install-2.jpeg)
+1. 在项目配置页，根据要求开始配置环境变量；
+2. 如勾选 允许外部访问 配置，请注意为配置的 web端口 开放安全组端口访问权限；
+3. 请确保你添加了正确的 Open Api Key，否则无法使用；当配置 OpenAI官方 提供的key(国内无法访问)，请配置代理地址；
+4. 建议配置 访问权限密码，否则部署后所有人均可使用已配置的 Open Api Key(当允许外部访问时)；
+5. 点击 确认 开始自动部署。
+
+## 如何访问
+![bt-install-3](./images/bt/bt-install-3.jpeg)
+通过根据 服务器IP地址 和配置的 web端口 http://$(host):$(port)，在浏览器中打开 ChatGPT-Next-Web。
+
+![bt-install-4](./images/bt/bt-install-4.jpeg)
+若配置了 访问权限密码，访问大模型前需要登录，请点击 登录，获取访问权限。
+
+![bt-install-5](./images/bt/bt-install-5.jpeg)
+
+![bt-install-6](./images/bt/bt-install-6.jpeg)
--- a/docs/images/bt/bt-install-1.jpeg
+++ b/docs/images/bt/bt-install-1.jpeg
--- a/docs/images/bt/bt-install-2.jpeg
+++ b/docs/images/bt/bt-install-2.jpeg
--- a/docs/images/bt/bt-install-3.jpeg
+++ b/docs/images/bt/bt-install-3.jpeg
--- a/docs/images/bt/bt-install-4.jpeg
+++ b/docs/images/bt/bt-install-4.jpeg
--- a/docs/images/bt/bt-install-5.jpeg
+++ b/docs/images/bt/bt-install-5.jpeg
--- a/docs/images/bt/bt-install-6.jpeg
+++ b/docs/images/bt/bt-install-6.jpeg
--- a/jest.setup.ts
+++ b/jest.setup.ts
@@ -1,2 +1,24 @@
 // Learn more: https://github.com/testing-library/jest-dom
 import "@testing-library/jest-dom";
+
+global.fetch = jest.fn(() =>
+  Promise.resolve({
+    ok: true,
+    status: 200,
+    json: () => Promise.resolve({}),
+    headers: new Headers(),
+    redirected: false,
+    statusText: "OK",
+    type: "basic",
+    url: "",
+    clone: function () {
+      return this;
+    },
+    body: null,
+    bodyUsed: false,
+    arrayBuffer: () => Promise.resolve(new ArrayBuffer(0)),
+    blob: () => Promise.resolve(new Blob()),
+    formData: () => Promise.resolve(new FormData()),
+    text: () => Promise.resolve(""),
+  }),
+);
--- a/next.config.mjs
+++ b/next.config.mjs
@@ -94,8 +94,12 @@ if (mode !== "export") {
        source: "/sharegpt",
        destination: "https://sharegpt.com/api/conversations",
      },
+      {
+        source: "/api/proxy/alibaba/:path*",
+        destination: "https://dashscope.aliyuncs.com/api/:path*",
+      },
    ];
-
+    
    return {
      beforeFiles: ret,
    };
--- a/package.json
+++ b/package.json
@@ -27,14 +27,15 @@
    "@vercel/analytics": "^0.1.11",
    "@vercel/speed-insights": "^1.0.2",
    "axios": "^1.7.5",
+    "clsx": "^2.1.1",
    "emoji-picker-react": "^4.9.2",
    "fuse.js": "^7.0.0",
    "heic2any": "^0.0.4",
    "html-to-image": "^1.11.11",
    "idb-keyval": "^6.2.1",
    "lodash-es": "^4.17.21",
-    "mermaid": "^10.6.1",
    "markdown-to-txt": "^2.0.1",
+    "mermaid": "^10.6.1",
    "nanoid": "^5.0.3",
    "next": "^14.1.1",
    "node-fetch": "^3.3.1",
@@ -51,14 +52,16 @@
    "sass": "^1.59.2",
    "spark-md5": "^3.0.2",
    "use-debounce": "^9.0.4",
-    "zustand": "^4.3.8"
+    "zustand": "^4.3.8",
+    "rt-client": "https://github.com/Azure-Samples/aoai-realtime-audio-sdk/releases/download/js/v0.5.0/rt-client-0.5.0.tgz"
  },
  "devDependencies": {
    "@tauri-apps/api": "^1.6.0",
    "@tauri-apps/cli": "1.5.11",
-    "@testing-library/jest-dom": "^6.4.8",
-    "@testing-library/react": "^16.0.0",
-    "@types/jest": "^29.5.13",
+    "@testing-library/dom": "^10.4.0",
+    "@testing-library/jest-dom": "^6.6.2",
+    "@testing-library/react": "^16.0.1",
+    "@types/jest": "^29.5.14",
    "@types/js-yaml": "4.0.9",
    "@types/lodash-es": "^4.17.12",
    "@types/node": "^20.11.30",
--- a/public/audio-processor.js
+++ b/public/audio-processor.js
@@ -0,0 +1,48 @@
+// @ts-nocheck
+class AudioRecorderProcessor extends AudioWorkletProcessor {
+  constructor() {
+    super();
+    this.isRecording = false;
+    this.bufferSize = 2400; // 100ms at 24kHz
+    this.currentBuffer = [];
+
+    this.port.onmessage = (event) => {
+      if (event.data.command === "START_RECORDING") {
+        this.isRecording = true;
+      } else if (event.data.command === "STOP_RECORDING") {
+        this.isRecording = false;
+
+        if (this.currentBuffer.length > 0) {
+          this.sendBuffer();
+        }
+      }
+    };
+  }
+
+  sendBuffer() {
+    if (this.currentBuffer.length > 0) {
+      const audioData = new Float32Array(this.currentBuffer);
+      this.port.postMessage({
+        eventType: "audio",
+        audioData: audioData,
+      });
+      this.currentBuffer = [];
+    }
+  }
+
+  process(inputs) {
+    const input = inputs[0];
+    if (input.length > 0 && this.isRecording) {
+      const audioData = input[0];
+
+      this.currentBuffer.push(...audioData);
+
+      if (this.currentBuffer.length >= this.bufferSize) {
+        this.sendBuffer();
+      }
+    }
+    return true;
+  }
+}
+
+registerProcessor("audio-recorder-processor", AudioRecorderProcessor);
--- a/src-tauri/src/stream.rs
+++ b/src-tauri/src/stream.rs
@@ -119,11 +119,22 @@ pub async fn stream_fetch(
      }
    }
    Err(err) => {
-      println!("Error response: {:?}", err.source().expect("REASON").to_string());
+      let error: String = err.source()
+        .map(|e| e.to_string())
+        .unwrap_or_else(|| "Unknown error occurred".to_string());
+      println!("Error response: {:?}", error);
+      tauri::async_runtime::spawn( async move {
+        if let Err(e) = window.emit(event_name, ChunkPayload{ request_id, chunk: error.into() }) {
+          println!("Failed to emit chunk payload: {:?}", e);
+        }
+        if let Err(e) = window.emit(event_name, EndPayload{ request_id, status: 0 }) {
+          println!("Failed to emit end payload: {:?}", e);
+        }
+      });
      StreamResponse {
        request_id,
        status: 599,
-        status_text: err.source().expect("REASON").to_string(),
+        status_text: "Error".to_string(),
        headers: HashMap::new(),
      }
    }
--- a/src-tauri/tauri.conf.json
+++ b/src-tauri/tauri.conf.json
@@ -9,7 +9,7 @@
  },
  "package": {
    "productName": "NextChat",
-    "version": "2.15.6"
+    "version": "2.15.8"
  },
  "tauri": {
    "allowlist": {
--- a/test/model-provider.test.ts
+++ b/test/model-provider.test.ts
@@ -0,0 +1,31 @@
+import { getModelProvider } from "../app/utils/model";
+
+describe("getModelProvider", () => {
+  test("should return model and provider when input contains '@'", () => {
+    const input = "model@provider";
+    const [model, provider] = getModelProvider(input);
+    expect(model).toBe("model");
+    expect(provider).toBe("provider");
+  });
+
+  test("should return model and undefined provider when input does not contain '@'", () => {
+    const input = "model";
+    const [model, provider] = getModelProvider(input);
+    expect(model).toBe("model");
+    expect(provider).toBeUndefined();
+  });
+
+  test("should handle multiple '@' characters correctly", () => {
+    const input = "model@provider@extra";
+    const [model, provider] = getModelProvider(input);
+    expect(model).toBe("model@provider");
+    expect(provider).toBe("extra");
+  });
+
+  test("should return empty strings when input is empty", () => {
+    const input = "";
+    const [model, provider] = getModelProvider(input);
+    expect(model).toBe("");
+    expect(provider).toBeUndefined();
+  });
+});
--- a/yarn.lock
+++ b/yarn.lock
@@ -27,6 +27,15 @@
  dependencies:
    "@babel/highlight" "^7.18.6"

+"@babel/code-frame@^7.10.4":
+  version "7.26.0"
+  resolved "https://registry.npmmirror.com/@babel/code-frame/-/code-frame-7.26.0.tgz#9374b5cd068d128dac0b94ff482594273b1c2815"
+  integrity sha512-INCKxTtbXtcNbUZ3YXutwMpEleqttcswhAdee7dhuoVrD2cnuc3PqtERBtxkX5nziX9vnBL8WXmSGwv8CuPV6g==
+  dependencies:
+    "@babel/helper-validator-identifier" "^7.25.9"
+    js-tokens "^4.0.0"
+    picocolors "^1.0.0"
+
 "@babel/code-frame@^7.12.13", "@babel/code-frame@^7.24.7":
  version "7.24.7"
  resolved "https://registry.npmmirror.com/@babel/code-frame/-/code-frame-7.24.7.tgz#882fd9e09e8ee324e496bd040401c6f046ef4465"
@@ -394,6 +403,11 @@
  resolved "https://registry.npmmirror.com/@babel/helper-validator-identifier/-/helper-validator-identifier-7.24.7.tgz#75b889cfaf9e35c2aaf42cf0d72c8e91719251db"
  integrity sha512-rR+PBcQ1SMQDDyF6X0wxtG8QyLCgUB0eRAGguqRLfkCA87l7yAP7ehq8SNj96OOGTO8OBV70KhuFYcIkHXOg0w==

+"@babel/helper-validator-identifier@^7.25.9":
+  version "7.25.9"
+  resolved "https://registry.npmmirror.com/@babel/helper-validator-identifier/-/helper-validator-identifier-7.25.9.tgz#24b64e2c3ec7cd3b3c547729b8d16871f22cbdc7"
+  integrity sha512-Ed61U6XJc3CVRfkERJWDz4dJwKe7iLmmJsbOGu9wSloNSFttHV0I8g6UAgb7qnK5ly5bGLPd4oXZlxCdANBOWQ==
+
 "@babel/helper-validator-option@^7.18.6", "@babel/helper-validator-option@^7.21.0":
  version "7.21.0"
  resolved "https://registry.yarnpkg.com/@babel/helper-validator-option/-/helper-validator-option-7.21.0.tgz#8224c7e13ace4bafdc4004da2cf064ef42673180"
@@ -1187,14 +1201,7 @@
  resolved "https://registry.yarnpkg.com/@babel/regjsgen/-/regjsgen-0.8.0.tgz#f0ba69b075e1f05fb2825b7fad991e7adbb18310"
  integrity sha512-x/rqGMdzj+fWZvCOYForTghzbtqPDZ5gPwaoNGHdgDfF2QA/XZbCBp4Moo5scrkAMPhB7z26XM/AaHuIJdgauA==

-"@babel/runtime@^7.12.1", "@babel/runtime@^7.20.7", "@babel/runtime@^7.23.2", "@babel/runtime@^7.8.4", "@babel/runtime@^7.9.2":
-  version "7.23.6"
-  resolved "https://registry.yarnpkg.com/@babel/runtime/-/runtime-7.23.6.tgz#c05e610dc228855dc92ef1b53d07389ed8ab521d"
-  integrity sha512-zHd0eUrf5GZoOWVCXp6koAKQTfZV07eit6bGPmJgnZdnSAvvZee6zniW2XMF7Cmc4ISOOnPy3QaSiIJGJkVEDQ==
-  dependencies:
-    regenerator-runtime "^0.14.0"
-
-"@babel/runtime@^7.12.5", "@babel/runtime@^7.21.0":
+"@babel/runtime@^7.12.1", "@babel/runtime@^7.12.5", "@babel/runtime@^7.20.7", "@babel/runtime@^7.21.0", "@babel/runtime@^7.23.2", "@babel/runtime@^7.8.4", "@babel/runtime@^7.9.2":
  version "7.25.0"
  resolved "https://registry.yarnpkg.com/@babel/runtime/-/runtime-7.25.0.tgz#3af9a91c1b739c569d5d80cc917280919c544ecb"
  integrity sha512-7dRy4DwXwtzBrPbZflqxnvfxLF8kdZXPkhymtDeFoFqE6ldzjQFgYTtYIFARcLEYDrqfBfYcZt1WqFxRoyC9Rw==
@@ -2093,13 +2100,26 @@
    "@tauri-apps/cli-win32-ia32-msvc" "1.5.11"
    "@tauri-apps/cli-win32-x64-msvc" "1.5.11"

-"@testing-library/jest-dom@^6.4.8":
-  version "6.4.8"
-  resolved "https://registry.npmmirror.com/@testing-library/jest-dom/-/jest-dom-6.4.8.tgz#9c435742b20c6183d4e7034f2b329d562c079daa"
-  integrity sha512-JD0G+Zc38f5MBHA4NgxQMR5XtO5Jx9g86jqturNTt2WUfRmLDIY7iKkWHDCCTiDuFMre6nxAD5wHw9W5kI4rGw==
+"@testing-library/dom@^10.4.0":
+  version "10.4.0"
+  resolved "https://registry.npmmirror.com/@testing-library/dom/-/dom-10.4.0.tgz#82a9d9462f11d240ecadbf406607c6ceeeff43a8"
+  integrity sha512-pemlzrSESWbdAloYml3bAJMEfNh1Z7EduzqPKprCH5S341frlpYnUEW0H72dLxa6IsYr+mPno20GiSm+h9dEdQ==
+  dependencies:
+    "@babel/code-frame" "^7.10.4"
+    "@babel/runtime" "^7.12.5"
+    "@types/aria-query" "^5.0.1"
+    aria-query "5.3.0"
+    chalk "^4.1.0"
+    dom-accessibility-api "^0.5.9"
+    lz-string "^1.5.0"
+    pretty-format "^27.0.2"
+
+"@testing-library/jest-dom@^6.6.2":
+  version "6.6.2"
+  resolved "https://registry.yarnpkg.com/@testing-library/jest-dom/-/jest-dom-6.6.2.tgz#8186aa9a07263adef9cc5a59a4772db8c31f4a5b"
+  integrity sha512-P6GJD4yqc9jZLbe98j/EkyQDTPgqftohZF5FBkHY5BUERZmcf4HeO2k0XaefEg329ux2p21i1A1DmyQ1kKw2Jw==
  dependencies:
    "@adobe/css-tools" "^4.4.0"
-    "@babel/runtime" "^7.9.2"
    aria-query "^5.0.0"
    chalk "^3.0.0"
    css.escape "^1.5.1"
@@ -2107,10 +2127,10 @@
    lodash "^4.17.21"
    redent "^3.0.0"

-"@testing-library/react@^16.0.0":
-  version "16.0.0"
-  resolved "https://registry.npmmirror.com/@testing-library/react/-/react-16.0.0.tgz#0a1e0c7a3de25841c3591b8cb7fb0cf0c0a27321"
-  integrity sha512-guuxUKRWQ+FgNX0h0NS0FIq3Q3uLtWVpBzcLOggmfMoUpgBnzBzvLLd4fbm6yS8ydJd94cIfY4yP9qUQjM2KwQ==
+"@testing-library/react@^16.0.1":
+  version "16.0.1"
+  resolved "https://registry.yarnpkg.com/@testing-library/react/-/react-16.0.1.tgz#29c0ee878d672703f5e7579f239005e4e0faa875"
+  integrity sha512-dSmwJVtJXmku+iocRhWOUFbrERC76TX2Mnf0ATODz8brzAZrMBbzLwQixlBSanZxR6LddK3eiwpSFZgDET1URg==
  dependencies:
    "@babel/runtime" "^7.12.5"

@@ -2144,6 +2164,11 @@
  resolved "https://registry.npmmirror.com/@tsconfig/node16/-/node16-1.0.4.tgz#0b92dcc0cc1c81f6f306a381f28e31b1a56536e9"
  integrity sha512-vxhUy4J8lyeyinH7Azl1pdd43GJhZH/tP2weN8TntQblOY+A0XbT8DJk1/oCPuOOyg/Ja757rG0CgHcWC8OfMA==

+"@types/aria-query@^5.0.1":
+  version "5.0.4"
+  resolved "https://registry.npmmirror.com/@types/aria-query/-/aria-query-5.0.4.tgz#1a31c3d378850d2778dabb6374d036dcba4ba708"
+  integrity sha512-rfT93uj5s0PRL7EzccGMs3brplhcrghnDoV26NqKhCAS1hVo+WdNsPvE/yb6ilfr5hi2MEk6d5EWJTKdxg8jVw==
+
 "@types/babel__core@^7.1.14":
  version "7.20.5"
  resolved "https://registry.npmmirror.com/@types/babel__core/-/babel__core-7.20.5.tgz#3df15f27ba85319caa07ba08d0721889bb39c017"
@@ -2263,10 +2288,10 @@
  dependencies:
    "@types/istanbul-lib-report" "*"

-"@types/jest@^29.5.13":
-  version "29.5.13"
-  resolved "https://registry.yarnpkg.com/@types/jest/-/jest-29.5.13.tgz#8bc571659f401e6a719a7bf0dbcb8b78c71a8adc"
-  integrity sha512-wd+MVEZCHt23V0/L642O5APvspWply/rGY5BcW4SUETo2UzPU3Z26qr8jC2qxpimI2jjx9h7+2cj2FwIr01bXg==
+"@types/jest@^29.5.14":
+  version "29.5.14"
+  resolved "https://registry.yarnpkg.com/@types/jest/-/jest-29.5.14.tgz#2b910912fa1d6856cadcd0c1f95af7df1d6049e5"
+  integrity sha512-ZN+4sdnLUbo8EVvVc2ao0GFW6oVrQRPn4K2lglySj7APvSrgzxHiNNK99us4WDMi57xxA2yggblIAMNhXOotLQ==
  dependencies:
    expect "^29.0.0"
    pretty-format "^29.0.0"
@@ -2738,20 +2763,13 @@ argparse@^2.0.1:
  resolved "https://registry.yarnpkg.com/argparse/-/argparse-2.0.1.tgz#246f50f3ca78a3240f6c997e8a9bd1eac49e4b38"
  integrity sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q==

-aria-query@^5.0.0:
+aria-query@5.3.0, aria-query@^5.0.0, aria-query@^5.1.3:
  version "5.3.0"
  resolved "https://registry.npmmirror.com/aria-query/-/aria-query-5.3.0.tgz#650c569e41ad90b51b3d7df5e5eed1c7549c103e"
  integrity sha512-b0P0sZPKtyu8HkeRAfCq0IfURZK+SuwMjY1UXGBU27wpAiTwQAIlq56IbIO+ytk/JjS1fMR14ee5WBBfKi5J6A==
  dependencies:
    dequal "^2.0.3"

-aria-query@^5.1.3:
-  version "5.1.3"
-  resolved "https://registry.yarnpkg.com/aria-query/-/aria-query-5.1.3.tgz#19db27cd101152773631396f7a95a3b58c22c35e"
-  integrity sha512-R5iJ5lkuHybztUfuOAznmboyjWq8O6sqNqtK7CLOqdydi54VNbORp49mb14KbWgG1QD3JFO9hJdZ+y4KutfdOQ==
-  dependencies:
-    deep-equal "^2.0.5"
-
 array-buffer-byte-length@^1.0.0:
  version "1.0.0"
  resolved "https://registry.yarnpkg.com/array-buffer-byte-length/-/array-buffer-byte-length-1.0.0.tgz#fabe8bc193fea865f317fe7807085ee0dee5aead"
@@ -3081,7 +3099,7 @@ chalk@^3.0.0:
    ansi-styles "^4.1.0"
    supports-color "^7.1.0"

-chalk@^4.0.0, chalk@^4.1.2:
+chalk@^4.0.0, chalk@^4.1.0, chalk@^4.1.2:
  version "4.1.2"
  resolved "https://registry.yarnpkg.com/chalk/-/chalk-4.1.2.tgz#aac4e2b7734a740867aeb16bf02aad556a1e7a01"
  integrity sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==
@@ -3171,6 +3189,11 @@ cliui@^8.0.1:
    strip-ansi "^6.0.1"
    wrap-ansi "^7.0.0"

+clsx@^2.1.1:
+  version "2.1.1"
+  resolved "https://registry.yarnpkg.com/clsx/-/clsx-2.1.1.tgz#eed397c9fd8bd882bfb18deab7102049a2f32999"
+  integrity sha512-eYm0QWBtUrBWZWG0d386OGAw16Z995PiOVo2B7bjWSbHedGl5e0ZWaq65kOGgUSNesEIDkB9ISbTg/JK9dhCZA==
+
 co@^4.6.0:
  version "4.6.0"
  resolved "https://registry.npmmirror.com/co/-/co-4.6.0.tgz#6ea6bdf3d853ae54ccb8e47bfa0bf3f9031fb184"
@@ -3877,6 +3900,11 @@ doctrine@^3.0.0:
  dependencies:
    esutils "^2.0.2"

+dom-accessibility-api@^0.5.9:
+  version "0.5.16"
+  resolved "https://registry.npmmirror.com/dom-accessibility-api/-/dom-accessibility-api-0.5.16.tgz#5a7429e6066eb3664d911e33fb0e45de8eb08453"
+  integrity sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg==
+
 dom-accessibility-api@^0.6.3:
  version "0.6.3"
  resolved "https://registry.npmmirror.com/dom-accessibility-api/-/dom-accessibility-api-0.6.3.tgz#993e925cc1d73f2c662e7d75dd5a5445259a8fd8"
@@ -6052,6 +6080,11 @@ lru-cache@^6.0.0:
  dependencies:
    yallist "^4.0.0"

+lz-string@^1.5.0:
+  version "1.5.0"
+  resolved "https://registry.npmmirror.com/lz-string/-/lz-string-1.5.0.tgz#c1ab50f77887b712621201ba9fd4e3a6ed099941"
+  integrity sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ==
+
 make-dir@^4.0.0:
  version "4.0.0"
  resolved "https://registry.npmmirror.com/make-dir/-/make-dir-4.0.0.tgz#c3c2307a771277cd9638305f915c29ae741b614e"
@@ -7018,6 +7051,15 @@ prettier@^3.0.2:
  resolved "https://registry.yarnpkg.com/prettier/-/prettier-3.0.2.tgz#78fcecd6d870551aa5547437cdae39d4701dca5b"
  integrity sha512-o2YR9qtniXvwEZlOKbveKfDQVyqxbEIWn48Z8m3ZJjBjcCmUy3xZGIv+7AkaeuaTr6yPXJjwv07ZWlsWbEy1rQ==

+pretty-format@^27.0.2:
+  version "27.5.1"
+  resolved "https://registry.npmmirror.com/pretty-format/-/pretty-format-27.5.1.tgz#2181879fdea51a7a5851fb39d920faa63f01d88e"
+  integrity sha512-Qb1gy5OrP5+zDf2Bvnzdl3jsTf1qXVMazbvCoKhtKqVs4/YK4ozX4gKQJJVyNe+cajNPn0KoC0MC3FUmaHWEmQ==
+  dependencies:
+    ansi-regex "^5.0.1"
+    ansi-styles "^5.0.0"
+    react-is "^17.0.1"
+
 pretty-format@^29.0.0, pretty-format@^29.7.0:
  version "29.7.0"
  resolved "https://registry.npmmirror.com/pretty-format/-/pretty-format-29.7.0.tgz#ca42c758310f365bfa71a0bda0a807160b776812"
@@ -7109,6 +7151,11 @@ react-is@^16.13.1, react-is@^16.7.0:
  resolved "https://registry.yarnpkg.com/react-is/-/react-is-16.13.1.tgz#789729a4dc36de2999dc156dd6c1d9c18cea56a4"
  integrity sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ==

+react-is@^17.0.1:
+  version "17.0.2"
+  resolved "https://registry.npmmirror.com/react-is/-/react-is-17.0.2.tgz#e691d4a8e9c789365655539ab372762b0efb54f0"
+  integrity sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w==
+
 react-is@^18.0.0:
  version "18.2.0"
  resolved "https://registry.yarnpkg.com/react-is/-/react-is-18.2.0.tgz#199431eeaaa2e09f86427efbb4f1473edb47609b"
@@ -7408,6 +7455,12 @@ robust-predicates@^3.0.0:
  resolved "https://registry.npmmirror.com/robust-predicates/-/robust-predicates-3.0.1.tgz#ecde075044f7f30118682bd9fb3f123109577f9a"
  integrity sha512-ndEIpszUHiG4HtDsQLeIuMvRsDnn8c8rYStabochtUeCvfuvNptb5TUbVD68LRAILPX7p9nqQGh4xJgn3EHS/g==

+"rt-client@https://github.com/Azure-Samples/aoai-realtime-audio-sdk/releases/download/js/v0.5.0/rt-client-0.5.0.tgz":
+  version "0.5.0"
+  resolved "https://github.com/Azure-Samples/aoai-realtime-audio-sdk/releases/download/js/v0.5.0/rt-client-0.5.0.tgz#abf2e9a850201e3571b8d36830f77bc52af3de9b"
+  dependencies:
+    ws "^8.18.0"
+
 run-parallel@^1.1.9:
  version "1.2.0"
  resolved "https://registry.yarnpkg.com/run-parallel/-/run-parallel-1.2.0.tgz#66d1368da7bdf921eb9d95bd1a9229e7f21a43ee"
@@ -8451,9 +8504,9 @@ write-file-atomic@^4.0.2:
    imurmurhash "^0.1.4"
    signal-exit "^3.0.7"

-ws@^8.11.0:
+ws@^8.11.0, ws@^8.18.0:
  version "8.18.0"
-  resolved "https://registry.npmmirror.com/ws/-/ws-8.18.0.tgz#0d7505a6eafe2b0e712d232b42279f53bc289bbc"
+  resolved "https://registry.yarnpkg.com/ws/-/ws-8.18.0.tgz#0d7505a6eafe2b0e712d232b42279f53bc289bbc"
  integrity sha512-8VbfWfHLbbwu3+N6OKsOMpBdT4kXPDDB9cJk2bJ6mh9ucxdlnNvH1e+roYkKmN9Nxw2yjz7VzeO9oOz2zJ04Pw==

 xml-name-validator@^4.0.0:
Author	SHA1	Message	Date
lloydzhou	009447a188	Merge remote-tracking branch 'origin/main' into website	2024-11-11 13:26:21 +08:00
lloydzhou	38fa3056df	update version	2024-11-11 13:26:08 +08:00
Lloyd Zhou	289aeec8af	Merge pull request #5786 from ConnectAI-E/feature/realtime-chat Feature/realtime chat	2024-11-11 13:19:26 +08:00
lloydzhou	7d71da938f	remove close-24 svg	2024-11-11 13:15:09 +08:00
Lloyd Zhou	f8f6954115	Merge pull request #5779 from ConnectAI-E/feature/model/claude35haiku add claude35haiku & not support vision	2024-11-11 13:13:09 +08:00
Lloyd Zhou	6e03f32871	Merge pull request #5795 from JingSyue/main fix: built-in plugin dalle3 error #5787	2024-11-11 13:10:00 +08:00
JingSyue	18a6571883	Update proxy.ts Update proxy.ts	2024-11-11 12:59:29 +08:00
Dogtiti	14f444e1f0	doc: realtime chat	2024-11-11 11:47:41 +08:00
JingSyue	2b0f2e5f9d	fix: built-in plugin dalle3 error #5787	2024-11-10 10:28:25 +08:00
Dogtiti	4629b39c29	chore: comment context history	2024-11-09 16:22:01 +08:00
Dogtiti	d33e772fa5	feat: voice print	2024-11-08 22:39:17 +08:00
Dogtiti	89136fba32	feat: voice print	2024-11-08 22:18:39 +08:00
Dogtiti	8b4ca133fd	feat: voice print	2024-11-08 22:02:31 +08:00
lloydzhou	a4c9eaf6cd	do not save empty audio file	2024-11-08 13:43:13 +08:00
lloydzhou	50e63109a3	merge code and get analyser data	2024-11-08 13:21:40 +08:00
Dogtiti	48a1e8a584	chore: i18n	2024-11-07 21:32:47 +08:00
Dogtiti	e44ebe3f0e	feat: realtime config	2024-11-07 21:28:23 +08:00
Lloyd Zhou	108069a0c6	Merge pull request #5788 from ConnectAI-E/fix-o1-maxtokens chore: o1模型使用max_completion_tokens	2024-11-07 20:06:30 +08:00
DDMeaqua	d5bda2904d	chore: o1模型使用max_completion_tokens	2024-11-07 19:45:27 +08:00
lloydzhou	283caba8ce	stop streaming play after get input audio.	2024-11-07 18:57:57 +08:00
lloydzhou	b78e5db817	add temperature config	2024-11-07 17:55:51 +08:00
lloydzhou	46c469b2d7	add voice config	2024-11-07 17:47:55 +08:00
lloydzhou	c00ebbea4f	update	2024-11-07 17:40:03 +08:00
lloydzhou	c526ff80b5	update	2024-11-07 17:23:20 +08:00
lloydzhou	0037b0c944	ts error	2024-11-07 17:03:04 +08:00
lloydzhou	6f81bb3b8a	add context after connected	2024-11-07 16:56:15 +08:00
lloydzhou	7bdc45ed3e	connect realtime model when open panel	2024-11-07 16:41:24 +08:00
Dogtiti	88cd3ac122	fix: ts error	2024-11-07 12:16:11 +08:00
Dogtiti	4988d2ee26	fix: ts error	2024-11-07 11:56:58 +08:00
lloydzhou	8deb7a92ee	hotfix for update target session	2024-11-07 11:53:01 +08:00
lloydzhou	db060d732a	upload save record wav file	2024-11-07 11:45:38 +08:00
lloydzhou	522627820a	upload save wav file logic	2024-11-07 09:36:22 +08:00
lloydzhou	cf46d5ad63	upload response audio, and update audio_url to session message	2024-11-07 01:12:08 +08:00
Dogtiti	a4941521d0	feat: audio to message	2024-11-06 22:30:02 +08:00
Dogtiti	f6e1f8398b	wip	2024-11-06 22:07:33 +08:00
Dogtiti	d544eead38	feat: realtime chat ui	2024-11-06 21:14:45 +08:00
Lloyd Zhou	fbb9385f23	Merge pull request #5782 from ConnectAI-E/style/classname style: improve classname by clsx	2024-11-06 20:33:51 +08:00
Dogtiti	18144c3d9c	chore: clsx	2024-11-06 20:16:38 +08:00
opchips	64aa760e58	update claude rank	2024-11-06 19:18:05 +08:00
Dogtiti	e0bbb8bb68	style: improve classname by clsx	2024-11-06 16:58:26 +08:00
opchips	6667ee1c7f	merge main	2024-11-06 15:08:18 +08:00
Lloyd Zhou	6ded4e96e7	Merge pull request #5778 from ConnectAI-E/fix/5436 fix: botMessage reply date	2024-11-06 15:04:46 +08:00
Dogtiti	85cdcab850	fix: botMessage reply date	2024-11-06 14:53:08 +08:00
Lloyd Zhou	f4c9410c29	Merge pull request #5776 from ConnectAI-E/feat-glm fix: glm chatpath	2024-11-06 14:02:20 +08:00
DDMeaqua	adf7d8200b	fix: glm chatpath	2024-11-06 13:55:57 +08:00
opchips	3086a2fa77	add claude35haiku not vision	2024-11-06 12:56:24 +08:00
Lloyd Zhou	f526d6f560	Merge pull request #5774 from ConnectAI-E/feature/update-target-session fix: updateCurrentSession => updateTargetSession	2024-11-06 11:16:33 +08:00
Dogtiti	106461a1e7	Merge branch 'main' of https://github.com/ConnectAI-E/ChatGPT-Next-Web into feature/update-target-session	2024-11-06 11:08:41 +08:00
Dogtiti	c4e19dbc59	fix: updateCurrentSession => updateTargetSession	2024-11-06 11:06:18 +08:00
Dogtiti	f3603e59fa	Merge pull request #5769 from ryanhex53/fix-model-multi@ Custom model names can include the `@` symbol by itself.	2024-11-06 10:49:28 +08:00
ryanhex53	8e2484fcdf	Refactor: Replace all provider split occurrences with getModelProvider utility method	2024-11-05 13:52:54 +00:00
lloydzhou	d7270a5840	Merge remote-tracking branch 'origin/main' into website	2024-11-05 17:47:01 +08:00
lloydzhou	00d6cb27f7	update version	2024-11-05 17:42:55 +08:00
ryanhex53	b844045d23	Custom model names can include the `@` symbol by itself. To specify the model's provider, append it after the model name using `@` as before. This format supports cases like `google vertex ai` with a model name like `claude-3-5-sonnet@20240620`. For instance, `claude-3-5-sonnet@20240620@vertex-ai` will be split by `split(/@(?!.*@)/)` into: `[ 'claude-3-5-sonnet@20240620', 'vertex-ai' ]`, where the former is the model name and the latter is the custom provider.	2024-11-05 07:44:12 +00:00
Lloyd Zhou	e49fe976d9	Merge pull request #5765 from ConnectAI-E/feature/onfinish feat: update real 'currentSession'	2024-11-05 15:07:52 +08:00
Dogtiti	14f751965f	Merge pull request #5767 from ConnectAI-E/feat-glm chore: update readme	2024-11-05 11:07:52 +08:00
DDMeaqua	0ec423389f	chore: update readme	2024-11-05 11:06:20 +08:00
Dogtiti	820ab54e2d	Merge pull request #5766 from ConnectAI-E/feature/add-claude-haiku3.5 Feature/add claude haiku3.5	2024-11-05 10:54:52 +08:00
lloydzhou	a6c1eb27a8	add claude 3.5 haiku	2024-11-05 10:23:15 +08:00
Lloyd Zhou	0dc4071ccc	Merge pull request #5464 from endless-learner/main Added 1-click deployment link for Alibaba Cloud.	2024-11-05 01:10:06 +08:00
Lloyd Zhou	4d3949718a	merge main	2024-11-05 01:09:27 +08:00
Dogtiti	aef535f1a7	Merge pull request #5753 from ChatGPTNextWeb/feat-bt-doc Feat bt doc	2024-11-04 21:41:11 +08:00
Dogtiti	686a80e727	Merge pull request #5764 from ChatGPTNextWeb/dependabot/npm_and_yarn/testing-library/react-16.0.1 chore(deps-dev): bump @testing-library/react from 16.0.0 to 16.0.1	2024-11-04 21:37:34 +08:00
Dogtiti	e49466fa05	feat: update real 'currentSession'	2024-11-04 21:25:56 +08:00
dependabot[bot]	4b93370814	chore(deps-dev): bump @testing-library/react from 16.0.0 to 16.0.1 Bumps [@testing-library/react](https://github.com/testing-library/react-testing-library) from 16.0.0 to 16.0.1. - [Release notes](https://github.com/testing-library/react-testing-library/releases) - [Changelog](https://github.com/testing-library/react-testing-library/blob/main/CHANGELOG.md) - [Commits](https://github.com/testing-library/react-testing-library/compare/v16.0.0...v16.0.1) --- updated-dependencies: - dependency-name: "@testing-library/react" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2024-11-04 10:24:30 +00:00
Dogtiti	5733e3c588	Merge pull request #5759 from ConnectAI-E/feature/onfinish Feature/onfinish	2024-11-04 17:16:44 +08:00
Dogtiti	44fc5b5cbf	fix: onfinish responseRes	2024-11-04 17:00:45 +08:00
Dogtiti	2d3f7c922f	fix: vision model dalle3	2024-11-04 15:51:04 +08:00
GH Action - Upstream Sync	fe8cca3730	Merge branch 'main' of https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web	2024-11-02 01:12:09 +00:00
weige	fbb7a1e853	fix	2024-11-01 18:20:16 +08:00
weige	fb2c15567d	fix	2024-11-01 17:45:50 +08:00
weige	c2c52a1f60	fix	2024-11-01 17:35:34 +08:00
weige	106ddc17cd	fix	2024-11-01 17:35:09 +08:00
weige	17d5209738	add bt install doc	2024-11-01 17:28:20 +08:00
Dogtiti	d66bfc6352	Merge pull request #5752 from ConnectAI-E/feat-glm fix: ts error	2024-11-01 14:16:50 +08:00
DDMeaqua	4d75b23ed1	fix: ts error	2024-11-01 14:15:12 +08:00
Dogtiti	36bfa2ef7c	Merge pull request #5741 from ConnectAI-E/feat-glm feat: [#5714] 支持GLM	2024-11-01 13:57:30 +08:00
DDMeaqua	afe12c212e	chore: update	2024-11-01 13:53:43 +08:00
GH Action - Upstream Sync	adf97c6d8b	Merge branch 'main' of https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web	2024-11-01 01:18:59 +00:00
DDMeaqua	7a8d557ea3	chore: 开启插件	2024-10-31 11:37:19 +08:00
DDMeaqua	d3f0a77830	chore: update Provider	2024-10-31 11:23:06 +08:00
Dogtiti	0581e37236	Merge pull request #5744 from mrcore/main add claude-3-5-sonnet-latest and claude-3-opus-latest	2024-10-31 11:19:34 +08:00
Core	44383a8b33	add claude-3-5-sonnet-latest and claude-3-opus-latest add claude-3-5-sonnet-latest and claude-3-opus-latest	2024-10-31 11:00:45 +08:00
GH Action - Upstream Sync	7c466c9b9c	Merge branch 'main' of https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web	2024-10-31 01:14:28 +00:00
Dogtiti	a0fa4d7e72	Merge pull request #5737 from hyiip/claude3.5 add constant to claude 3.5 sonnet 20241022	2024-10-31 00:13:16 +08:00
DDMeaqua	d357b45e84	feat: [#5714 ] 支持GLM	2024-10-30 19:24:03 +08:00
Lloyd Zhou	d0bd1bf8fd	Merge pull request #5740 from yuxuan-ctrl/main feat: 新增阿里系模型代码配置	2024-10-30 16:56:53 +08:00
yuxuan-ctrl	86ffa1e643	feat: 新增阿里系模型代码配置	2024-10-30 16:30:01 +08:00
endless-learner	b0d28eb77e	Merge branch 'main' into main	2024-10-29 14:38:49 -07:00
hyiip	736cbdbdd1	add constant to claude 3.5 sonnet 20241022	2024-10-30 02:18:41 +08:00
Dogtiti	613d67eada	Merge pull request #5729 from ConnectAI-E/feature/jest chore: improve jest	2024-10-29 19:39:59 +08:00
Dogtiti	89cea18955	Merge branch 'main' of https://github.com/ConnectAI-E/ChatGPT-Next-Web into feature/jest	2024-10-29 19:26:52 +08:00
Dogtiti	56bc77d20b	Merge pull request #5731 from ChatGPTNextWeb/dependabot/npm_and_yarn/testing-library/jest-dom-6.6.2 Bump @testing-library/jest-dom from 6.4.8 to 6.6.2	2024-10-28 21:52:08 +08:00
Dogtiti	6d93d37963	Merge pull request #5732 from ChatGPTNextWeb/dependabot/npm_and_yarn/types/jest-29.5.14 Bump @types/jest from 29.5.13 to 29.5.14	2024-10-28 21:51:59 +08:00
dependabot[bot]	24df85cf9d	Bump @types/jest from 29.5.13 to 29.5.14 Bumps [@types/jest](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jest) from 29.5.13 to 29.5.14. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jest) --- updated-dependencies: - dependency-name: "@types/jest" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2024-10-28 10:31:34 +00:00
dependabot[bot]	a4d7a2c6e3	Bump @testing-library/jest-dom from 6.4.8 to 6.6.2 Bumps [@testing-library/jest-dom](https://github.com/testing-library/jest-dom) from 6.4.8 to 6.6.2. - [Release notes](https://github.com/testing-library/jest-dom/releases) - [Changelog](https://github.com/testing-library/jest-dom/blob/main/CHANGELOG.md) - [Commits](https://github.com/testing-library/jest-dom/compare/v6.4.8...v6.6.2) --- updated-dependencies: - dependency-name: "@testing-library/jest-dom" dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>	2024-10-28 10:31:27 +00:00
Dogtiti	49d42bb45d	chore: improve jest	2024-10-28 16:47:05 +08:00
Lloyd Zhou	4f49626303	Merge pull request #5722 from ElricLiu/main Update README.md	2024-10-26 12:09:09 +08:00
ElricLiu	45db20c1c3	Update README.md	2024-10-26 11:16:43 +08:00
Lloyd Zhou	82994843f5	Merge pull request #5719 from ConnectAI-E/hotfix/status_text_error hotfix for statusText is non ISO-8859-1 #5717	2024-10-25 20:34:15 +08:00
Dogtiti	1110a087a0	Merge pull request #5720 from ConnectAI-E/hotfix/gemini_invald_argument hotfix for gemini invald argument #5715	2024-10-25 18:25:46 +08:00
lloydzhou	f0b3e10a6c	hotfix for gemini invald argument #5715	2024-10-25 18:19:22 +08:00
lloydzhou	f89872b833	hotfix for gemini invald argument #5715	2024-10-25 18:12:09 +08:00
lloydzhou	90ced92876	update	2024-10-25 18:05:29 +08:00
lloydzhou	2c74559010	hitfix	2024-10-25 18:02:51 +08:00
lloydzhou	e3ca7e8b44	hotfix for statusText is non ISO-8859-1 #5717	2024-10-25 17:52:08 +08:00
lloydzhou	a0c78b8260	Merge branch 'main' into website	2024-10-24 16:36:12 +08:00
lloydzhou	801dc412f9	add claude-3.5-haiku	2024-10-24 15:28:05 +08:00
lloydzhou	143fdf4c9d	Merge branch 'main' into website	2024-10-16 00:46:25 +08:00
lloydzhou	37e48b431d	Merge remote-tracking branch 'origin/main' into website	2024-10-15 17:29:42 +08:00
lloydzhou	eed651ddde	Merge branch 'main' into website	2024-10-10 12:56:59 +08:00
lloydzhou	22fa98d73d	Merge branch 'main' into website	2024-10-09 19:34:01 +08:00
lloydzhou	7d3b5f1ac8	Merge remote-tracking branch 'origin/main' into website	2024-09-26 13:57:05 +08:00
lloydzhou	1df5a16141	Merge remote-tracking branch 'origin/main' into website	2024-09-25 15:54:41 +08:00
endless-learner	064e964d75	Updated link to deploy on Alibaba Cloud, readable when not logged in, also, able to choose region.	2024-09-24 23:05:32 -07:00
endless-learner	47fb40d572	Merge branch 'ChatGPTNextWeb:main' into main	2024-09-24 23:03:03 -07:00
endless-learner	9e18cc260b	Update README.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>	2024-09-24 13:55:00 -07:00
lloydzhou	2b944b154c	hotfix openai function call tool_calls no index	2024-09-22 19:09:28 +08:00
lloydzhou	9b7cb795ca	hotfix openai function call tool_calls no index	2024-09-22 19:09:18 +08:00
endless-learner	03268ce4d8	Added 1-click deployment link for Alibaba Cloud.	2024-09-18 20:38:20 -07:00
lloydzhou	20ae4f54e6	Merge remote-tracking branch 'origin/main' into website	2024-09-13 17:41:44 +08:00
lloydzhou	78facb282b	Merge remote-tracking branch 'origin/main' into website	2024-09-07 22:13:20 +08:00
lloydzhou	6f3d7530b9	Merge remote-tracking branch 'origin/main' into website	2024-09-06 20:18:21 +08:00
lloydzhou	5e1064a5c8	Merge branch 'main' into website	2024-08-16 16:58:30 +08:00
lloydzhou	faac0d9817	Merge remote-tracking branch 'origin/main' into website	2024-08-06 22:45:16 +08:00
lloydzhou	c440637ad0	Merge remote-tracking branch 'origin/main' into website	2024-07-27 01:32:47 +08:00
lloydzhou	284d33bcdf	Merge remote-tracking branch 'origin/main' into website	2024-07-19 18:37:32 +08:00
lloydzhou	d9573973ca	Merge remote-tracking branch 'origin/main' into website	2024-07-13 21:31:15 +08:00
fred-bf	cd354cf045	Merge pull request #4685 from ChatGPTNextWeb/main feat: update upstream	2024-05-14 17:40:46 +08:00
fred-bf	1cce87acaa	Merge pull request #4181 from ChatGPTNextWeb/main merge main	2024-03-01 11:10:11 +08:00
fred-bf	78c4084501	Merge pull request #4148 from ChatGPTNextWeb/main feat: catch up latest commit	2024-02-27 10:43:15 +08:00
Fred Liang	1d0a40b9e8	chore: low the google safety setting to avoid unexpected blocking	2023-12-31 19:50:06 +08:00