ChatGLM2-6B 在线聊天 - 使用 Gradio 构建网页应用

ChatGLM2-6B 在线聊天/n/n本网页使用 ChatGLM2-6B 模型构建了简单的在线聊天应用。你可以输入文字，与模型进行对话。/n/n### 功能介绍/n/n* 输入文字: 在文本框中输入你的问题或想法，点击“Submit”按钮发送。/n* 参数调整: 你可以调节以下参数来控制模型的输出行为：/n * Maximum length: 最大生成长度/n * Top P: 控制生成结果的概率分布/n * Temperature: 控制生成结果的随机性/n* 清空历史: 点击“Clear History”按钮清空聊天记录。/n/n### 代码示例/n/npython/nfrom transformers import AutoModel, AutoTokenizer/nimport gradio as gr/nimport mdtex2html/nfrom utils import load_model_on_gpus/n/ntokenizer = AutoTokenizer.from_pretrained('D://work//ChatGLM2-6B//THUDM//chatglm2-6b', trust_remote_code=True)/nmodel = AutoModel.from_pretrained('D://work//ChatGLM2-6B//THUDM//chatglm2-6b', trust_remote_code=True).cuda()/n# 多显卡支持，使用下面两行代替上面一行，将num_gpus改为你实际的显卡数量/n# from utils import load_model_on_gpus/n# model = load_model_on_gpus('THUDM/chatglm2-6b', num_gpus=2)/nmodel = model.eval()/n/n'Override Chatbot.postprocess'/n/ndef postprocess(self, y):/n if y is None:/n return []/n for i, (message, response) in enumerate(y):/n y[i] = (/n None if message is None else mdtex2html.convert((message)),/n None if response is None else mdtex2html.convert(response),/n )/n return y/n/ngr.Chatbot.postprocess = postprocess/n/n/ndef parse_text(text):/n 'copy from https://github.com/GaiZhenbiao/ChuanhuChatGPT/'/n lines = text.split('/n')/n lines = [line for line in lines if line != '']/n count = 0/n for i, line in enumerate(lines):/n if '' in line:/n count += 1/n items = line.split('')/n if count % 2 == 1:/n lines[i] = f'<pre><code class='language-{items[-1]}'>' /n else:/n lines[i] = f'<br></code></pre>'/n else:/n if i > 0:/n if count % 2 == 1:/n line = line.replace('', '/')/n line = line.replace('<', '&lt;')/n line = line.replace('>', '&gt;')/n line = line.replace(' ', '&nbsp;')/n line = line.replace('*', '&ast;')/n line = line.replace('_', '&lowbar;')/n line = line.replace('-', '&#45;')/n line = line.replace('.', '&#46;')/n line = line.replace('!', '&#33;')/n line = line.replace('(', '&#40;')/n line = line.replace(')', '&#41;')/n line = line.replace('$', '&#36;')/n lines[i] = '<br>'+line/n text = ''.join(lines)/n return text/n/n/ndef predict(input, chatbot, max_length, top_p, temperature, history, past_key_values):/n chatbot.append((parse_text(input), ''))/n for response, history, past_key_values in model.stream_chat(tokenizer, input, history, past_key_values=past_key_values,/n return_past_key_values=True,/n max_length=max_length, top_p=top_p,/n temperature=temperature):/n chatbot[-1] = (parse_text(input), parse_text(response))/n/n yield chatbot, history, past_key_values/n/n/ndef reset_user_input():/n return gr.update(value='')/n/n/ndef reset_state():/n return [], [], None/n/n/nwith gr.Blocks() as demo:/n gr.HTML('''<h1 align='center'>ChatGLM2-6B</h1>''')/n/n chatbot = gr.Chatbot()/n with gr.Row():/n with gr.Column(scale=4):/n with gr.Column(scale=12):/n user_input = gr.Textbox(show_label=False, placeholder='Input...', lines=10).style(/n container=False)/n with gr.Column(min_width=32, scale=1):/n submitBtn = gr.Button('Submit', variant='primary')/n with gr.Column(scale=1):/n emptyBtn = gr.Button('Clear History')/n max_length = gr.Slider(0, 32768, value=8192, step=1.0, label='Maximum length', interactive=True)/n top_p = gr.Slider(0, 1, value=0.8, step=0.01, label='Top P', interactive=True)/n temperature = gr.Slider(0, 1, value=0.95, step=0.01, label='Temperature', interactive=True)/n/n history = gr.State([])/n past_key_values = gr.State(None)/n/n submitBtn.click(predict, [user_input, chatbot, max_length, top_p, temperature, history, past_key_values],/n [chatbot, history, past_key_values], show_progress=True)/n submitBtn.click(reset_user_input, [], [user_input])/n/n emptyBtn.click(reset_state, outputs=[chatbot, history, past_key_values], show_progress=True)/n/ndemo.queue().launch(share=False, inbrowser=True)/n```/n/n**注意:**/n/n* 为了运行这段代码，你需要安装 transformers, gradio, mdtex2html和sentencepiece库。/n* 如果你遇到错误“DLL load failed while importing _sentencepiece: 找不到指定的模块”，请确保你已经安装了sentencepiece库。/n* 如果你使用的是多个 GPU，请修改代码中的num_gpus参数。/n* 该代码示例仅供参考，你需要根据自己的需求进行修改。/n/n**如何使用:**/n/n1. 将代码保存为.py文件，例如chat_demo.py。/n2. 运行命令 python chat_demo.py`。/n3. 在浏览器中打开生成的链接，即可使用在线聊天应用。/n/n希望这对你有所帮助！