操作流程
- 1、下载安装
- 2、配置
- 3、测试
- 官网：https://www.modelscope.cn/models
下载安装
注意：越狱版
创建虚拟环境
## 查看效果：conda info --envsconda create--nameDeepSeek-R1-Distill-Qwen-7B-abliterated-v2python=3.12.7-y# 激活环境conda activate DeepSeek-R1-Distill-Qwen-7B-abliterated-v2# 退出环境# 当前的conda虚拟环境里，只需要执行conda deactivate# conda deactivate DeepSeek-R1-Distill-Qwen-7B-abliterated-v2# 删除环境# conda remove -n DeepSeek-R1-Distill-Qwen-7B-abliterated-v2 --all -y- 下载模型：DeepSeek-R1-Distill-Qwen-7B-abliterated-v2
cd/usr/local/softwaremkdir-p/mnt/D/modelscope/DeepSeek-R1-Distill-Qwen-7B-abliterated-v2chmod0777-R/mnt/D/modelscope/DeepSeek-R1-Distill-Qwen-7B-abliterated-v2chown$USER:$USER-R/mnt/D/modelscope/DeepSeek-R1-Distill-Qwen-7B-abliterated-v2MAX_JOBS=$(nproc)pip3installmodelscope modelscope download--modelhuihui-ai/DeepSeek-R1-Distill-Qwen-7B-abliterated-v2--local_dir/mnt/D/modelscope/DeepSeek-R1-Distill-Qwen-7B-abliterated-v2配置
- 编写推理脚本:
# 安装依赖MAX_JOBS=$(nproc)pipinstalltorch==2.9.0torchvision==0.24.0torchaudio==2.9.0 --index-url https://download.pytorch.org/whl/cu126MAX_JOBS=$(nproc)pipinstalltransformers datasets acceleratecat>/mnt/D/modelscope/inference.py<<'EOF' from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("/mnt/D/modelscope/DeepSeek-R1-Distill-Qwen-7B-abliterated-v2") model = AutoModelForCausalLM.from_pretrained("/mnt/D/modelscope/DeepSeek-R1-Distill-Qwen-7B-abliterated-v2") messages = [ {"role": "user", "content": "你是谁?能帮助我做什么？"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) EOF测试
- 选择命令行：运行与测试
chmod0777-R/mnt/D/modelscope/inference.pychown$USER:$USER-R/mnt/D/modelscope/inference.py# 查看输出结果python3 /mnt/D/modelscope/inference.py
