<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[bagepython的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/bagepython</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; bagepython]]></copyright><item><title><![CDATA[LInux+ollama的并发数,prompt和显存占用]]></title><link>https://blog.youkuaiyun.com/bagepython/article/details/146178937</link><guid>https://blog.youkuaiyun.com/bagepython/article/details/146178937</guid><author>bagepython</author><pubDate>Tue, 11 Mar 2025 15:30:52 +0800</pubDate><description><![CDATA[测试结果:1.对于请求不超过并发数的请求,响应速度在0.2s以内,超出的请求响应时间较长.2.如果是初次请求,模型的启动时间较长.3.模型的prompt处理长度默认2k,在num_ctx值增大时,超过一个显卡显存数,num_ctx再增大,显存占用明显变大.]]></description><category></category></item><item><title><![CDATA[ollama+qwq]]></title><link>https://blog.youkuaiyun.com/bagepython/article/details/146063380</link><guid>https://blog.youkuaiyun.com/bagepython/article/details/146063380</guid><author>bagepython</author><pubDate>Thu, 06 Mar 2025 11:22:37 +0800</pubDate><description><![CDATA[2.单词 “strawberry” 中有多少个字母 “r”？本次验证,直接使用的默认q4量化版本。1.9.9 和 9.11 哪个更大？这里用到的就是三个简单的问题。3.李白的风格写一首七言绝句。通过ollama启动的qwq。回答结果没有任何问题。]]></description><category></category></item><item><title><![CDATA[Ktransformers-0.2.2+DeepSeek-R1-Q4_K_M]]></title><link>https://blog.youkuaiyun.com/bagepython/article/details/146045673</link><guid>https://blog.youkuaiyun.com/bagepython/article/details/146045673</guid><author>bagepython</author><pubDate>Wed, 05 Mar 2025 16:29:03 +0800</pubDate><description><![CDATA[官方连接来自官方的测试结果llama启动DeepSeek-R1-Q4_K_M 4090*2  decode token: 3.73token/s 速度太慢下面就没再测。]]></description><category></category></item><item><title><![CDATA[Linux+llama.cpp编译+DeepSeek-R1-UD-IQ1_S本地部署]]></title><link>https://blog.youkuaiyun.com/bagepython/article/details/145881835</link><guid>https://blog.youkuaiyun.com/bagepython/article/details/145881835</guid><author>bagepython</author><pubDate>Wed, 26 Feb 2025 16:37:39 +0800</pubDate><description><![CDATA[提示词处理速度(token/s)输出文本速度(token/s)克隆llama.cpp。treads/线程数。]]></description><category></category></item><item><title><![CDATA[Xinference+Transformers+vLLM 部署qwen2.5-72B]]></title><link>https://blog.youkuaiyun.com/bagepython/article/details/145161991</link><guid>https://blog.youkuaiyun.com/bagepython/article/details/145161991</guid><author>bagepython</author><pubDate>Wed, 15 Jan 2025 16:15:24 +0800</pubDate><description><![CDATA[数值越大，占用显存越大，torch.OutOfMemoryError: [address=0.0.0.0:41165, pid=1393874] CUDA out of memory.可以通过界面最下方的vLLM参数设置gpu_memory_utilization，max_model_len。--model-engine vLLM：模型占用显存大小会受到"gpu_memory_utilization"参数影响，默认0.9。在下载模型的时候如果显示上面错误可能是因为huggingface源问题。]]></description><category></category></item></channel></rss>