u010899985的博客

curl请求 curl http://localhost:8000/v1/completions \ -H "Content-Type: application/json" \ -d '{ "model": "Qwen/Qwen-14B-Chat-Int4-hf", "prompt": "San Francisco is a", "max_tokens": 7, "temperature": 0 }'解决方法：将序列中token按固定长度划分为多个块，并与系统内存进行映射，解决不连续填满问题。