背景
在本地搭建了Ray的单节点服务,提交任务后,Ray报错提示空间不够
报错信息示例:
(raylet) [2023-03-24 11:06:38,311 E 821713 821740] (raylet) file_system_monitor.cc:105:
/tmp/ray/session_2023-03-24_11-03-06_248168_820779 is over 95% full,
available space: 5726846976; capacity: 1006449913856.
Object creation will fail if spilling is required.
解决方案
You can free up any crud that you might have in /tmp/. It wants to spill objects over onto disk if object store is full. You can specify in ray.init()
where you want objects to be spilled to.
在ray init时指定一个新的tmp目录
ray.init(_temp_dir="一个新的目录")
例如:
/xxx/xxx/ray/tmp