Pytorch [Errno 32] Broken pipe 错误

最近自学pytorch,强迫自己多写代码,今天在定义好dataset后,使用dataloader生成数据的时候,我想检查一下数据生成是否正常,于是写了个简单测试,如下

for i,d in enumerate(dataloader):
    print("i=", i)
    a,b,c=d
    print("center word :",a.shape)
    print("pos_word:", b.shape)
    print("neg_word:",c.shape)
    if i==5:
        break

喝了口咖啡,从容的按下了shift+回车,然后看到华丽丽的报错信息。。。

---------------------------------------------------------------------------
BrokenPipeError                           Traceback (most recent call last)
<ipython-input-23-6e527f357819> in <module>()
----> 1 for i,d in enumerate(dataloader):
      2     print("i=", i)
      3     a,b,c=d
      4     print("center word :",a.shape)
      5     print("pos_word:", b.shape)

e:\python3.6\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in __iter__(self)
    277             return _SingleProcessDataLoaderIter(self)
    278         else:
--> 279             return _MultiProcessingDataLoaderIter(self)
    280 
    281     @property

e:\python3.6\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in __init__(self, loader)
    717             #     before it starts, and __del__ tries to join but will get:
    718             #     AssertionError: can only join a started process.
--> 719             w.start()
    720             self._index_queues.append(index_queue)
    721             self._workers.append(w)

e:\python3.6\Anaconda3\lib\multiprocessing\process.py in start(self)
    103                'daemonic processes are not allowed to have children'
    104         _cleanup()
--> 105         self._popen = self._Popen(self)
    106         self._sentinel = self._popen.sentinel
    107         # Avoid a refcycle if the target function holds an indirect

e:\python3.6\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
    221     @staticmethod
    222     def _Popen(process_obj):
--> 223         return _default_context.get_context().Process._Popen(process_obj)
    224 
    225 class DefaultContext(BaseContext):

e:\python3.6\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
    320         def _Popen(process_obj):
    321             from .popen_spawn_win32 import Popen
--> 322             return Popen(process_obj)
    323 
    324     class SpawnContext(BaseContext):

e:\python3.6\Anaconda3\lib\multiprocessing\popen_spawn_win32.py in __init__(self, process_obj)
     63             try:
     64                 reduction.dump(prep_data, to_child)
---> 65                 reduction.dump(process_obj, to_child)
     66             finally:
     67                 set_spawning_popen(None)

e:\python3.6\Anaconda3\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
     58 def dump(obj, file, protocol=None):
     59     '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60     ForkingPickler(file, protocol).dump(obj)
     61 
     62 #

BrokenPipeError: [Errno 32] Broken pipe

这是啥东东,以前还真没见过。。。我以为是哪写的不对,比着源代码检查了好几遍,没找到问题,无奈请教了度娘,找到了原因 ,https://blog.youkuaiyun.com/qq_33666011/article/details/81873217,官方bug,臣妾也没办法啊。。。。把dataloader的参数num_workers修改成0,果然好了。。。

i= 0
center word : torch.Size([128])
pos_word: torch.Size([128, 6])
neg_word: torch.Size([128, 600])
i= 1
center word : torch.Size([128])
pos_word: torch.Size([128, 6])
neg_word: torch.Size([128, 600])
i= 2
center word : torch.Size([128])
pos_word: torch.Size([128, 6])
neg_word: torch.Size([128, 600])
i= 3
center word : torch.Size([128])
pos_word: torch.Size([128, 6])
neg_word: torch.Size([128, 600])
i= 4
center word : torch.Size([128])
pos_word: torch.Size([128, 6])
neg_word: torch.Size([128, 600])
i= 5
center word : torch.Size([128])
pos_word: torch.Size([128, 6])
neg_word: torch.Size([128, 600])

 

评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值