Loss was due to java.lang.UnsatisfiedLinkError: org.jblas.NativeBlas.dposv

最新推荐文章于 2021-05-31 17:14:47 发布

谷雨hadoop

最新推荐文章于 2021-05-31 17:14:47 发布

阅读量1.2k

点赞数 1

分类专栏： spark 文章标签： spark

spark 专栏收录该内容

8 篇文章

订阅专栏

14/11/04 17:22:55 INFO TaskSetManager: Starting task 120.0:16 as TID 299 on executor 2: namenode1 (PROCESS_LOCAL)
14/11/04 17:22:55 INFO TaskSetManager: Serialized task 120.0:16 as 10621 bytes in 1 ms
14/11/04 17:22:55 WARN TaskSetManager: Lost TID 281 (task 120.0:61)
14/11/04 17:22:55 INFO TaskSetManager: Loss was due to java.lang.UnsatisfiedLinkError: org.jblas.NativeBlas.dposv(CII[DII[DII)I [duplicate 6]
14/11/04 17:22:55 INFO TaskSetManager: Starting task 120.0:61 as TID 300 on executor 2: namenode1 (PROCESS_LOCAL)
14/11/04 17:22:55 INFO TaskSetManager: Serialized task 120.0:61 as 10621 bytes in 1 ms
14/11/04 17:22:55 WARN TaskSetManager: Lost TID 224 (task 120.0:4)
14/11/04 17:22:55 INFO TaskSetManager: Loss was due to java.lang.UnsatisfiedLinkError: org.jblas.NativeBlas.dposv(CII[DII[DII)I [duplicate 7]
14/11/04 17:22:55 INFO TaskSetManager: Starting task 120.0:4 as TID 301 on executor 2: namenode1 (PROCESS_LOCAL)
14/11/04 17:22:55 INFO TaskSetManager: Serialized task 120.0:4 as 10621 bytes in 1 ms
14/11/04 17:22:55 WARN TaskSetManager: Lost TID 278 (task 120.0:58)
14/11/04 17:22:55 INFO TaskSetManager: Loss was due to java.lang.UnsatisfiedLinkError: org.jblas.NativeBlas.dposv(CII[DII[DII)I [duplicate 8]
14/11/04 17:22:55 INFO TaskSetManager: Starting task 120.0:58 as TID 302 on executor 2: namenode1 (PROCESS_LOCAL)
14/11/04 17:22:55 INFO TaskSetManager: Serialized task 120.0:58 as 10621 bytes in 0 ms
14/11/04 17:22:55 WARN TaskSetManager: Lost TID 257 (task 120.0:37)
14/11/04 17:22:55 INFO TaskSetManager: Loss was due to java.lang.UnsatisfiedLinkError: org.jblas.NativeBlas.dposv(CII[DII[DII)I [duplicate 9]
14/11/04 17:22:55 INFO TaskSetManager: Starting task 120.0:37 as TID 303 on executor 2: namenode1 (PROCESS_LOCAL)
14/11/04 17:22:55 INFO TaskSetManager: Serialized task 120.0:37 as 10621 bytes in 0 ms
14/11/04 17:22:55 WARN TaskSetManager: Lost TID 299 (task 120.0:16)
14/11/04 17:22:55 INFO TaskSetManager: Loss was due to java.io.FileNotFoundException: /tmp/spark-local-20141104172133-7284/0b/shuffle_0_15_16 (No such file or directory) [duplicate 2]
14/11/04 17:22:55 ERROR TaskSetManager: Task 120.0:16 failed 4 times; aborting job
14/11/04 17:22:55 WARN TaskSetManager: Loss was due to java.io.FileNotFoundException
java.io.FileNotFoundException: /tmp/spark-local-20141104172133-7284/1f/shuffle_0_39_61 (No such file or directory)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
        at org.apache.spark.storage.DiskStore.getBytes(DiskStore.scala:98)
        at org.apache.spark.storage.DiskStore.getValues(DiskStore.scala:124)
        at org.apache.spark.storage.BlockManager.getLocalFromDisk(BlockManager.scala:332)
        at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator$$anonfun$getLocalBlocks$1.apply(BlockFetcherIterator.scala:204)
        at org.apache.spark.storage.BlockFetcherIterator$BasicBlockFetcherIterator$$anonfun$getLocalBlocks$1.apply(BlockFetcherIterator.scala:203)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)

以上问题解决：

You have to make sure that libgfortran library exists on all nodes.

for debian/ubuntu use: sudo apt-get install libgfortran3

for centos use: sudo yum install gcc-gfortran