Seatunnel 2.3.0 本地模式 Hazelcast instance is not active!

Apache SeaTunnel 2.1.0部署及踩坑 简介 SeaTunnel 原名Waterdrop,自2021年10月12日改名为SeaTunnel。 SeaTunnel是一个非常易于使用的超高性能分布式数据集成平台,支持海量数据的实时同步。它每天可以稳定高效地同步数百亿数据,已在近100家公司的生产中使用。 特点 易于使用,配置灵活,低代码开发 实时流媒体 离线多源数据分析 高性能、海量数据处理能力 模块化和插入式机构,易于扩展 支持通过SQL进行数据处理和聚合 支持Spark结构化流媒体 支持Spark 2.x 这里我们踩了一个坑,因为我们测试的s 阅读详情

已解决,打包好了,方便内网使用。
https://download.youkuaiyun.com/download/keepandkeep/87623446
---------------------------------------------------------分界线---------------------------------------------
Seatunnel 2.3.0本地模式的demo跑不通,烦死了,记录下。

按官网的运行本地模式的demo,报错Hazelcast instance is not active!:
好像是它自己的包冲突了,error java.lang.NoClassDefFoundError:org/apache/hadoop/conf/Configuration

完整报错日志输出如下:

2023-03-28 16:43:29,617 ERROR org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] submit job 693017334083747841 error java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.LocalConfiguration.buildConfiguration(LocalConfiguration.java:37)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.getConfiguration(HdfsStorage.java:77)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.initStorage(HdfsStorage.java:64)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.<init>(HdfsStorage.java:55)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.HdfsFileStorageInstance.getOrCreateStorage(HdfsFileStorageInstance.java:52)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorageFactory.create(HdfsStorageFactory.java:68)
        at org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.<init>(CheckpointManager.java:91)
        at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:183)
        at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$4(CoordinatorService.java:338)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 14 more
 
2023-03-28 16:43:29,623 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job Job SeaTunnel (693017334083747841) waiting for scheduler finished
2023-03-28 16:43:29,625 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel (693017334083747841) turn from state CREATED to SCHEDULED.
2023-03-28 16:43:29,626 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel_default_cluster-867616] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2023-03-28 16:43:29,629 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel (693017334083747841), Pipeline: [(1/1)] turn from state CREATED to SCHEDULED.
2023-03-28 16:43:29,630 INFO  com.hazelcast.internal.server.tcp.TcpServerConnection - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:42149, qualifier=null, endpoint=[127.0.0.1]:42149, remoteUuid=a7689156-c01a-4c7a-958b-776d6ec787f7, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2023-03-28 16:43:29,632 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel_default_cluster-867616] [5.1] Removed connection to endpoint: [localhost]:5801:029dd39f-7c48-436a-96d0-305d34c1decf, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:42149->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2023-03-28 16:43:29.621, lastWriteTime=2023-03-28 16:43:28.973, closedTime=2023-03-28 16:43:29.628, connected server version=5.1}
2023-03-28 16:43:29,632 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel_default_cluster-867616] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2023-03-28 16:43:29,634 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel (693017334083747841), Pipeline: [(1/1)], task: [Jdbc-SplitEnumerator (1/1)] turn from state CREATED to SCHEDULED.
2023-03-28 16:43:29,634 INFO  com.hazelcast.client.impl.ClientEndpointManager - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:42149, qualifier=null, endpoint=[127.0.0.1]:42149, remoteUuid=a7689156-c01a-4c7a-958b-776d6ec787f7, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=a7689156-c01a-4c7a-958b-776d6ec787f7, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1679993007923, latest clientAttributes=lastStatisticsCollectionTime=1679993008040,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1679993007910,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=49664143360,os.freePhysicalMemorySize=223351263232,os.freeSwapSpaceSize=42949668864,os.maxFileDescriptorCount=4096,os.openFileDescriptorCount=50,os.processCpuTime=10580000000,os.systemLoadAverage=0.04,os.totalPhysicalMemorySize=269871099904,os.totalSwapSpaceSize=42949668864,runtime.availableProcessors=64,runtime.freeMemory=2279923312,runtime.maxMemory=28631367680,runtime.totalMemory=2374500352,runtime.uptime=1969,runtime.usedMemory=94577040, labels=[]}
2023-03-28 16:43:29,641 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel_default_cluster-867616] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2023-03-28 16:43:29,643 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] [localhost]:5801 is SHUTTING_DOWN
2023-03-28 16:43:29,644 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - waiting current worker register to resource manager...
2023-03-28 16:43:29,651 INFO  com.hazelcast.internal.partition.impl.MigrationManager - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] Shutdown request of Member [localhost]:5801 - 029dd39f-7c48-436a-96d0-305d34c1decf this is handled
2023-03-28 16:43:29,660 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] Shutting down connection manager...
2023-03-28 16:43:29,662 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] Shutting down node engine...
2023-03-28 16:43:29,677 ERROR com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] Error while shutting down service[org.apache.seatunnel.engine.server.SeaTunnelServer@5b5a4aed]: null
java.lang.NullPointerException: null
        at org.apache.seatunnel.engine.server.master.JobMaster.interrupt(JobMaster.java:443) ~[seatunnel-starter.jar:2.3.0]
        at java.util.concurrent.ConcurrentHashMap$ValuesView.forEach(ConcurrentHashMap.java:4707) ~[?:1.8.0_311]
        at org.apache.seatunnel.engine.server.CoordinatorService.clearCoordinatorService(CoordinatorService.java:291) ~[seatunnel-starter.jar:2.3.0]
        at org.apache.seatunnel.engine.server.CoordinatorService.shutdown(CoordinatorService.java:456) ~[seatunnel-starter.jar:2.3.0]
        at org.apache.seatunnel.engine.server.SeaTunnelServer.shutdown(SeaTunnelServer.java:121) ~[seatunnel-starter.jar:2.3.0]
        at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.shutdownService(ServiceManagerImpl.java:308) [seatunnel-starter.jar:2.3.0]
        at com.hazelcast.spi.impl.servicemanager.impl.ServiceManagerImpl.shutdown(ServiceManagerImpl.java:299) [seatunnel-starter.jar:2.3.0]
        at com.hazelcast.spi.impl.NodeEngineImpl.shutdown(NodeEngineImpl.java:549) [seatunnel-starter.jar:2.3.0]
        at com.hazelcast.instance.impl.Node.shutdownServices(Node.java:605) [seatunnel-starter.jar:2.3.0]
        at com.hazelcast.instance.impl.Node.shutdown(Node.java:544) [seatunnel-starter.jar:2.3.0]
        at com.hazelcast.instance.impl.LifecycleServiceImpl.shutdown(LifecycleServiceImpl.java:101) [seatunnel-starter.jar:2.3.0]
        at com.hazelcast.instance.impl.LifecycleServiceImpl.shutdown(LifecycleServiceImpl.java:84) [seatunnel-starter.jar:2.3.0]
        at com.hazelcast.instance.impl.HazelcastInstanceProxy.shutdown(HazelcastInstanceProxy.java:311) [seatunnel-starter.jar:2.3.0]
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:103) [seatunnel-starter.jar:2.3.0]
        at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:39) [seatunnel-starter.jar:2.3.0]
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:31) [seatunnel-starter.jar:2.3.0]
2023-03-28 16:43:30,154 ERROR org.apache.seatunnel.engine.server.master.JobMaster - Job SeaTunnel (693017334083747841) run error with: java.util.concurrent.CompletionException: com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
        at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
        at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1643)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance is not active!
        at com.hazelcast.spi.impl.operationservice.impl.InvocationRegistry.register(InvocationRegistry.java:140)
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:570)
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:541)
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:241)
        at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:61)
        at com.hazelcast.map.impl.proxy.MapProxySupport.invokeOperation(MapProxySupport.java:478)
        at com.hazelcast.map.impl.proxy.MapProxySupport.getInternal(MapProxySupport.java:371)
        at com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:123)
        at org.apache.seatunnel.engine.server.dag.physical.SubPlan.getPipelineState(SubPlan.java:391)
        at org.apache.seatunnel.engine.server.dag.physical.SubPlan.cancelPipeline(SubPlan.java:240)
        at org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler.schedulerPipeline(PipelineBaseScheduler.java:102)
        at org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler.lambda$startScheduling$0(PipelineBaseScheduler.java:66)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
        at org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler.startScheduling(PipelineBaseScheduler.java:67)
        at org.apache.seatunnel.engine.server.master.JobMaster.lambda$run$1(JobMaster.java:223)
        at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at ------ submitted from ------.()
        at com.hazelcast.internal.util.ExceptionUtil.cloneExceptionWithFixedAsyncStackTrace(ExceptionUtil.java:336)
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.returnOrThrowWithGetConventions(InvocationFuture.java:112)
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveAndThrowIfException(InvocationFuture.java:100)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.get(AbstractInvocationFuture.java:609)
        at com.hazelcast.map.impl.proxy.MapProxySupport.invokeOperation(MapProxySupport.java:479)
        at com.hazelcast.map.impl.proxy.MapProxySupport.getInternal(MapProxySupport.java:371)
        at com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:123)
        at org.apache.seatunnel.engine.server.dag.physical.SubPlan.getPipelineState(SubPlan.java:391)
        at org.apache.seatunnel.engine.server.dag.physical.SubPlan.cancelPipeline(SubPlan.java:240)
        at org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler.schedulerPipeline(PipelineBaseScheduler.java:102)
        at org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler.lambda$startScheduling$0(PipelineBaseScheduler.java:66)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
        at org.apache.seatunnel.engine.server.scheduler.PipelineBaseScheduler.startScheduling(PipelineBaseScheduler.java:67)
        at org.apache.seatunnel.engine.server.master.JobMaster.lambda$run$1(JobMaster.java:223)
        at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
        ... 3 more

2023-03-28 16:43:32,703 INFO  com.hazelcast.instance.impl.NodeExtension - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] Destroying node NodeExtension.
2023-03-28 16:43:32,704 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] Hazelcast Shutdown is completed in 3056 ms.
2023-03-28 16:43:32,705 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel_default_cluster-867616] [5.1] [localhost]:5801 is SHUTDOWN
2023-03-28 16:43:32,705 ERROR org.apache.seatunnel.core.starter.Seatunnel - 

===============================================================================


2023-03-28 16:43:32,705 ERROR org.apache.seatunnel.core.starter.Seatunnel - Fatal Error, 

2023-03-28 16:43:32,705 ERROR org.apache.seatunnel.core.starter.Seatunnel - Please submit bug report in https://github.com/apache/incubator-seatunnel/issues

2023-03-28 16:43:32,705 ERROR org.apache.seatunnel.core.starter.Seatunnel - Reason:java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration 

2023-03-28 16:43:32,708 ERROR org.apache.seatunnel.core.starter.Seatunnel - Exception StackTrace:java.util.concurrent.CompletionException: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
        at com.hazelcast.spi.impl.AbstractInvocationFuture.wrapInCompletionException(AbstractInvocationFuture.java:1347)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.cascadeException(AbstractInvocationFuture.java:1340)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.access$200(AbstractInvocationFuture.java:65)
        at com.hazelcast.spi.impl.AbstractInvocationFuture$ApplyNode.execute(AbstractInvocationFuture.java:1478)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockOtherNode(AbstractInvocationFuture.java:797)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockAll(AbstractInvocationFuture.java:759)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.complete0(AbstractInvocationFuture.java:1235)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionallyInternal(AbstractInvocationFuture.java:1223)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionally(AbstractInvocationFuture.java:709)
        at com.hazelcast.client.impl.spi.impl.ClientInvocation.completeExceptionally(ClientInvocation.java:294)
        at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyExceptionWithOwnedPermission(ClientInvocation.java:321)
        at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:304)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.handleResponse(ClientResponseHandlerSupplier.java:164)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.process(ClientResponseHandlerSupplier.java:141)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.access$300(ClientResponseHandlerSupplier.java:60)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:251)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:243)
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnection.handleClientMessage(TcpClientConnection.java:245)
        at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:135)
        at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.onRead(ClientMessageDecoder.java:89)
        at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:136)
        at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383)
        at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368)
        at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294)
        at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249)
        at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.LocalConfiguration.buildConfiguration(LocalConfiguration.java:37)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.getConfiguration(HdfsStorage.java:77)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.initStorage(HdfsStorage.java:64)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.<init>(HdfsStorage.java:55)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.HdfsFileStorageInstance.getOrCreateStorage(HdfsFileStorageInstance.java:52)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorageFactory.create(HdfsStorageFactory.java:68)
        at org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.<init>(CheckpointManager.java:91)
        at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:183)
        at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$4(CoordinatorService.java:338)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
 
2023-03-28 16:43:32,708 ERROR org.apache.seatunnel.core.starter.Seatunnel - 
===============================================================================



Exception in thread "main" java.util.concurrent.CompletionException: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
        at com.hazelcast.spi.impl.AbstractInvocationFuture.wrapInCompletionException(AbstractInvocationFuture.java:1347)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.cascadeException(AbstractInvocationFuture.java:1340)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.access$200(AbstractInvocationFuture.java:65)
        at com.hazelcast.spi.impl.AbstractInvocationFuture$ApplyNode.execute(AbstractInvocationFuture.java:1478)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockOtherNode(AbstractInvocationFuture.java:797)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockAll(AbstractInvocationFuture.java:759)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.complete0(AbstractInvocationFuture.java:1235)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionallyInternal(AbstractInvocationFuture.java:1223)
        at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionally(AbstractInvocationFuture.java:709)
        at com.hazelcast.client.impl.spi.impl.ClientInvocation.completeExceptionally(ClientInvocation.java:294)
        at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyExceptionWithOwnedPermission(ClientInvocation.java:321)
        at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:304)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.handleResponse(ClientResponseHandlerSupplier.java:164)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.process(ClientResponseHandlerSupplier.java:141)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.access$300(ClientResponseHandlerSupplier.java:60)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:251)
        at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:243)
        at com.hazelcast.client.impl.connection.tcp.TcpClientConnection.handleClientMessage(TcpClientConnection.java:245)
        at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:135)
        at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.onRead(ClientMessageDecoder.java:89)
        at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:136)
        at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383)
        at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368)
        at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294)
        at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249)
        at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.LocalConfiguration.buildConfiguration(LocalConfiguration.java:37)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.getConfiguration(HdfsStorage.java:77)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.initStorage(HdfsStorage.java:64)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.<init>(HdfsStorage.java:55)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.HdfsFileStorageInstance.getOrCreateStorage(HdfsFileStorageInstance.java:52)
        at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorageFactory.create(HdfsStorageFactory.java:68)
        at org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.<init>(CheckpointManager.java:91)
        at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:183)
        at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$4(CoordinatorService.java:338)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

使用Apache SeaTunnel进行数据库同步(MySQL to MySQL) 使用Apache seaTunnel同步mysql数据库 阅读详情

相关推荐

Flink Checkpoint超时问题

Flink Checkpoint超时问题排查 文章目录Flink Checkpoint超时问题排查问题现象问题分析问题1:TaskManager进程挂掉问题2:任务长时间处于CANCELING问题3:Checkpoint超时问题4:数据无法正常同步解决思路总结参考文档 问题现象 业务部门最近使用Flink来做数据实时同步,通过同步工具把CDC消息接入Kafka,其中上百张表同步到单个topic里,然后通过Flink来消费Kafka,做数据解析、数据分发、然后发送到目标数据库(mysql/oracle),整个

lisenyeahyeah的博客 1万+

Seatunnel部署及一些出错

Seatunnel部署及我部署时的一些出错

underworld33的博客 4634

超10万,两台Mac Studio在家就能跑满血DeepSeek,网友:这是性价比最高的大模型一体机

在AI计算领域,没有比这更强的个人电脑了。这要从它的架构说起。对于那些有本地部署满血版DeepSeek需求的用户,目前建议的配置是选购512GB统一内存 + 1TB SSD,搭载M3 Ultra芯片的Mac Studio,官网售价74249元,可以叠加国补、教育优惠、12期分期等,这可能是目前门槛最低、性价比最高的满血版DeepSeek本地部署方案。对于企业而言,尽管Mac Studio确实可以部署6000亿参数的大模型,但在内存带宽、模型精调服务等方面,跟专用的大模型一体机仍有差距,采购需要谨慎评估。

2501_90444774的博客 1917

call-helloworld在tuscany-extension上运行失败,需要预处理!

Exception in thread "main" java.lang.reflect.InvocationTargetException     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMet

ndzj981479673的专栏 1104

使用MiniExcel.SaveAsByTemplate快速地保存百万数据

该代码展示了如何使用MiniExcel快速生成并导出百万级销售数据。核心方法GenerateLargeData()可生成指定数量的随机销售记录,包含客户名、日期、VIP状态、积分等20个字段。通过随机函数模拟真实数据分布,并支持自定义数据量(默认100万条)。配合Excel模板使用MiniExcel.SaveAsByTemplate()方法,能高效地将大数据量导出为Excel文件,且处理速度极快。代码结构清晰,包含数据生成、随机日期辅助方法和数据模型类,适合需要批量生成测试数据或导出大型报表的场景。

大坡3D软件开发 253

【ArcGIS微课1000例】0145:如何按照自定义形状裁剪数据框?

使用绘图 工具条上的新建圆工具 可创建一个椭圆,使其包含要在该数据框中显示的数据范围。. 在基础工具工具条上单击选择元素工具 ,然后选择圆形。. 在内容列表中右键单击要裁剪的数据框,然后单击属性。. 单击裁剪选项下拉菜单,然后选择裁剪至形状。. 单击数据框属性 对话框中的数据框选项卡。还可以更改裁剪形状轮廓的边框选项。打开软件,添加配套实验数据包中。修改椭圆的属性,使其透明。. 单击所选图形的轮廓。. 单击指定形状按钮。

智慧空间实验室 953

Apache SeaTunel & Apache SeaTunel WEB安装部署(一)

1.下载解压安装seatunel2.安装连接插件自动安装(推荐)手动安装从 Maven 仓库下载所需连接器 JAR,放入:3.测试。

Mrzhou的博客 298

华为OD机试真题 新系统 2026-09-26 Java&Go&C【数据中心最佳维护时间窗】

数据中心使用一个整数数组 scores,记录 N 小时内每小时的服务器得分(可正可负)。0 表示服务器故障(无效小时)。

MISAYAONE的博客 97

Apache seatunel-数据同步测试(mysql to mysql)安装部署(二)

在上一次文章中,安装了seatunel 和 seatunel web,在这期文章中我们继续进行数据源的连接配置,以及数据同步测试。

Mrzhou的博客 304

C#部署YOLO三种方案深度实测:OpenCV DNN vs ONNX Runtime vs TensorRT

在工业视觉项目落地阶段,C#生态下部署YOLO模型主要有三条技术路线。很多开发者仅凭经验选型,往往在部署后期踩中性能瓶颈、兼容故障、维护成本超标的坑。本文基于工控机常见硬件配置,针对YOLOv8系列模型做控制变量实测,从推理性能、部署难度、工业稳定性、综合成本等多个维度做横向对比,给出可直接落地的选型结论。

这里只有干货:从 Modbus 通信到 MES 对接,从 YOLO 训练到工控机部署,带你搞定工业软件开发全流程。 464

【四足机器人项目实战】MIT无刷电机FOC硬件电路设计

系列文章目录 提示:这里可以添加系列文章的所有文章的目录,目录需要自己手动添加 TODO:写完再整理 文章目录系列文章目录前言一、MCU最小系统--STM32F446RE1.usb串口通讯连接口+程序下载连接口2.MCU电源滤波二、磁编码器角度传感器三、三相驱动桥四、CAN通讯收发器电路五、温度ADC检测六、电源滤波及分压电路1、DC电源输入端2、DC电源滤波电路3、DC电源分压电路4、三相绕线接线端总结参考资料 前言 认知有限,望大家多多包涵,有什么问题也希望能够与大家多交流,共同成长! 本文先对

qq_35635374的博客 7061

部署SeaTunnel分布式集群

基于Hazelcast,SeaTunnel Engine 集群是运行 SeaTunnel Engine Server 的集群成员网络。只需要将$SEATUNNEL_HOMESeaTunnel Engine节点上的目录复制到Client节点,并配置SEATUNNEL_HOME类似的SeaTunnel Engine Server节点即可。这个参数主要是用来做集群隔离的,我们可以用这个来区分不同的集群,比如cluster1,cluster2,这个也用来区分不同的业务。N是集群节点的编号。

zhengzaifeidelushang的博客 7442

(ICLR 2025)【源码讲解+复现】TIMER-XL: Long-Context Transformers for Unified Time Series Forecasting

本文提出了一种统一的多变量时序预测模型Timer-XL,其核心创新包括:1)采用自回归解码器架构,将多变量时序预测统一为next-token预测问题;2)设计TimeAttention机制,通过相对位置编码和因果掩码捕获时序位置和变量间交互;3)支持长上下文预训练以提升零样本迁移能力。模型架构包含嵌入层、时间感知注意力块和预测头,采用MSE损失进行端到端训练。实验表明该方法在多种时序任务上表现优异,特别是在长跨度预测和少样本场景下。代码实现提供了完整的模型定义、数据加载和训练流程,支持自定义配置参数。

1146

Openfire异常备忘录

1. XMPP无法连接  1.1 异常信息  1.1.1 error.log 2014.10.31 12:03:15 org.jivesoftware.util.cache.CacheFactory - Hazelcast instance is not active! com.hazelcast.core.HazelcastInstanceNotActiveException: Ha...

我的博客 558

L2正则化与Ridge回归的关系

非常感谢您提供的详细任务描述和要求。我会尽力按照您的指引来撰写这篇技术博客文章。 L2正则化与Ridge回归的关系 1. 背景介绍 机器学习模型在训练过程中容易出现过拟合的问题,这会导致模型在训练集上表现很好但在测试集上泛化性能较差。为了解决这一问题,正则化技术应运而生。其中

AGI×大数据,开启智能时代的认知跃迁;解码AGI,赋能数据驱动的智能革命。 657

锂电池充电器电路设计 里面有电路

锂电池充电器电路设计 里面有电路 3.7V升压到5V电路设计。

Hazelcast 分布式缓存 在Seatunnel中的使用

本文只是介绍了hazelcast的最基本用法,如果按照案例中的使用,完全可以用redis或者本地缓存。但是如果有了更高级(实际中的使用),那么hazelcast的分布式计算特性还是很好用的。源码也只是分析了本地的调用。如果感兴趣其实可以debug跟进去看下远程调用的方式。其实想想本质还是一样,远程调用就需要1、发现节点;2、注册节点;3、网络调用其他节点。而seatunnel的调用就相对来说更高级一些,它进行了一系列的封装。最后也还是网络调用其他节点。

TNT_D的博客 1930

IEM模型,地表粗糙度模拟

利用微波反演土壤水分时候,利用IEM模型可以实现对随机地表的模拟

hazelcast的坑爹事

简介 开源中国的简介: Hazelcast是一个高度可扩展的数据分发和集群平台。特性包括: 提供java.util.{Queue, Set, List, Map}分布式实现。提供java.util.concurrency.locks.Lock分布式实现。提供java.util.concurrent.ExecutorService分布式实现。提供用于一对多关系的分布式MultiMa

横云断岭的专栏 3万+

SAP B1中文培訓教材_SDK

公司内部培训资料,对开发工具 ,DIAPI, UIAPI,Add-ON有详细解说

复分析 阿尔福斯(Ahlfors) 第三版答案 英文

答案分为十章,有部分习题没有答案,全英文复分析阿尔福斯(Ahlfors) 第三版答案

数据同步利器之seatunnel篇

至此,seatunnel集群就已经部署完成了,看过seatunel官网介绍过跟datax的数据同步效率对比,大概提升了20%-40%,本人经过实测,速度确实更快了,而且配置比datax的json格式更为简单,seatunnel还提供了transform中间转换功能,当然了由于seatunel是后起之秀,还有不少需要完善改进的空间,但发展前景还是挺好的,希望有越来越多优秀的工具能够出现,由于篇幅有限,今天就聊到这里,如果有兴趣的话,你可以去试试这个工具。

以茉萱的博客 4917

Apache SeaTunnel:新一代高性能、分布式、海量数据集成工具从入门到实践

本文介绍Apache SeaTunnel 以及关于Apache SeaTunnel的部署和使用。 Apache SeaTunnel是新一代高性能、分布式、海量数据集成工具,支持上百种数据源 ( Database/Cloud/SaaS ) 支持海量数据的实时 CDC 和批量同步,可以稳定高效地同步万亿级数据,

分享云原生大数据技术和个人成长、职业发展相关内容 1万+

Linux安装并使用seatunnel2.3.1

通常不需要所有的连接器插件,所以你可以通过配置指定你需要的插件config/plugin_config,比如你只需要connector-console插件,那么你可以修改plugin_config为。修改了config下的hazelcast-client.yaml 和hazelcast.yaml。SeaTunnel是一个非常易用的超高性能分布式数据集成平台,支持海量数据的实时同步。将5801改为6801,任意一个没被使用的端口即可。这个命令执行的时间有点长,可以按需加载插件。可以直接通过命令执行。

u010479989的博客 2161

seatunnel k8s运行zeta引擎(cluster-mode模式)

我之前已经解压并执行过install-plugin.sh,这里为了方便,我直接用的执行过install-plugin.sh脚本之后的seatunnel做演示。在安装seatunnel的同级文件夹下创建Dockerfile。10.待所有节点都更新完毕处于running状态后 可进入容器内查看路径是否已修改。我们可以再新打开一个连接,登录另一个pod节点执行任务来测试集群。4.将镜像load进k8s,这里用的minikube做演示。执行过install-plugin后的lib目录包含如下。

weixin_41854429的博客 900

Hazelcast在Spring Boot中使用发生的问题(一)

最近项目中有开发人员使用在SpringBoot中使用hazelcast,出现了一些问题,然后分析了一下问题原因,记录一下。 一、背景 开发人员依照经验,在SpringBoot中是这样使用hazelcast的: 1. 在Classpath下创建了hazelcast的配置文件,其中<network>是这么配置的: <network> <p...

JJFuture的专栏 2902

Hazelcast的消息驱动架构:构建实时通信应用

1.背景介绍 Hazelcast是一个开源的分布式计算平台,它提供了一种高性能的数据存储和处理方法,以及一种高性能的实时通信机制。Hazelcast的消息驱动架构是其核心功能之一,它允许开发者轻松地构建实时通信应用,例如聊天室、实时位置共享、实时数据同步等。 在本文中,我们将深入探讨Hazelcast的消息驱动架构,包括其核心概念、算法原理、具体操作步骤以及数学模型公式。此外,我们还将通过具体...

AI天才研究院 665

Apache Seatunnel部署(apache-seatunnel-2.3.3-bin.tar.gz + apache-seatunnel-web-1.0.0-bin.tar.gz)

还处在开发早期阶段,并不成熟,功能也不完善🍓安装seatunnel的目的,就是想参考下web界面,寻求针对表的拖拽处理(减少sql编写),目前看效果不理想,也只是一个简单的输入、输出表字段映射,因此暂告一段落。

4967

Hazelcast系列(八):数据结构

前面已经对 Hazelcast 集成与各种发现机制进行了探讨,接下来,让我们来一起了解一下数据结构的特点与使用。当前hazelcast 版本:5.1.7。

Show Me The Code 1140

从启动到关闭 | SeaTunnel2.1.1源码解析

点亮 ⭐️ Star · 照亮开源之路​本文转载自Adobee Chen的博客-优快云博客,看看是否有你感兴趣的吧!如有出错,请多指正。来,和社区一同成长! Apache SeaTunnel(Incubating) 是一个分布式、高性能、易扩展、用于海量数据(离线 & 实时)同步和转化的数据集成平台。

weixin_54625990的博客 947
上一篇: Linux查看某一端口是否开放
下一篇: neo4j删除大规模数据
keepandkeep
博客等级 码龄8年 380粉丝 34原创
评论 2
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值