Spark-杂项

最新推荐文章于 2025-07-24 10:02:16 发布

原创最新推荐文章于 2025-07-24 10:02:16 发布 · 463 阅读

1 ·

CC 4.0 BY-SA版权

spark 专栏收录该内容

23 篇文章

订阅专栏

本文深入探讨了Spark中的输入、序列化、部分结果等关键概念，并介绍了SSL配置选项及其实现方式。主要内容包括如何使用JobListener监听结果，以及如何通过正态分布和T分布进行结果估计。此外，文章还详细解释了SSLOptions类的作用及其在不同通信协议中的配置方法，特别指出由于Akka不支持SSL客户端认证，因此SSLOptions也无法支持。

Spark-杂项

@(spark)[input|serializer|partial]

input

主要是一些inputformat之类的

serializer

class的serializer

partial

Support for approximate results. This provides convenient api and also implementation for approximate calculation.
基本上是通过一个JobListener来监听结果，通过正态分布和T分布来估计结果。

SSLOptions

SSLOptions class is a common container for SSL configuration options. It offers methods to generate specific objects to configure SSL for different communication protocols.
SSLOptions is intended to provide the maximum common set of SSL settings, which are supported by the protocol, which it can generate the configuration for. Since Akka doesn’t support client authentication with SSL, SSLOptions cannot support it either.