flink maven依赖自用（项目用）

最新推荐文章于 2025-06-22 16:49:11 发布
最新推荐文章于 2025-06-22 16:49:11 发布
阅读量1.5k 收藏 1
点赞数
CC 4.0 BY-SA版权
文章标签： flink 大数据
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.youkuaiyun.com/SQL_mjkl963124412/article/details/125074753 
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>ShaoXing_flink</artifactId>
    <packaging>pom</packaging>
    <version>1.0.0</version>
    <modules>
        <module>ssjs_common</module>
        <module>ssjs_cfzl</module>
    </modules>
    <properties>
        <log4g.version>2.12.1</log4g.version>
        <junit.version>4.12</junit.version>
        <fastjson.version>1.2.83</fastjson.version>
        <elasticsearch.version>7.5.1</elasticsearch.version>
        <mysql.version>5.1.39</mysql.version>
        <antlr.version>4.7</antlr.version>
        <flink.version>1.12.3</flink.version>
        <c3p0.version>0.9.1.2</c3p0.version>
        <lombok.version>1.16.16</lombok.version>
        <json4s.version>3.5.3</json4s.version>
        <vertx.version>3.5.2</vertx.version>
        <hbase.version>1.4.3</hbase.version>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.example</groupId>
                <artifactId>ssjs_common</artifactId>
                <version>1.0.0</version>
            </dependency>

            <dependency>
                <groupId>org.apache.commons</groupId>
                <artifactId>commons-lang3</artifactId>
                <version>3.3.2</version>
            </dependency>
            <!--http-->
            <dependency>
                <groupId>org.apache.httpcomponents</groupId>
                <artifactId>httpclient</artifactId>
                <version>4.5.2</version>
            </dependency>

            <dependency>
                <groupId>org.apache.httpcomponents</groupId>
                <artifactId>httpcore</artifactId>
                <version>4.4.5</version>
            </dependency>


            <!--ck存放数据库-->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-statebackend-rocksdb_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <!--日志-->
            <dependency>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-api</artifactId>
                <version>${log4g.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-core</artifactId>
                <version>${log4g.version}</version>
            </dependency>


            <!--单元测试-->
            <dependency>
                <groupId>junit</groupId>
                <artifactId>junit</artifactId>
                <version>${junit.version}</version>
            </dependency>


            <!--flink-->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-clients_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>


            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-core</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-streaming-java_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-json</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-runtime-web_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <!--kafka source-->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-connector-kafka_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <!--吞吐量 内存和cpu使用情况，ck稳定性-->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-metrics-dropwizard</artifactId>
                <version>${flink.version}</version>
            </dependency>
            <!--flink-hbase-->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-connector-hbase-1.4_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <!--flink table-->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-table-api-java-bridge_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-table-planner-blink_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-table-common</artifactId>
                <version>${flink.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-runtime_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <!--hadoop-->
            <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-common</artifactId>
                <version>2.7.2</version>
            </dependency>

            <!--lombok-->
            <dependency>
                <groupId>org.projectlombok</groupId>
                <artifactId>lombok</artifactId>
                <version>${lombok.version}</version>
            </dependency>

            <!--误删-->
            <dependency>
                <groupId>org.antlr</groupId>
                <artifactId>antlr4-runtime</artifactId>
                <version>${antlr.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-runtime</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <!--连接池-->
            <dependency>
                <groupId>c3p0</groupId>
                <artifactId>c3p0</artifactId>
                <version>${c3p0.version}</version>
            </dependency>

            <dependency>
                <groupId>mysql</groupId>
                <artifactId>mysql-connector-java</artifactId>
                <version>${mysql.version}</version>
            </dependency>

            <dependency>
                <groupId>io.vertx</groupId>
                <artifactId>vertx-jdbc-client</artifactId>
                <version>${vertx.version}</version>
            </dependency>

            <dependency>
                <groupId>io.vertx</groupId>
                <artifactId>vertx-core</artifactId>
                <version>${vertx.version}</version>
            </dependency>

            <!--json-->
            <dependency>
                <groupId>com.alibaba</groupId>
                <artifactId>fastjson</artifactId>
                <version>${fastjson.version}</version>
            </dependency>

            <!--dtwave-->
            <dependency>
                <groupId>com.dtwave</groupId>
                <artifactId>common</artifactId>
                <version>1.0</version>
            </dependency>

            <!--es-->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-connector-elasticsearch7_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-connector-elasticsearch-base_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <dependency>
                <groupId>org.elasticsearch</groupId>
                <artifactId>elasticsearch</artifactId>
                <version>${elasticsearch.version}</version>
                <exclusions>
                    <exclusion>
                        <groupId>jackson-core</groupId>
                        <artifactId>com.fasterxml.jackson.core</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>

            <!--jdbc-->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-connector-jdbc_2.11</artifactId>
                <version>${flink.version}</version>
            </dependency>

            <!--hbase-->
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-client</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-common</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-hadoop-compat</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-hadoop2-compat</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-metrics</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-metrics-api</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-prefix-tree</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-protocal</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hbase</groupId>
                <artifactId>hbase-server</artifactId>
                <version>${hbase.version}</version>
            </dependency>
            <dependency>
                <groupId>common-dbcp</groupId>
                <artifactId>common-dbcp</artifactId>
                <version>1.4</version>
            </dependency>

            <dependency>
                <groupId>common-pool</groupId>
                <artifactId>common-pool</artifactId>
                <version>1.6</version>
            </dependency>

            <dependency>
                <groupId>com.oracle</groupId>
                <artifactId>ojdbc6</artifactId>
                <version>11.1.0.7.0</version>
            </dependency>

            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-api</artifactId>
                <version>1.7.26</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
                <version>1.7.26</version>
            </dependency>
            <dependency>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
                <version>1.2.17</version>
            </dependency>
        </dependencies>
    </dependencyManagement>
</project>

Flink Maven 依赖
王小工小工历程
08-05 1442
定义：Apache Flink ML是Apache Flink的扩展库，提供了一套全面的机器学习（ML）API和基础设施，旨在简化机器学习管道的构建过程。开发者：Apache软件基金会编程语言：支持Python和Java，方便不同编程背景的开发者使用。
Flink所使用到maven依赖
影子
03-12 6478
Flink本身包含系统运行所需的类和依赖项，如协调、网络、检查点、故障转移、操作、资源管理等 <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-core</artifactId> <version>1.9.0</version> </dependency> java版本依赖： <d
Flink SQL Connector Kafka 核心参数全解析与实战指南
最新发布
qq_42773076的博客
06-22 1106
本文详细介绍了Flink SQL Kafka连接器的使用方法，涵盖环境配置、表定义、参数设置和实战场景。主要内容包括：1) Maven依赖和环境要求；2) Kafka表定义与元数据映射规则；3) 核心参数分类解析，包括连接配置、消费模式、数据格式和写入保证；4) 高级特性如动态主题发现、CDC支持和安全认证；5) 典型应用场景示例。通过本文，开发者可快速掌握构建实时数据管道的关键技术要点，实现Flink与Kafka的高效集成。
Flink开发MAVEN依赖
weixin_43803793的博客
11-09 1256
<properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <flink.version>1.12.3</flink.version>
大数据——Flink Maven工程依赖
m0_67265464的博客
08-28 414
【代码】大数据——Flink Maven工程依赖。
分享一个较全的Flink maven依赖文件
Black_8的博客
08-07 3212
使用的是 scala 版本的flink （flink1.11.0）依赖 <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.v
大数据原生集群 (Hadoop2.X为核心) 本地测试环境搭建五
别人笑我太疯癫，我笑他人看不穿。
11-19 1121
本篇安装版本 flink1.7 presto 0.196 druid (imply-2.7.10) Azkaban-2.5.0 开始安装 Flink 一、解压缩 flink-1.7.2-bin-hadoop27-scala_2.11.tgz，进入conf目录中。 二、修改配置 1）修改 flink/conf/flink-conf.yaml 文件，里面开头位置有一个jobmanager.rpc.address，这个jobmanager所在地址集群运行的时候有rpc协议交互，我给的是hdp2 2）修改 con
mavenflink.rar flink整理的依赖包，方便实用
07-26
mavenflink.rar flink整理的依赖包，方便实用
Flink基础之配置Maven依赖、连接、库
热门推荐
主要写大数据集群一线运维和学习工作经验
08-15 1万+
目录 核心依赖和应用程序依赖 项目配置基础Maven依赖 添加Connector和库 依赖 Scala版本 Hadoop依赖 附录：用于构建具有依赖项的Jar的模板 使用Flink所需要的基础maven依赖包，还有一些数据来源的驱动，如Kafka Connector，还有保证Flink应用程序运行的Flink运行库。 核心依赖和应用程序依赖 Flink Core Dependen...
58、Flink 的项目配置使用 Maven 详解
猫猫爱吃小鱼粮
07-04 1758
Flink 的项目配置使用 Maven 详解
Flink依赖整理
qq_46411309的博客
07-22 5072
一、依赖下载地址Maven Repository https://mvnrepository.com/ 二、Flink的项目依赖 2.1Flink 核心依赖 Flink 核心依赖项包含系统运行所需的类和依赖项，在启动Flink应用程序时必须存在， 不包含任何连接器或库，保证尽可能小，避免依赖项冲突。 <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-core --> <dependency> <
JavaFlink系列之一：Maven程序搭建及Java入门案例多种写法
qq_44000055的博客
10-20 1788
一、Flink项目依赖配置。二、JavaFlink案例。
flink学习（一）---Maven配置简单易学
galaxiansheng的博客
07-13 1855
一、下载Maven 进入官网http://maven.apache.org/ 点击Download 进入官网之后： 我演示的是windows的配置步骤： 1.将安装包放入C盘，进行解压，将maven文件夹放入c盘的工程文件夹中ProgramFiles. 2.进入windows的环境变量编辑中，先编辑用户变量,创建：MAVEN_HOME，变量值是：maven工程目录的绝对路径。 3.在path中添加maven的bin文件夹的绝对路径 4.在系统变量中的Pat...
flink开发maven idea配置
weixin_38251332的博客
04-19 4637
pom.xml文件： <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.
Flink之小白Maven搭建和本地部署
打工日常
10-04 1099
Maven搭建 使用Flink官方提供的flink-quickstart-scala原型来创建我们的工程，因为Flink官方提供的pom.xml经过了精心配置，可以打包出很小的Jar包。 依赖包下载完毕后，会出现BUILD SUCCESS quickstart/ ├── pom.xml └── src └── main ├── resources │ └── log4j.properties └── scala └── org └── example └── quickstart ├──
Maven 构建 Flink 应用程序的最佳实践（根除各种类冲突/类加载问题）
Laurence的技术博客
04-27 2533
作为开发者，在构建 Flink 应用程序时的体验真是一言难尽，想必大家都曾遇到过各种 ClassNotFoundException、NoSuchMethodError 以及 Could not find any factory for identifier kafka/jdbc/hive/hudi that implements org.apache.flink.table.factories.DynamicTableFactory in the classpath 这样的错误。坦率地说，Flink 的应用
【Flink部署】java部署、本地部署(maven-shade工具)、平台部署（windows部署、 linux部署）；端口号修改
qq_47450919的博客
08-22 821
1.flink的java部署 01.直接main方法启动 02.idea中可以直接创建web界面并提交直接启动(了解) 2.flink的本地部署(打包成jar包,在黑窗口中运行) 01.maven-shade工具进行打包操作 3.flink的平台部署 01.windows部署(了解) 02.linux部署(掌握) 端口号修改
使用 Maven 构建 Flink 项目的正确姿势
qq_34412579的博客
10-11 4544
通过本文你能 get 到以下点： 使用 Maven 构建的 Flink 项目 module 如何规划 Shade 插件解决 Jar 包依赖冲突（不限于 Flink，Spark 同样适用） 使用 Maven 构建的 Flink 项目 module 如何规划 痛点 先从痛点开始讲起，通常由 Maven 来构建 Flink 项目，如下图所示，一般会按照业务来划分 module。 本项目是 z...
flink1.19 maven 依赖
02-18
### Flink 1.19 Maven Dependency Configuration Example For configuring Apache Flink version `1.19.1` within a Maven project, the following dependencies are essential and should be included in the `pom.xml`. These dependencies cover core functionalities as well as integrations with other systems such as Kafka, Paimon, Aliyun OSS, Hadoop shaded filesystems, and FastJSON. #### Core Dependencies The primary dependency is for Apache Flink itself: ```xml <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.12</artifactId> <version>${apache.flink.version}</version> </dependency> ``` This sets up the environment to work with stream processing capabilities provided by Flink[^1]. #### Integration Dependencies To integrate with specific technologies or services, additional dependencies can be added based on requirements: - **Kafka Connector** ```xml <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-connector-kafka_${scala.binary.version}</artifactId> <version>${flink-kafka.version}</version> </dependency> ``` - **Paimon Support** ```xml <dependency> <groupId>com.alibaba.paimon</groupId> <artifactId>paimon-flink</artifactId> <version>${apache.paimon.version}</version> </dependency> ``` - **Aliyun OSS SDK** ```xml <dependency> <groupId>com.aliyun.oss</groupId> <artifactId>aliyun-sdk-oss</artifactId> <version>${aliyun.oss.version}</version> </dependency> ``` - **Hadoop Shaded File System** ```xml <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-shaded-hadoop-3.3-uber-${fs.hadoopshaded.version}_2.12</artifactId> <version>${fs.hadoopshaded.version}</version> </dependency> ``` - **FastJSON Library** ```xml <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>${fastjson.version}</version> </dependency> ``` Each of these configurations ensures that all necessary libraries are available at compile time and runtime when working with Flink applications integrated into various data sources and sinks like Kafka, storage solutions like Aliyun OSS, serialization/deserialization tools like FastJSON, and file system operations through Hadoop compatibility layers. --related questions-- 1. How does one configure Scala versions specifically for Flink projects? 2. What changes occur between different minor releases of Flink regarding dependency management? 3. Can you provide an example setup integrating Flink with another real-time messaging platform besides Kafka? 4. Are there any known issues or limitations using this particular combination of library versions together? 5. In what scenarios would it not be advisable to use shaded artifacts over standard ones?
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值