Hive UDF 环境搭建(Eclipse+Maven)

1. 安装Maven (https://blog.youkuaiyun.com/rav009/article/details/79469303)
2. 安装Eclipse
3. 安装Eclipse的Maven插件 m2e

 

使用Eclipse创建Maven项目

Group ID一般是org.yourname.projectname, Group ID会变成你代码中类的前缀

Artifact ID是Projectname, 就是项目名称

 

创建项目后找到pom.xml, 在dependencies节点里添加:

    <dependency>
    	<groupId>org.apache.hive</groupId>
    	<artifactId>hive-exec</artifactId>
    	<version>2.3.2</version>
    </dependency>

版本号根据hive的情况修改, 我写这篇文章的时候 hive已经有2.3.3了

 

来到项目目录下, 这个目录里应该有pom.xml, 运行命令行

mvn install

如果命令行报错 ,说某个jar包 invalid LOC header (bad signature), 就去repository里删掉这个jar包的文件夹,重新运行命令行, 会自动重新下载. 对于ubuntu来说repository在~/.m2

在src/main/java下添加新文件HelloWorld.java,代码如下:

package cn.pywei.HiveUDF;

import org.apache.hadoop.hive.ql.exec.Description;
import org.apache.hadoop.hive.ql.exec.UDF;

@Description(name="HelloWorld",value="_FUNC_(input), return the string \"HelloWorld\".",extended ="E.g. \n select hello(1);")


public class HelloWorld extends UDF {


	public String evaluate(String s) {
		return "HelloWorld";
	}
	
	public String evaluate(int s) {
		return "HelloWorld";
	}
	
	public String evaluate(boolean s) {
		return "HelloWorld";
	}
}

 

export成jar文件

 

在Hive中导入jar文件:

add jar /path/name.jar;

在Hive中创建临时函数:

create temporary function hello as 'cn.pywei.HiveUDF.HelloWorld';

执行:

select hello(1);
select hello('abc');
select hello(True);
describe function hello;
describe function extended hello;

 

此外还可以用以下命令操作jar包:

list jar;
delete jar /path/name.jar;
delete jar; --delete all jar;

 

maven的一些小问题:

Import cannot be resolved in eclispe project using maven?

Go to Project => check Build automatically and Clean.

If this doesn't solve the problem..

Right click the "Maven Dependencies" => "Build Path" => "Remove from the build path";
Right click the project, go to "Maven" => "Update project";

转自: https://stackoverflow.com/questions/29206772/import-cannot-be-resolved-in-eclispe-project-using-maven

 

Pom.xml中scope项 compile 和 provided 的区别?

Dependency Scope

Dependency scope is used to limit the transitivity of a dependency, and also to affect the classpath used for various build tasks.

There are 6 scopes available:

• compile
  This is the default scope, used if none is specified. Compile dependencies are available in all classpaths of a project. Furthermore, those dependencies are propagated to dependent projects.
• provided
  This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime. For example, when building a web application for the Java Enterprise Edition, you would set the dependency on the Servlet API and related Java EE APIs to scope provided because the web container provides those classes. This scope is only available on the compilation and test classpath, and is not transitive.
• runtime
  This scope indicates that the dependency is not required for compilation, but is for execution. It is in the runtime and test classpaths, but not the compile classpath.
• test
  This scope indicates that the dependency is not required for normal use of the application, and is only available for the test compilation and execution phases. This scope is not transitive.
• system
  This scope is similar to provided except that you have to provide the JAR which contains it explicitly. The artifact is always available and is not looked up in a repository.
• import (only available in Maven 2.0.9 or later)
  This scope is only supported on a dependency of type pom in the <dependencyManagement> section. It indicates the dependency to be replaced with the effective list of dependencies in the specified POM's <dependencyManagement> section. Since they are replaced, dependencies with a scope of import do not actually participate in limiting the transitivity of a dependency.

转自: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html

 

 

 

参考链接:

https://blog.youkuaiyun.com/u010376788/article/details/50532166

https://www.jianshu.com/p/7ebc8f9c9b78

http://www.crazyant.net/2160.html