elasticsearch ik分词器的安装和使用

本文详细介绍了如何安装和配置Elasticsearch的IK分词器,包括下载对应版本的软件包,解压并安装到ES节点的plugins目录下,处理版本不匹配问题,以及设置IK分词器生效和配置扩展词典。在遇到版本不兼容问题时,通过修改配置文件解决了错误。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

elasticsearch几种常用分词器如下:

分词器分词方式
StandardAnalyzer单字分词
CJKAnalyzer二分法
IKAnalyzer词库分词

其中常用的是IKAnalyzer,但IK是第三方插件,需要安装。

安装分词器

下载

IK软件包下载地址:
https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v5.6.16

在tags栏选择与自己es相同版本的包,如下图对应方式:
在这里插入图片描述

可以选择源码自己编译安装也可以选择编译好的版本安装。由于我是内网环境没有办法自己编译所以选择了已经编译好的版本。

安装

1、解压 elasticsearch-analysis-ik-5.6.16.zip
unzip elasticsearch-analysis-ik-5.6.16.zip
解压后得到一个名为elasticsearch的文件夹,内容如下:
在这里插入图片描述2、安装ik
在每个ES节点的,plugins目录下创建ik文件夹,将解压出来的文件全部拷贝到ik文件夹下。

mkdir -p ${your_es_dir}/plugins/ik
cp -a elasticsearch/* ${your_es_dir}/plugins/ik

3、重启ES。
由于做了修改,最好先前台重启一下,看下ES启动是否成功。
通常可能会遇到以下几种报错。
1) 5.3版本信息不全报错。

[2019-05-07T18:39:13,452][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [hdzx_elk_02] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Could not load plugin descriptor for existing plugin [ik]. Was the plugin built before 2.0?
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:58) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
Caused by: java.lang.IllegalStateException: Could not load plugin descriptor for existing plugin [ik]. Was the plugin built before 2.0?
        at org.elasticsearch.plugins.PluginsService.getPluginBundles(PluginsService.java:295) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:131) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.node.Node.<init>(Node.java:302) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.node.Node.<init>(Node.java:238) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]
        ... 6 more
Caused by: java.nio.file.NoSuchFileException: /home/admin/data/esdata/elasticsearch-5.3.0/plugins/ik/plugin-descriptor.properties
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[?:?]
        at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_111]
        at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0_111]
        at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384) ~[?:1.8.0_111]
        at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0_111]
        at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:86) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.plugins.PluginsService.getPluginBundles(PluginsService.java:292) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:131) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.node.Node.<init>(Node.java:302) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.node.Node.<init>(Node.java:238) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]


解决办法
以上报错是由于我的es是5.3版本的所以我最初下载的是5.3版本的ik包,根据关键词java.nio.file.NoSuchFileException: /home/admin/data/esdata/elasticsearch-5.3.0/plugins/ik/plugin-descriptor.properties可以判断出是因为plugins/ik路径下缺少必要组件,但5.3版本的包并没有提供,于是便下载了5.6.16的ik 将jar包换成5.3的jar包继续启动。

  1. 使用5.6.16的版本启动报错如下。
[2019-05-07T18:53:04,881][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [hdzx_elk_02] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Plugin [analysis-ik] is incompatible with Elasticsearch [5.3.0]. Was designed for version [5.6.16]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:58) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
Caused by: java.lang.IllegalArgumentException: Plugin [analysis-ik] is incompatible with Elasticsearch [5.3.0]. Was designed for version [5.6.16]
        at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:108) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.plugins.PluginsService.getPluginBundles(PluginsService.java:292) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:131) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.node.Node.<init>(Node.java:302) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.node.Node.<init>(Node.java:238) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]

解决办法
根据关键词org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Plugin [analysis-ik] is incompatible with Elasticsearch [5.3.0]. Was designed for version [5.6.16]判断是插件的版本与ES 版本不符,但是ik插件官方文档说明5.* 版本的ik插件可以支持5.*版本的ES,所以修改ik插件的配置文件即可。
${your_es_dir}/plugins/ik/plugin-descriptor.properties中修改如下配置,将5.6.16修改成5.3.0:
如下,将:

plugin-descriptor.properties:version=5.6.16
plugin-descriptor.properties:elasticsearch.version=5.6.16

改成

plugin-descriptor.properties:version=5.3.0
plugin-descriptor.properties:elasticsearch.version=5.3.0

重新启动ES即可。

配置IK分词器生效

1、ES5.*之前 可以直接在配置文件增加一行配置如下即可(我选择的是ik_smart,这个根据业务自行选择)

index.analysis.analyzer.default.type: "ik_smart"
但5.*之后,不支持在配置文件中做索引级别的配置,会报错如下:

*************************************************************************************
Found index level settings on node level configuration.

Since elasticsearch 5.x index level settings can NOT be set on the nodes 
configuration like the elasticsearch.yaml, in system properties or command line 
arguments.In order to upgrade all indices the settings must be updated via the 
/${index}/_settings API. Unless all settings are dynamic all indices must be closed 
in order to apply the upgradeIndices created in the future should use index templates 
to set default values. 

Please ensure all required values are updated on all indices by executing: 

curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
  "index.analysis.analyzer.default.type" : "ik_smart"
}'
*************************************************************************************

[2019-05-07T21:18:27,927][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [hdzx_elk_02] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: node settings must not contain any index level settings
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:58) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
Caused by: java.lang.IllegalArgumentException: node settings must not contain any index level settings
        at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:132) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.node.Node.<init>(Node.java:335) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.node.Node.<init>(Node.java:238) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap$6.<init>(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]
        ... 6 more

2、按照提示执行

curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
  "index.analysis.analyzer.default.type" : "ik_smart"
}'

会报错,返回json字符串,贴到json.cn里格式化一下,会发现主要报错如下:

"Can't update non dynamic settings [[index.analysis.analyzer.default.type]] for open indices

解决办法如下

curl -XPOST 'localhost:9200/_all/_close'
curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{"index.analysis.analyzer.default.type" : "ik_smart"}'
curl -XPOST 'localhost:9200/_all/_open'

配置扩展词典

${your_es_dir}/plugins/ik/config/IKAnalyzer.cfg.xml中如图框出来的位置增加字典。

在这里插入图片描述所添加的字典文件中,添加你要扩展的字典。

在这里插入图片描述

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值