elasticSearch
原理
倒排索引
词语->文档(位置,---信息) 像搜索引擎
”中国“ -》文档1 位置 2 ...
IK分词器
中国人民-中国 、人民
粗分 细分
安装
提前装好npm
ElasticSearch: https://mirrors.huaweicloud.com/elasticsearch/?C=N&O=D logstash: https://mirrors.huaweicloud.com/logstash/?C=N&O=D kibana: https://mirrors.huaweicloud.com/kibana/?C=N&O=D
ik分词器:https://github.com/medcl/elasticsearch-analysis-ik
ik分词器解压复制到elasticSearch plugin 里,也可以定义自己的词典 xx.dic
kibana 汉化配置
head可视化工具
启动
cnpm install
npm run start http://localhost:9100/
elasticsearch 启动
启动 elasticsearch.bat http://localhost:9200/
跨域配置 elasticsearch.yml
http.cors.enabled: true http.cors.allow-origin: "*"
kibana 启动
启动 kibana.bat
启动日志 启动报错:在 kibana.yml 中加入
network.bind_host:127.0.0.1
开发工具:kibana控制台-小扳手那里
查询短语
分词
GET _analyze { "analyzer": "ik_max_word", "text": "中国家" } GET _analyze { "analyzer": "ik_smart", "text": "中国家" } GET _analyze { "analyzer": "ik_max_word", "text": "中国家" }
添加
官方现在好像不推荐使用type了就是下面对应的table1可以删掉?? /**put /数据库/哪个表/id**/ PUT /zly/table1/3 { "name":"zhou3" ,"age":30 } /**获取 /数据库 **/ GET zly2 PUT /zly2/table2/1 { "age":123 } DELETE zly2 /**创建数据库并设置类型 */ PUT mytestdb { "mappings": { "properties": { "name":{ "type": "text" }, "age":{ "type": "integer" }, "sign":{ "type": "text" } } } } GET mytestdb/_doc/1 GET mytestdb/_search?q=name:三 /**更新 */ POST mytestdb/_doc/1/_update { "doc":{"name":"张三22"} } /**创建数据库并设置类型 */ PUT usr { "mappings": { "properties": { "name":{ "type": "text" }, "age":{ "type": "integer" }, "desc": { "type": "text" } } } } /**插入一条数据 */ PUT usr/_doc/1 { "name": "邓小胖", "age": 3, "desc": "一波操作猛如虎,一看工资两万五" } /**插入一条数据 */ PUT usr/_doc/2 { "name": "邓小胖说", "age": 10, "desc": "一波操作猛如虎,一看工资三万五" } /**插入一条数据 */ PUT usr/_doc/3 { "name":"邓小胖说Java", "age": 20, "desc": "一波操作猛如虎,一看工资五万五" }
查询
/*查询分页 from size*/ get usr/_search { "query":{ "match":{ "name":"邓小胖" } }, "_source":["age","desc","name"] ,"sort":{ "age":{"order":"desc"} } ,"from":0 ,"size":2 } /**查询bool-must必须有-should(or)-must_not(不包含) */ get usr/_search { "query": { "bool": { "must_not": [ { "match": { "name": "小胖" } }, { "match": { "age": 20 } } ] } } } /**查询高亮 */ get usr/_search { "query": { "bool": { "should": [ { "match": { "name": "小胖" } }, { "match": { "age": 20 } } ] } } ,"highlight":{ "fields": {"name": {}}, "pre_tags": "<span class='color:red' >", "post_tags": "</span>" } } /**查询精准查询 */ GET usr/_search { "query": { "term": { "age": 30 } } }
官方文档地址
更新 :
springboot
官方文档
https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#reference
The TransportClient
is deprecated as of Elasticsearch 7 and will be removed in Elasticsearch 8. (see the Elasticsearch documentation). Spring Data Elasticsearch will support the TransportClient
as long as it is available in the used Elasticsearch version but has deprecated the classes using it since version 4.0.
We strongly recommend to use the High Level REST Client instead of the TransportClient
.
依赖
<properties> <java.version>1.8</java.version> <elasticsearch.version>7.6.2</elasticsearch.version> </properties> <!--es客户端--> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>7.6.2</version> </dependency> <!--springboot的elasticsearch服务--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency>
客户端引入
package com.dzhou.ssmdemo.coinfig; import org.apache.http.HttpHost; import org.elasticsearch.client.RestClient; import org.elasticsearch.client.RestHighLevelClient; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration public class ElasticSearchConfigClient { @Bean public RestHighLevelClient restHighLevelClient() { return new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); } }
API
package com.dzhou.ssmdemo; import cn.hutool.json.JSONUtil; import com.dzhou.ssmdemo.testcode.entity.News; import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest; import org.elasticsearch.action.delete.DeleteRequest; import org.elasticsearch.action.delete.DeleteResponse; import org.elasticsearch.action.index.IndexRequest; import org.elasticsearch.action.search.SearchRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.action.support.master.AcknowledgedResponse; import org.elasticsearch.action.update.UpdateRequest; import org.elasticsearch.action.update.UpdateResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.client.indices.CreateIndexRequest; import org.elasticsearch.client.indices.CreateIndexResponse; import org.elasticsearch.common.xcontent.XContentType; import org.elasticsearch.index.query.BoolQueryBuilder; import org.elasticsearch.index.query.QueryBuilders; import org.elasticsearch.search.SearchHit; import org.elasticsearch.search.SearchHits; import org.elasticsearch.search.builder.SearchSourceBuilder; import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import java.io.IOException; import java.util.Date; @RunWith(SpringJUnit4ClassRunner.class) @SpringBootTest(classes = SsmdemoApplication.class) public class ElasticsearchTest { @Autowired public RestHighLevelClient restHighLevelClient; /** * 创建索引 * * @throws IOException */ @Test public void testCreateIndex() throws IOException { CreateIndexRequest createIndexRequest = new CreateIndexRequest("spring-test"); CreateIndexResponse response = this.restHighLevelClient.indices().create(createIndexRequest, RequestOptions.DEFAULT); System.out.println(response.index()); } /** * 删除索引 */ @Test public void testDeleteIndex() throws IOException { DeleteIndexRequest del = new DeleteIndexRequest("spring-test"); AcknowledgedResponse response = this.restHighLevelClient.indices().delete(del, RequestOptions.DEFAULT); System.out.println(response.isAcknowledged()); } /*创建文档*/ @Test public void testCreateDocument() throws IOException { IndexRequest request = new IndexRequest("spring-test"); request.id("3"); News news = new News(); news.setId(3); news.setTag("3"); news.setPublishTime(new Date()); news.setTitle("新闻3"); request.source(JSONUtil.toJsonStr(news), XContentType.JSON); this.restHighLevelClient.index(request, RequestOptions.DEFAULT); } /*更新文档*/ /*https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-document-update.html*/ @Test public void testUpdateDocument() throws IOException { UpdateRequest updateRequest = new UpdateRequest("spring-test", "3"); News news = new News(); news.setId(3); news.setTag("3gai"); news.setPublishTime(new Date()); news.setTitle("新闻3+1"); updateRequest.doc(JSONUtil.toJsonStr(news), XContentType.JSON); UpdateResponse update = this.restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT); System.out.println(update.getGetResult()); } @Test /*删除文档*/ public void testDeleteDocument() throws IOException { DeleteRequest deleteRequest=new DeleteRequest("spring-test"); deleteRequest.id("3"); DeleteResponse result = this.restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT); System.out.println(result.getResult()); } @Test /*查询文档*/ public void testQueryDocument() throws IOException { SearchRequest searchRequest=new SearchRequest("spring-test"); SearchSourceBuilder builder=new SearchSourceBuilder(); /*高亮*/ HighlightBuilder highlight=new HighlightBuilder(); highlight.preTags("<em>"); highlight.postTags("</em>"); builder.highlighter(highlight); /*匹配查询*/ builder.query(QueryBuilders.matchQuery("tag","3")); /*精准查询*/ // builder.query(QueryBuilders.termQuery("tag","3")); searchRequest.source(builder); SearchResponse result = this.restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); SearchHits hist = result.getHits(); SearchHit[] hists = hist.getHits(); for (SearchHit hit:hists ) { System.out.println(hit.getSourceAsString()); } } @Test /*高级查询布尔查询*/ public void testHighQueryDocument() throws IOException { SearchRequest searchRequest=new SearchRequest("spring-test"); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); BoolQueryBuilder boolquery= QueryBuilders.boolQuery(); searchSourceBuilder.query(boolquery); /*高亮*/ HighlightBuilder highlight=new HighlightBuilder(); highlight.preTags("<em>"); highlight.postTags("</em>"); searchSourceBuilder.highlighter(highlight); //或者 boolquery.should(QueryBuilders.matchQuery("title","新闻")); //必须 boolquery.must(QueryBuilders.termQuery("tag","3")); searchRequest.source(searchSourceBuilder); SearchResponse result = this.restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT); SearchHits hist = result.getHits(); SearchHit[] hists = hist.getHits(); System.out.println("查询结果"+hists.length); for (SearchHit hit:hists ) { System.out.println(hit.getSourceAsString()); } } /**/ }
查询 searchRequest博客例子
https://blog.youkuaiyun.com/qq_2300688967/article/details/83902943
https://blog.youkuaiyun.com/guohao_1/article/details/89951192 [全一点]
官方文档查询
RestHighLevelClient
官方文档
[ElasticsearchRestTemplate] 主方法在core里
https://docs.spring.io/spring-data/elasticsearch/docs/current/api/
new ElasticsearchRestTemplate(new RestHighLevelClient ());
springBoot中的template也就是对client进行了一层封装
安装ELK
https://my.oschina.net/itblog/blog/547250
logstash下载地址
https://repo.huaweicloud.com/logstash/7.5.0/
数据:servers[多个]=>log4j]=>logstash收集]=>导入到elasticsearch]=>kibana可视搜索
启动 logstash -f config/log4j.conf
filebean下载地址
https://repo.huaweicloud.com/filebeat/5.5.0/
安装修改
https://blog.youkuaiyun.com/dwyane__wade/article/details/80169051
启动 filebeat -e -c filebeat.yml
https://www.cnblogs.com/daijiting/p/10250374.html
https://blog.youkuaiyun.com/wsdc0521/article/details/106236499