elasticsearch命令XPOST上传json文件

最新推荐文章于 2024-09-08 20:18:03 发布

banyanmars

最新推荐文章于 2024-09-08 20:18:03 发布

阅读量2.7k

点赞数 1

CC 4.0 BY-SA版权

分类专栏： elasticsearch 文章标签： elasticsearch

本文链接：https://blog.youkuaiyun.com/banyanmars/article/details/93178869

elasticsearch 专栏收录该内容

2 篇文章

订阅专栏

博客介绍了通过特定网站生成json文件，进行格式转换后批量导入Elasticsearch的方法。同时列举了导入过程中遇到的多种错误，如Content-Type不支持、文件格式结尾问题、缺少特定字段、版本不允许一个index有多个type等，并给出相应的解决办法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

通过https://www.json-generator.com/生成json文件，生成模板如下：

[
  '{{repeat(1000, 1000)}}',[{
	"index": {
		"_id": '{{objectId()}}'
	}
}, {
	"account_number": '{{index()}}',
	"balance": '{{floating(1000, 4000, 2, "$0,0.00")}}',
	"firstname": '{{firstName()}}',
	"lastname": '{{surname()}}',
	"age": '{{integer(20, 40)}}',
	"gender": '{{bool()}}',
	"address": '{{integer(100, 999)}} {{street()}}, {{city()}}, {{state()}}, {{integer(100, 10000)}}',
	"employer": '{{firstName()}}',
	"email": '{{email()}}',
	"city": '{{city()}}',
	"state": '{{state()}}'
}]
]

下载json文件，需要转换格式，最后一行必须以\n结尾：

{"index":{"_id":"5d0c5220ae42606e9f4fd569"}}
{"account_number":1,"balance":"$1,906.15","firstname":"Bartlett","lastname":"Cunningham","age":37,"gender":true,"address":"714 Malta Street, Salix, Virginia, 128","employer":"Lakeisha","email":"lakeishacunningham@ersum.com","city":"Kirk","state":"Kentucky"}
{"index":{"_id":"5d0c5220ae42606e9f4fd569"}}
{"account_number":1,"balance":"$1,906.15","firstname":"Bartlett","lastname":"Cunningham","age":37,"gender":true,"address":"714 Malta Street, Salix, Virginia, 128","employer":"Lakeisha","email":"lakeishacunningham@ersum.com","city":"Kirk","state":"Kentucky"}

批量导入文件命令行为：

curl -H "Content-Type: application/json" -XPOST 'ip:9200/customer/account/_bulk?pretty' --data-binary @generated.json

错误解决：

命令为：

curl -H -XPOST 'ip:9200/customer/account/_bulk?pretty' --data-binary @generated.json
错误提示：

{
"error" : "Content-Type header [application/x-www-form-urlencoded] is not supported",
"status" : 406
}

解决：增加 -H "Content-Type: application/json"

错误提示：

"The bulk request must be terminated by a newline [\n]"
"status" : 400

原因：json文件格式最后一行不以\n结尾

错误提示：

Malformed action/metadata line [1], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]

原因：json文件没有{"index":{"_id":"1"}}

错误提示：

Rejecting mapping update to [customer] as the final mapping would have more than 1 type: [external, account]

原因：6.0的版本不允许一个index下面有多个type，并且官方说是在接下来的7.0版本中会删掉type

解决： curl -H "Content-Type: application/json" -XPOST '10.10.100.90:9200/customer/external/_bulk?pretty' --data-binary @generated.json

参考：https://endymecy.gitbooks.io/elasticsearch-guide-chinese/content/getting-started/exploring-data.html

https://www.cnblogs.com/tonglin0325/p/8446975.html

https://blog.youkuaiyun.com/gwd1154978352/article/details/82855837