elasticsearch批量导入数据注意事项

刚刚初始化启动kiabna后是没有索引的，当然，如果elasticsearch中导入过数据那么kibana会自动匹配索引

现在按照官方例子开始批量给elasticsearch导入数据

链接如下https://www.elastic.co/guide/en/kibana/6.1/tutorial-load-dataset.html

我们会依次导入如下 三块数据 

1.The Shakespeare data 莎士比亚文集的数据结构

{

    "line_id": INT,

    "play_name": "String",

    "speech_number": INT,

    "line_number": "String",

    "speaker": "String",

    "text_entry": "String",

}

2.The accounts data  账户数据结构

{

    "account_number": INT,

    "balance": INT,

    "firstname": "String",

    "lastname": "String",

    "age": INT,

    "gender": "M or F",

    "address": "String",

    "employer": "String",

    "email": "String",

    "city": "String",

    "state": "String"

}

3.The schema for the logs data 日志数据

{

    "memory": INT,

    "geo.coordinates": "geo_point"

    "@timestamp": "date"

}

然后向elasticsearch设置字段映射

Use the following command in a terminal (eg bash) to set up a mapping for the Shakespeare data set:

以下是莎士比亚的字段映射 可以用postman或者curl等发出请求~完整的url应该是localhost:9200/shakespear

PUT /shakespeare

{

 "mappings": {

  "doc": {

   "properties": {

    "speaker": {"type": "keyword"},

    "play_name": {"type": "keyword"},

    "line_id": {"type": "integer"},

    "speech_number": {"type": "integer"}

   }

  }

 }

}

Use the following commands to establish geo_point mapping for the logs:

这是 logs的字段映射

PUT /logstash-2015.05.18

{

  "mappings": {

    "log": {

      "properties": {

        "geo": {

          "properties": {

            "coordinates": {

              "type": "geo_point"

            }

          }

        }

      }

    }

  }

}

 

PUT /logstash-2015.05.19

{

  "mappings": {

    "log": {

      "properties": {

        "geo": {

          "properties": {

            "coordinates": {

              "type": "geo_point"

            }

          }

        }

      }

    }

  }

}

COPY AS CURLVIEW IN CONSOLE 

PUT /logstash-2015.05.20

{

  "mappings": {

    "log": {

      "properties": {

        "geo": {

          "properties": {

            "coordinates": {

              "type": "geo_point"

            }

          }

        }

      }

    }

  }

}

账户信息没有字段映射。。。

现在批量导入

curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/bank/account/_bulk?pretty' --data-binary @accounts.json

curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/doc/_bulk?pretty' --data-binary @shakespeare_6.0.json

curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/_bulk?pretty' --data-binary @logs.jsonl

windows下的curl命令可以到https://curl.haxx.se/download.html#Win64下载，解压后设置环境变量即可

这里要注意的是 @accounts.json，@shakespeare_6.0.json，@logs.json这些文件的位置应该是你所在的当前目录，

如果你当前位置是D盘~那么这些文件位置就要放在D盘下，否则读不到

还有一点~~~windows下要把命令行中的单引号换成双引号，，。。。否则会报

curl: (6) Could not resolve host: application这样的错误