一个基于Telegraf+InfluxDB+Grafana的数据采集、存储和可视化分析解决方案

1 引言

不管是在互联网，大数据，物联网 还是工业大数据领域，数据的采集、预处理、存储和可视化展示分析都是不可或缺的功能。

如下图所示的一个数据全生命周期服务图：

本文提供一种 基于Telegraf+InfluxDB+Grafana的数据采集、存储和可视化分析解决方案。

• Telegraf is used to pre-process data according to customer requirements, and convert to the unified TS DB format.
• InfluxDB is a time series database which is used to store the data, and it is also used to provide uniform SQL interfaces to external applications (like the simulation/Machine Learning).
• Grafana is used to provide the visualization and monitor function

2 软件介绍

 

1 Telegraf

Web site:  https://www.influxdata.com/time-series-platform/telegraf/

Telegraf is a plugin-driven server agent for collecting, pre-parsing and sending metrics and events from systems, files and IoT sensors. Here gives a software architecture of telegraf:

 

2 InfluxDB

https://www.influxdata.com/products/influxdb-overview/

InfluxDB is an open source time series database. It is designed to handle high write and query loads and provides a SQL-like query language called InfluxQL for interacting with data.

3 Grafana

https://grafana.com/

Grafana is an open source analytics & monitoring solution for every database. It’s completely open source, and backed by a vibrant community. It’s developed by GO, and is very easy to installation.

3.1 Support 60+ visualization plugins

 

 

 

3.2 Support 30+ Dada sources

It supports TSDB, SQL DD, Cloud, logging&document DB etc.

 

3.3 Support almost all the measuring units

      

 

3.4 Support Alert notification

If the data values across one threshold, Grafana can send out alert notifications.

It supports 15+ notification types like Email, Dingding etc.

 

3.5 User authority management

It has full user authority management function.

4 Installation and configuration

It’s very easy to install and configure the software, because they are developed by GO language which doesn’t have dependencies on other software.

4.1 telegraf

4.1.1 download package

> wget https://dl.influxdata.com/telegraf/releases/telegraf_1.13.0-1_amd64.deb

4.1.2 install

> sudo dpkg -i telegraf_1.13.0-1_amd64.deb

4.1.3 start

> sudo systemctl start telegraf

> sudo systemctl enable telegraf

> sudo systemctl restart telegraf

> sudo systemctl status telegraf

 

4.1.4 configuration

Edit file /etc/telegraf/telegraf.conf

Details refer to https://docs.influxdata.com/telegraf/v1.13/

 

# String fields must be added to this array

json_string_fields = []

 

 

After edit the files, run “sudo systemctl restart telegraf” to restart telegraf.

4.1.5 code

https://github.com/influxdata/telegraf

 

4.2 InfluxDB

4.2.1 Download package

> wget https://dl.influxdata.com/influxdb/releases/influxdb_1.7.9_amd64.deb

4.2.2 install

> sudo dpkg -i influxdb_1.7.9_amd64.deb

4.2.3 start

> sudo systemctl start influxdb

> systemctl enable influxdb //auto start influxdb when reboot

> sudo systemctl status influxdb //check the status

4.2.4 Configuration & usage

Run “influx” to enter influxDB

> show users

> create user "tsuser" with password 'tsuser'

> show databases;

> create database tsDB

> use tsDB

> show measurements

> select * from TableName limit 1 

> drop measurement execlocGeo   //delete one table

Details refer to https://docs.influxdata.com/influxdb/v1.7/

4.2.5 Code

https://github.com/influxdata/influxdb

4.3 Grafana

4.3.1 Download packages

 > wget https://dl.grafana.com/oss/release/grafana_6.5.2_amd64.deb

4.3.2 install

 > sudo dpkg -i grafana_6.5.2_amd64.deb

4.3.3 start

> sudo /bin/systemctl daemon-reload

 > sudo /bin/systemctl enable grafana-server

 > sudo /bin/systemctl start grafana-server

4.3.4 Configuration & usage

 Access URL http://localhost:3000

 Login admin/admin

 Then configure the page

Details: https://grafana.com/docs/grafana/latest/

4.3.5 code

https://github.com/grafana/grafana