使用Prometheus+grafana+blackbox监控ICMP

安装proemtheus

1.下载prometheus-2.13.0.linux-386.tar.gz

mkdir /data/prometheus
cd /data/prometheus
wget https://github.com/prometheus/prometheus/releases/download/v2.13.0/prometheus-2.13.0.linux-386.tar.gz
tar xf prometheus-2.13.0.linux-386.tar.gz 
mv prometheus-2.13.0.linux-386 prometheus

安装blackbox-exporter模块，来进行icmp+http+ssl监控

blackbox_exporter 是 Prometheus 官方提供的 exporter之一，可以提供 http、dns、tcp、icmp（ping）的监控数据采集。

1.下载blackbox-export二进制文件

cd /data/prometheus/
wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.15.1/blackbox_exporter-0.15.1.linux-amd64.tar.gz
tar xf blackbox_exporter-0.15.1.linux-amd64.tar.gz
mv blackbox_exporter-0.15.1.linux-amd64.tar.gz blackbox_ecporter

安装Grafana，来图形化显示数据

Grafana 是开源的,功能齐全的度量仪表盘和图形编辑器,支持 Graphite,OpenTSDB,Elasticsearch,Cloudwatch,Prometheus,InfluxDB,小米监控
等。
Grafana的默认端口是:3000
使用yum进行安装

yum install initscripts urw-fonts
yum -y install grafana
systemctl daemon-reload
systemctl start grafana-servcer.service
systemctl enable grafana-server.service

安装完成后，默认的用户名和登陆密码都是：admin/admin

修改相关配置文件

ICMP： 配置文件修改

cd /data/prometheus
vim /prometheus/prometheus.yml
    # 全局配置选项
    global:
      scrape_interval:     5s 
      evaluation_interval: 3s 

    # 配置告警方式
    alerting:
      alertmanagers:
      - static_configs:
        - targets:
          # - alertmanager:9093

    # 指定告警规则
    rule_files:
      # - "first_rules.yml"
      # - "second_rules.yml"


    # 配置被监控端，就是配置我们需要监控点额东西
    scrape_configs:

      # 指定job的名称
      - job_name: 'prometheus'
        scrape_interval: 3s
        metrics_path: '/api/v1/prometheus/metrics'
        static_configs:
        - targets: ['192.168.50.222:9090']


## 下面的内容是需要新添加到配置文件里面，主要用于测试服务器的连通性
      - job_name: 'ping_all'
        scrape_interval: 3s
        metrics_path: /metrics
        params:
          module: [icmp] #ping
        static_configs:
          - targets: ['ipaddress']   # 在配置是需要将ipaddress 设置为需要监控的IP
            labels:
              group: '电信泉州'
        relabel_configs:
          - source_labels: [__address__]
            regex: (.*)(:80)?
            target_label: __param_target
            replacement: ${1}
          - source_labels: [__param_target]
            regex: (.*)
            target_label: ping
            replacement: ${1}
          - source_labels: []
            regex: .*
            target_label: __address__
            replacement: 192.168.50.222:9115    # 上面的9115是blackbox程序的启动端口

TCP： 配置文件修改

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
    
  - job_name: '本地 --> QingDao'
    scrape_interval: 15s
    metrics_path: /probe
    params:
      module: [tcp_connect]
    static_configs:
      - targets: ['ipaddress:port']   # 在配置是需要将ipaddress:port 设置为需要监控的IP:port
       labels:
          group: 'QingDao-server'
    relabel_configs:
      - source_labels: [__address__]
        #regex: (.*)(:80)?
        target_label: __param_target
       #replacement: ${1}
      - source_labels: [__param_target]
        #regex: (.*)
        target_label: instance
        #replacement: ${1}
      - source_labels: []
        #regex: .*
        target_label: __address__
        replacement: 192.168.50.222:9115    # 上面的9115是blackbox程序的启动端口
    
    
  - job_name: '东莞移动 --> HongKong'
    scrape_interval: 15s
    metrics_path: /probe
    params:
      module: [tcp_connect]
    static_configs:
      - targets:                        # 设置为需要监控的IP:port
          - ipaddress:port
          - ipaddress:port
          - ipaddress:port
        labels:
          group: 'HongKong-server'
    relabel_configs:
      - source_labels: [__address__]
       #regex: (.*)(:80)?
        target_label: __param_target
        #replacement: ${1}
      - source_labels: [__param_target]
        #regex: (.*)
        target_label: instance
        #replacement: ${1}
      - source_labels: []
        #regex: (.*)
        target_label: instance
        #replacement: ${1}
      - source_labels: []
        #regex: .*
        target_label: __address__
        replacement: xxx.xxx.xxx.xxx:9115

启动服务

systemctl restart grafana-server.service
/root/
/data/prometheus/blackbox_exporter/blackbox_exporter --config.file=/data/prometheus/blackbox_exporter/blackbox.yml &
/data/prometheus/prometheus --config.file=/data/prometheus/prometheus.yml   &

验证是否正常启用

http://192.168.50.222:9090/tages

相关web后台操作可视化图形见下一篇文章

错误如何在Prometheus中修复’\“ INVALID \”不是有效的开始令牌”

在http://192.168.50.222:9090/targets发现不是最初始令牌
解决方法
在配置文件里面将下面的设置为默认的metrics

metrics_path: /metrics

在prometheus, version 2.12.0中使用icmp进行ping测的时候，可以使用下面的语法：

static_configs:
      - targets: ['ipaddress', 'ipaddress', 'ipaddress']             # 设置为需要监控的IP:port
        labels:
          group: '电信泉州'
    relabel_configs:
      - source_labels: [__address__]
        regex: (.*)(:80)?
        target_label: __param_target
        replacement: ${1}
      - source_labels: [__param_target]
        regex: (.*)
        target_label: ping
        replacement: ${1}
      - source_labels: []
        regex: .*
        target_label: __address__
        replacement: 192.168.50.222:9115

## 需要注意的是：建议使用static_configs格式。