zabbix3.4監(jiān)控zookeeper

zookeeper監(jiān)控要點(diǎn)系統(tǒng)監(jiān)控 這個(gè)監(jiān)控linux系統(tǒng)以及修改linux服務(wù)器參數(shù)即可
內(nèi)存使用量 ZooKeeper應(yīng)當(dāng)完全運(yùn)行在內(nèi)存中，不能使用到SWAP。Java Heap大小不能超過可用內(nèi)存。
Swap使用量 使用Swap會(huì)降低ZooKeeper的性能，設(shè)置vm.swappiness = 0
網(wǎng)絡(luò)帶寬占用 如果發(fā)現(xiàn)ZooKeeper性能降低關(guān)注下網(wǎng)絡(luò)帶寬占用情況和丟包情況，通常情況下ZooKeeper是20%寫入80%讀入
磁盤使用量 ZooKeeper數(shù)據(jù)目錄使用情況需要注意
磁盤I/O ZooKeeper的磁盤寫入是異步的，所以不會(huì)存在很大的I/O請(qǐng)求，如果ZooKeeper和其他I/O密集型服務(wù)公用應(yīng)該關(guān)注下磁盤I/O情況

ZooKeeper監(jiān)控
zk_avg/min/max_latency 響應(yīng)一個(gè)客戶端請(qǐng)求的時(shí)間，建議這個(gè)時(shí)間大于10個(gè)Tick就報(bào)警 平均延遲/最小延遲/最大延遲
zk_outstanding_requests 排隊(duì)請(qǐng)求的數(shù)量，當(dāng)ZooKeeper超過了它的處理能力時(shí)，這個(gè)值會(huì)增大，建議設(shè)置報(bào)警閥值為10 堆積請(qǐng)求數(shù)
zk_packets_received 接收到客戶端請(qǐng)求的包數(shù)量 收包數(shù)
zk_packets_sent 發(fā)送給客戶單的包數(shù)量，主要是響應(yīng)和通知 發(fā)包數(shù)
zk_max_file_descriptor_count 最大允許打開的文件數(shù)，由ulimit控制 最大文件描述符數(shù)量
zk_open_file_descriptor_count 打開文件數(shù)量，當(dāng)這個(gè)值大于允許值得85%時(shí)報(bào)警 打開的文件描述符數(shù)量
Mode 運(yùn)行的角色，如果沒有加入集群就是standalone,加入集群式follower或者leader
zk_followers leader角色才會(huì)有這個(gè)輸出,集合中follower的個(gè)數(shù)。正常的值應(yīng)該是集合成員的數(shù)量減1 follower數(shù)量
zk_pending_syncs leader角色才會(huì)有這個(gè)輸出，pending syncs的數(shù)量 準(zhǔn)備同步數(shù)
zk_znode_count znodes的數(shù)量 znode數(shù)量
zk_watch_count watches的數(shù)量 watch數(shù)量
Java Heap Size ZooKeeper Java進(jìn)程的

監(jiān)控腳本

[root@lanzhu-linux-nginx-cn summer]# cat check_zookeeper.sh
#!/bin/bash

function imok {
        echo ruok|nc 127.0.0.1 2181 | wc -l
}

function  zk_min_latency {
        echo mntr | nc 127.0.0.1 2181 | grep “$1” |awk ‘{print $2}’| awk “NR == 4” 2>/dev/null
}

function  zk_avg_latency {
        echo mntr | nc 127.0.0.1 2181 | grep “$1” |awk ‘{print $2}’| awk “NR == 2” 2>/dev/null
}

function  zk_max_latency {
        echo mntr | nc 127.0.0.1 2181 | grep “$1” |awk ‘{print $2}’| awk “NR == 3” 2>/dev/null
}

function  zk_outstanding_requests {
        echo mntr | nc 127.0.0.1 2181 | grep “$1” |awk ‘{print $2}’| awk “NR == 8” 2>/dev/null
}

function  zk_packets_received {
        echo mntr | nc 127.0.0.1 2181 | grep “$1” |awk ‘{print $2}’| awk “NR == 5” 2>/dev/null
}

function  zk_packets_sent {
        echo mntr | nc 127.0.0.1 2181 | grep “$1” |awk ‘{print $2}’| awk “NR == 6” 2>/dev/null
}

function  zk_znode_count {
        echo mntr | nc 127.0.0.1 2181 | grep “$1” |awk ‘{print $2}’| awk “NR == 10” 2>/dev/null
}

function  zk_watch_count {
        echo mntr | nc 127.0.0.1 2181 | grep “$1” |awk ‘{print $2}’| awk “NR == 11” 2>/dev/null
}

#excute function
$1

zabbix_agentd.conf配置文件添加

UserParameter=zookeeper.status[*],/alidata/summer/check_zookeeper.sh $1

zabbix-web端添加zookeeper模板見附件

<?xml version=”1.0″ encoding=”UTF-8″?>
<zabbix_export>
<version>3.4</version>
<date>2018-09-28T08:08:15Z</date>
<groups>
<group>
<name>Templates</name>
</group>
</groups>
<templates>
<template>
<template>Template App zookeeper</template>
<name>Template App zookeeper</name>
<description/>
<groups>
<group>
<name>Templates</name>
</group>
</groups>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<items>
<item>
<name>zookeeper status running</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[imok]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>zookeeper status zk_avg_latency</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[zk_avg_latency]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>zookeeper status zk_max_latency</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[zk_max_latency]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>zookeeper status zk_min_latency</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[zk_min_latency]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>zookeeper status zk_outstanding_requests</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[zk_outstanding_requests]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>zookeeper status zk_packets_received</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[zk_packets_received]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>zk_packets_sent</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[zk_packets_sent]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>zk_watch_count</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[zk_watch_count]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>zookeeper status zk_znode_count</name>
<type>0</type>
<snmp_community/>
<snmp_oid/>
<key>zookeeper.status[zk_znode_count]</key>
<delay>60</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>zookeeper</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
</items>
<discovery_rules/>
<httptests/>
<macros/>
<templates/>
<screens/>
</template>
</templates>
<triggers>
<trigger>
<expression>{Template App zookeeper:zookeeper.status[imok].last()}<>0</expression>
<recovery_mode>1</recovery_mode>
<recovery_expression>{Template App zookeeper:zookeeper.status[imok].diff()}=0</recovery_expression>
<name>php_39.108.161.17_zookeeper is down</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>3</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger>
</triggers>
<graphs>
<graph>
<name>zookeeper request time</name>
<width>900</width>
<height>200</height>
<yaxismin>0.0000</yaxismin>
<yaxismax>100.0000</yaxismax>
<show_work_period>1</show_work_period>
<show_triggers>1</show_triggers>
<type>0</type>
<show_legend>1</show_legend>
<show_3d>0</show_3d>
<percent_left>0.0000</percent_left>
<percent_right>0.0000</percent_right>
<ymin_type_1>0</ymin_type_1>
<ymax_type_1>0</ymax_type_1>
<ymin_item_1>0</ymin_item_1>
<ymax_item_1>0</ymax_item_1>
<graph_items>
<graph_item>
<sortorder>0</sortorder>
<drawtype>0</drawtype>
<color>1A7C11</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[zk_avg_latency]</key>
</item>
</graph_item>
<graph_item>
<sortorder>1</sortorder>
<drawtype>0</drawtype>
<color>F63100</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[zk_max_latency]</key>
</item>
</graph_item>
<graph_item>
<sortorder>2</sortorder>
<drawtype>0</drawtype>
<color>2774A4</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[zk_min_latency]</key>
</item>
</graph_item>
</graph_items>
</graph>
<graph>
<name>zookeeper server status</name>
<width>900</width>
<height>200</height>
<yaxismin>0.0000</yaxismin>
<yaxismax>100.0000</yaxismax>
<show_work_period>1</show_work_period>
<show_triggers>1</show_triggers>
<type>0</type>
<show_legend>1</show_legend>
<show_3d>0</show_3d>
<percent_left>0.0000</percent_left>
<percent_right>0.0000</percent_right>
<ymin_type_1>0</ymin_type_1>
<ymax_type_1>0</ymax_type_1>
<ymin_item_1>0</ymin_item_1>
<ymax_item_1>0</ymax_item_1>
<graph_items>
<graph_item>
<sortorder>0</sortorder>
<drawtype>0</drawtype>
<color>1A7C11</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[zk_packets_sent]</key>
</item>
</graph_item>
<graph_item>
<sortorder>1</sortorder>
<drawtype>0</drawtype>
<color>F63100</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[zk_watch_count]</key>
</item>
</graph_item>
<graph_item>
<sortorder>2</sortorder>
<drawtype>0</drawtype>
<color>2774A4</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[zk_outstanding_requests]</key>
</item>
</graph_item>
<graph_item>
<sortorder>3</sortorder>
<drawtype>0</drawtype>
<color>A54F10</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[zk_packets_received]</key>
</item>
</graph_item>
<graph_item>
<sortorder>4</sortorder>
<drawtype>0</drawtype>
<color>FC6EA3</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[zk_znode_count]</key>
</item>
</graph_item>
</graph_items>
</graph>
<graph>
<name>zookeeper status</name>
<width>900</width>
<height>200</height>
<yaxismin>0.0000</yaxismin>
<yaxismax>100.0000</yaxismax>
<show_work_period>1</show_work_period>
<show_triggers>1</show_triggers>
<type>0</type>
<show_legend>1</show_legend>
<show_3d>0</show_3d>
<percent_left>0.0000</percent_left>
<percent_right>0.0000</percent_right>
<ymin_type_1>0</ymin_type_1>
<ymax_type_1>0</ymax_type_1>
<ymin_item_1>0</ymin_item_1>
<ymax_item_1>0</ymax_item_1>
<graph_items>
<graph_item>
<sortorder>0</sortorder>
<drawtype>1</drawtype>
<color>1A7C11</color>
<yaxisside>0</yaxisside>
<calc_fnc>2</calc_fnc>
<type>0</type>
<item>
<host>Template App zookeeper</host>
<key>zookeeper.status[imok]</key>
</item>
</graph_item>
</graph_items>
</graph>
</graphs>
</zabbix_export>