0
点赞
收藏
分享

微信扫一扫

Linux--uboot移植(一)

目录

一. 前言

二. 分层存储监控(Tiered Storage Monitoring)

三. KRaft 监控指标(KRaft Monitoring Metrics)

3.1.  KRaft 投票人数监控指标(KRaft Quorum Monitoring Metrics)

3.2. KRaft 控制器监控指标(KRaft Controller Monitoring Metrics)

3.3. KRaft Broker 监控指标(KRaft Broker Monitoring Metrics)


一. 前言

    和任何一个分布式系统一样,Kafka 的存储和网络使用情况也是我们需要关注和监控的指标,只有对存储和网络状态进行充分的监控才能及时发现问题并规避风险。

二. 分层存储监控(Tiered Storage Monitoring)

以下一组指标可用于监视分层存储功能:

METRIC/ATTRIBUTE NAMEDESCRIPTIONMBEAN NAME
Remote Fetch Bytes Per SecRate of bytes read from remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate

kafka.server:type=BrokerTopicMetrics,

name=RemoteFetchBytesPerSec,topic=([-.\w]+)

Remote Fetch Requests Per SecRate of read requests from remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate

kafka.server:type=BrokerTopicMetrics,

name=RemoteFetchRequestsPerSec,topic=([-.\w]+)

Remote Fetch Errors Per SecRate of read errors from remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate

kafka.server:type=BrokerTopicMetrics,

name=RemoteFetchErrorsPerSec,topic=([-.\w]+)

Remote Copy Bytes Per SecRate of bytes copied to remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate

kafka.server:type=BrokerTopicMetrics,

name=RemoteCopyBytesPerSec,topic=([-.\w]+)

Remote Copy Requests Per SecRate of write requests to remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate

kafka.server:type=BrokerTopicMetrics,

name=RemoteCopyRequestsPerSec,topic=([-.\w]+)

Remote Copy Errors Per SecRate of write errors from remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate

kafka.server:type=BrokerTopicMetrics,

name=RemoteCopyErrorsPerSec,topic=([-.\w]+)

RemoteLogReader Task Queue SizeSize of the queue holding remote storage read tasks

org.apache.kafka.storage.internals.log:type=

RemoteStorageThreadPool,

name=RemoteLogReaderTaskQueueSize

RemoteLogReader Avg Idle PercentAverage idle percent of thread pool for processing remote storage read tasks

org.apache.kafka.storage.internals.log:type=

RemoteStorageThreadPool,

name=RemoteLogReaderAvgIdlePercent

RemoteLogManager Tasks Avg Idle PercentAverage idle percent of thread pool for copying data to remote storage

kafka.log.remote:type=RemoteLogManager,

name=RemoteLogManagerTasksAvgIdlePercent

三. KRaft 监控指标(KRaft Monitoring Metrics)

允许监视 KRaft 投票数和元数据日志的一组度量。

请注意,一些公开的度量取决于 process.roles 定义的节点的角色

3.1.  KRaft 投票人数监控指标(KRaft Quorum Monitoring Metrics)

KRaft 集群中的控制器和 Broker 都报告了这些指标:

METRIC/ATTRIBUTE NAMEDESCRIPTIONMBEAN NAME
Current StateThe current state of this member; possible values are leader, candidate, voted, follower, unattached, observer.kafka.server:type=raft-metrics,name=current-state
Current LeaderThe current quorum leader's id; -1 indicates unknown.kafka.server:type=raft-metrics,name=current-leader
Current VotedThe current voted leader's id; -1 indicates not voted for anyone.kafka.server:type=raft-metrics,name=current-vote
Current EpochThe current quorum epoch.kafka.server:type=raft-metrics,name=current-epoch
High WatermarkThe high watermark maintained on this member; -1 if it is unknown.kafka.server:type=raft-metrics,name=high-watermark
Log End OffsetThe current raft log end offset.

kafka.server:type=raft-metrics,

name=log-end-offset

Number of Unknown Voter ConnectionsNumber of unknown voters whose connection information is not cached. This value of this metric is always 0.

kafka.server:type=raft-metrics,

name=number-unknown-voter-connections

Average Commit LatencyThe average time in milliseconds to commit an entry in the raft log.kafka.server:type=raft-metrics,name=commit-latency-avg
Maximum Commit LatencyThe maximum time in milliseconds to commit an entry in the raft log.kafka.server:type=raft-metrics,name=commit-latency-max
Average Election LatencyThe average time in milliseconds spent on electing a new leader.kafka.server:type=raft-metrics,name=election-latency-avg
Maximum Election LatencyThe maximum time in milliseconds spent on electing a new leader.kafka.server:type=raft-metrics,name=election-latency-max
Fetch Records RateThe average number of records fetched from the leader of the raft quorum.kafka.server:type=raft-metrics,name=fetch-records-rate
Append Records RateThe average number of records appended per sec by the leader of the raft quorum.kafka.server:type=raft-metrics,name=append-records-rate
Average Poll Idle RatioThe average fraction of time the client's poll() is idle as opposed to waiting for the user code to process records.

kafka.server:type=raft-metrics,

name=poll-idle-ratio-avg

Current Metadata VersionOutputs the feature level of the current effective metadata version.

kafka.server:type=MetadataLoader,

name=CurrentMetadataVersion

Metadata Snapshot Load CountThe total number of times we have loaded a KRaft snapshot since the process was started.

kafka.server:type=MetadataLoader,

name=HandleLoadSnapshotCount

Latest Metadata Snapshot SizeThe total size in bytes of the latest snapshot that the node has generated. If none have been generated yet, this is the size of the latest snapshot that was loaded. If no snapshots have been generated or loaded, this is 0.

kafka.server:type=SnapshotEmitter,

name=LatestSnapshotGeneratedBytes

Latest Metadata Snapshot AgeThe interval in milliseconds since the latest snapshot that the node has generated. If none have been generated yet, this is approximately the time delta since the process was started.

kafka.server:type=SnapshotEmitter,

name=LatestSnapshotGeneratedAgeMs

3.2. KRaft 控制器监控指标(KRaft Controller Monitoring Metrics)

METRIC/ATTRIBUTE NAMEDESCRIPTIONMBEAN NAME
Active Controller CountThe number of Active Controllers on this node. Valid values are '0' or '1'.

kafka.controller:type=KafkaController,

name=ActiveControllerCount

Event Queue Time MsA Histogram of the time in milliseconds that requests spent waiting in the Controller Event Queue.

kafka.controller:type=ControllerEventManager,

name=EventQueueTimeMs

Event Queue Processing Time MsA Histogram of the time in milliseconds that requests spent being processed in the Controller Event Queue.

kafka.controller:type=ControllerEventManager,

name=EventQueueProcessingTimeMs

Fenced Broker CountThe number of fenced brokers as observed by this Controller.

kafka.controller:type=KafkaController,

name=FencedBrokerCount

Active Broker CountThe number of active brokers as observed by this Controller.

kafka.controller:type=KafkaController,

name=ActiveBrokerCount

Global Topic CountThe number of global topics as observed by this Controller.

kafka.controller:type=KafkaController,

name=GlobalTopicCount

Global Partition CountThe number of global partitions as observed by this Controller.

kafka.controller:type=KafkaController,

name=GlobalPartitionCount

Offline Partition CountThe number of offline topic partitions (non-internal) as observed by this Controller.

kafka.controller:type=KafkaController,

name=OfflinePartitionCount

Preferred Replica Imbalance CountThe count of topic partitions for which the leader is not the preferred leader.

kafka.controller:type=KafkaController,

name=PreferredReplicaImbalanceCount

Metadata Error CountThe number of times this controller node has encountered an error during metadata log processing.

kafka.controller:type=KafkaController,

name=MetadataErrorCount

Last Applied Record OffsetThe offset of the last record from the cluster metadata partition that was applied by the Controller.

kafka.controller:type=KafkaController,

name=LastAppliedRecordOffset

Last Committed Record OffsetThe offset of the last record committed to this Controller.

kafka.controller:type=KafkaController,

name=LastCommittedRecordOffset

Last Applied Record TimestampThe timestamp of the last record from the cluster metadata partition that was applied by the Controller.

kafka.controller:type=KafkaController,

name=LastAppliedRecordTimestamp

Last Applied Record Lag MsThe difference between now and the timestamp of the last record from the cluster metadata partition that was applied by the controller. For active Controllers the value of this lag is always zero.

kafka.controller:type=KafkaController,

name=LastAppliedRecordLagMs

ZooKeeper Write Behind LagThe amount of lag in records that ZooKeeper is behind relative to the highest committed record in the metadata log. This metric will only be reported by the active KRaft controller.

kafka.controller:type=KafkaController,

name=ZkWriteBehindLag

ZooKeeper Metadata Snapshot Write TimeThe number of milliseconds the KRaft controller took reconciling a snapshot into ZooKeeper.

kafka.controller:type=KafkaController,

name=ZkWriteSnapshotTimeMs

ZooKeeper Metadata Delta Write TimeThe number of milliseconds the KRaft controller took writing a delta into ZK.

kafka.controller:type=KafkaController,

name=ZkWriteDeltaTimeMs

Timed-out Broker Heartbeat CountThe number of broker heartbeats that timed out on this controller since the process was started. Note that only active controllers handle heartbeats, so only they will see increases in this metric.

kafka.controller:type=KafkaController,

name=TimedOutBrokerHeartbeatCount

Number Of Operations Started In Event QueueThe total number of controller event queue operations that were started. This includes deferred operations.

kafka.controller:type=KafkaController,

name=EventQueueOperationsStartedCount

Number of Operations Timed Out In Event QueueThe total number of controller event queue operations that timed out before they could be performed.

kafka.controller:type=KafkaController,

name=EventQueueOperationsTimedOutCount

Number Of New Controller ElectionsCounts the number of times this node has seen a new controller elected. A transition to the "no leader" state is not counted here. If the same controller as before becomes active, that still counts.

kafka.controller:type=KafkaController,

name=NewActiveControllersCount

3.3. KRaft Broker 监控指标(KRaft Broker Monitoring Metrics)

METRIC/ATTRIBUTE NAMEDESCRIPTIONMBEAN NAME
Last Applied Record OffsetThe offset of the last record from the cluster metadata partition that was applied by the brokerkafka.server:type=broker-metadata-metrics,name=last-applied-record-offset
Last Applied Record TimestampThe timestamp of the last record from the cluster metadata partition that was applied by the broker.kafka.server:type=broker-metadata-metrics,name=last-applied-record-timestamp
Last Applied Record Lag MsThe difference between now and the timestamp of the last record from the cluster metadata partition that was applied by the brokerkafka.server:type=broker-metadata-metrics,name=last-applied-record-lag-ms
Metadata Load Error CountThe number of errors encountered by the BrokerMetadataListener while loading the metadata log and generating a new MetadataDelta based on it.kafka.server:type=broker-metadata-metrics,name=metadata-load-error-count
Metadata Apply Error CountThe number of errors encountered by the BrokerMetadataPublisher while applying a new MetadataImage based on the latest MetadataDelta.kafka.server:type=broker-metadata-metrics,name=metadata-apply-error-count
举报

相关推荐

0 条评论