didi/knowstreaming

★ 7,172JavaAudience · ops devopsComplexity · 4/5Setup · hard

Mindmap

mindmap
  root((KnowStreaming))
    What it does
      Manage Kafka clusters
      Monitor metrics
      Health checks
      Send alerts
    Kafka Objects
      Brokers
      Topics
      Consumer groups
      Kafka Connect
    Operations
      Load balancing
      Partition scaling
      Partition migration
    Architecture
      Cloud-native design
      Plugin-based
      No Kafka changes needed

mindmap root((KnowStreaming)) What it does Manage Kafka clusters Monitor metrics Health checks Send alerts Kafka Objects Brokers Topics Consumer groups Kafka Connect Operations Load balancing Partition scaling Partition migration Architecture Cloud-native design Plugin-based No Kafka changes needed

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Monitor multiple Kafka clusters from a single web interface without modifying your existing Kafka setup.

USE CASE 2

Get alerts when a Kafka cluster shows anomalies, and see health dashboards across brokers, topics, and consumer groups.

USE CASE 3

Rebalance topic partitions or migrate them between brokers through a GUI instead of command-line tools.

Tech stack

JavaApache KafkaZooKeeper

Getting it running

Difficulty · hard Time to first run · 1h+

Requires an existing Apache Kafka cluster, also needs the KnowStreaming backend service running before the GUI is accessible.

License type is not stated in the documentation.

In plain English

KnowStreaming is an open-source management and monitoring platform for Apache Kafka clusters, developed by Didi, the Chinese ride-hailing company. Apache Kafka is a widely used system for moving large volumes of data between applications in real time. Managing Kafka clusters, especially many of them at once, can be complex, and KnowStreaming provides a web-based graphical interface to make that work more accessible. The platform requires no changes to your existing Kafka installation. It can connect to and manage Kafka versions from 0.10.x through 3.x.x, covering both ZooKeeper-based and Raft-based cluster configurations. The README describes the setup as needing about five minutes to get started through the GUI. KnowStreaming provides visual management panels for the main Kafka components: clusters, brokers, topics, consumer groups, messages, access control lists, and Kafka Connect. It also includes monitoring dashboards with metrics displayed across multiple dimensions, health-check tools that continuously inspect cluster state, and alerting for detected anomalies. Additional capabilities listed include load balancing across cluster nodes, topic partition scaling, and partition migration between nodes. The architecture is described as cloud-native and plugin-based, meaning you can add nodes to increase capacity without reconfiguring the core system, and enterprise-specific features can be added as plugins. The README and documentation are primarily written in Chinese. Documentation, a demo environment, and the project website are linked from the README. Community channels include a Chinese-language Kafka discussion group and a WeChat group. The license is not specified in the README text, but a license file exists in the repository.

Copy-paste prompts

Prompt 1

I'm running KnowStreaming to monitor my Kafka clusters. How do I add a new cluster to the dashboard and start tracking its consumer group lag?

Prompt 2

My KnowStreaming health check flagged an anomaly on a broker. Walk me through what to check first and how to investigate using the monitoring dashboards.

Prompt 3

I want to rebalance partitions across my Kafka brokers using KnowStreaming. What steps do I follow in the GUI?

Prompt 4

How do I configure alerting in KnowStreaming so I get notified when a consumer group falls behind?

Prompt 5

I need to add a plugin to KnowStreaming for enterprise features. How does the plugin system work and where do I start?

Open on GitHub → Explain another repo

← didi on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.