Using kPow with Aiven for Apache Kafka®

Aiven for Apache Kafka is a fully managed streaming platform that is deployable in the cloud of your choice. You can snap it into your existing workflows with the click of a button, automating mundane tasks away in minutes – allowing you and your team to focus on building your core apps.

Aiven’s managed Kafka product is fully compatible with the kPow management and monitoring toolkit and when used together they give teams the ability to create great software with ease.

Aiven for Apache Kafka, in a Nutshell

Aiven for Apache Kafka includes Kafka Connect, MirrorMaker, Schema Registry, REST, Access Control Lists (ACLs), alongside 99.9% server uptime, 24X7 support and simple, all-inclusive, transparent pricing.

Aiven is committed to security and includes features such as dedicated virtual machines, end-to-end encryption, VPC peering and flexible authentication methods including SASL/PLAIN and SASL/SCRAM. 

You can launch and work with your clusters through their web console, API, CLI or even Terraform. They also offer an unrestricted, free 30-day trial of their Managed Kafka service so you can evaluate it before buying.

In this post I’ll outline the benefits of using Aiven and kPow together, and how to quickly install and start using kPow with your Aiven Managed Kafka clusters.

Benefits for Engineers using Aiven + kPow

Using Aiven to manage the provisioning, deployment, and running of your Kafka clusters enables you to focus on building applications and eliminates the need for costly operations teams and technology. kPow provides the toolkit that empowers your engineers to build those cutting-edge applications. 

Once kPow is installed and running, you’ll have the ability to:

  • search tens of thousands of messages a second with kJQ
  • gain x-ray vision of the brokers, topics, and groups that make up your cluster
  • identify inactive topics, groups, and other areas of concern
  • visualize data distribution through your cluster
  • monitor lag in multiple dimensions

Explore Topics with kJQ

Search tens of thousands of messages a second with kPow’s unique, custom implementation of JQ-like queries for Kafka topics.

Easy to use and fast – 600,000 records filtered in a few seconds..

We got to the bottom of something in a few hours that could have dragged on for _days_

– Alex  Hilton, FX Engineering Lead, Airwallex

kJQ works with JSON or JSON-like data including Apache Avro®, Transit, EDN, and even Protobuf messages where you have configured a custom serdes.

Role Based Access Control for user management, Data Policies to mask sensitive data, an Audit Log for data governance, and the power to export query results and produce new messages make kPow the essential tool for your engineers.

kPow Data Screen

X-Ray Vision for Apache Kafka

Use kPow’s intuitive Kafka UI to gain instant visibility into your Kafka brokers, topics, partitions, offsets, consumer groups, members, assignments, streams, and more.

Get straight to the root-cause of production issues with live mode for real-time monitoring of your Kafka resources and kPow’s multidimensional view into your consumer group message lag and throughput.

Manage multiple clusters from a single installation with Kafka Connect and Schema Registry integration fully supported out of the box.

kPow Data Screen

Control your Compute

kPow provides in-depth multi-dimensional metrics that allow you to understand how your Apache Kafka applications are performing.

Understand the lag and message throughput of your consumer groups and streaming compute topologies by group, broker, topic, and consumer group host. Take control of your compute with kPow’s functions for resetting consumer offsets and more.

The Best of the Rest

  • OpenID, SAML, LDAP, and HashUser (Jetty) options for User Authentication.
  • Fine grained control of user actions with Role Based Access Control (RBAC).
  • Data Policies for masking and redacting sensitive data fields.
  • Schema Registry and Kafka Connect supported as standard.
  • Multi-cluster management supported as standard.
  • HTTPS connections supported out-of-the-box.

Integrating Operatr.IO with Aiven 

You can find detailed steps to integrating kPow with Aiven for Kafka, in Aiven’s support hub article Kafka data visualization with Operatr (now kPow).

If you are already an Aiven customer and you’d like to evaluate kPow simply sign up for a free trial.

If you aren’t an Aiven customer and you are wanting to evaluate the two products together, you can also trial Aiven free for 30 days.

Need Help?

Enjoy this article?

Sign-up to the mailing list for news, product updates and Kafka insights.

Manage, Monitor and Learn Apache Kafka with kPow by Operatr.IO

We know how easy Apache Kafka® can be with the right tools. We built kPow to make the developer experience with Kafka simple and enjoyable, and to save businesses time and money while growing their Kafka expertise. A single Docker container or JAR file that installs in minutes, kPow’s unique Kafka UI gives you instant visibility of your clusters and immediate access to your data.

kPow is compatible with Apache Kafka+1.0, Amazon MSK, Red Hat AMQ Streams, Instaclustr, Aiven, Vectorized, Azure Event Hubs, Confluent Platform, and Confluent Cloud.

Start with a free 30-day trial and solve your Kafka issues within minutes.

Kylie Troy-West is the Co-Founder and COO of Operatr.IO and the Director of Operations at Troy-West. Her focus is on empowering the Operatr.IO team to deliver on their vision of building world-class tools for Apache Kafka®.