Overview
Following our recent improvements in Kpow release 94.3, which introduced AI-powered filtering and automatic deserialization, release 94.5 continues to enhance the data inspect experience. This latest version introduces several key additions, including comma-separated kJQ Projection expressions, in-browser search, multiple deserialization options, and attribute sorting. It also features a significant optimization with high-performance streaming and an enhancement to our query language with expanded kJQ capabilities. You can try out the new transforms and functions on the new interactive examples page on our Factor House docs. These updates provide more granular control and deeper insights into your Kafka topics, empowering developers to navigate and analyze their data with greater efficiency and precision.
About Factor House
Factor House is a leader in real-time data tooling, empowering engineers with innovative solutions for Apache Kafka® and Apache Flink®.
Our flagship product, Kpow for Apache Kafka, is the market-leading enterprise solution for Kafka management and monitoring.
Explore our live multi-cluster demo environment or grab a free Community license and dive into streaming tech on your laptop with Factor House Local.
Targeted data views with comma-separated kJQ Projection expressions
Kpow 94.5 introduces support for comma-separated projection expressions in kJQ, such as .value.base, .value.rates. This powerful feature allows you to extract multiple fields from Kafka records in a single query. Now you can create targeted data views without unnecessary information, streamlining your workflow and keeping your output clean. This functionality is available for both key and value sub-paths, offering greater flexibility in how you inspect your data.
Faster navigation with in-browser search
You can now use your browser's native search functionality (Ctrl + F) to quickly find records by JSON path or value when using kJQ filters. This eliminates the need to re-run queries, saving you time and effort. The results component is fully keyboard-friendly, adhering to the Listbox pattern for improved accessibility. This ensures a smoother and more predictable navigation experience for all users, including those who rely on screen readers.
Deeper insights into schemas and deserialization
Data Inspect now offers detailed schema metadata for each message, including schema IDs and deserializer types. This makes it easier to identify misaligned schemas and poison messages. To handle these problematic records, Kpow 94.5 provides several deserialization options:
- Drop record (default): This option ignores erroneous records, displaying only the well-formatted ones.
- Retain record: This includes both well-formatted and erroneous records. Problematic records are flagged with a 'Deserialization exception' message instead of displaying the raw, poisonous value.
- Poison only: This option displays only the erroneous records, with the value recorded as 'Deserialization exception'.
Improved readability with attribute sorting
When you select the 'Pretty printed (sorted)' display option, the attributes of the key or value are now sorted alphabetically by name. This improves the readability and consistency of your data during inspection, making it easier to compare records and identify specific fields.
High-performance streaming for large datasets
Kpow 94.5 is optimized for performance, allowing you to stream over 500,000 records without any UI lag. This enables the efficient analysis of large datasets, so you can work with high-volume topics without compromising on speed or responsiveness.
Expanded kJQ capabilities with new transforms and functions
The kJQ language has been significantly expanded with a host of new transforms, including parse-json, floor, ceil, upper-case, lower-case, trim, ltrim, rtrim, reverse, sort, unique, first, last, keys, values, is-empty, and flatten. Additionally, new functions such as within, split, and join have been added to enable more complex data manipulation directly within your kJQ queries.
For more details on these new features, please refer to the updated kJQ manual. Also, be sure to visit the new interactive examples page on our new Factor House docs site—it's a great way to quickly verify your kJQ queries.
Conclusion
Kpow 94.5 builds upon the foundation of previous releases to deliver a more powerful and user-friendly data inspection experience. The latest additions—including kJQ Projection expressions, in-browser search, and flexible deserialization options—provide more granular control over your data. Together with high-performance streaming and expanded kJQ capabilities, these enhancements solidify Kpow as an essential tool for any developer working with Apache Kafka. Be sure to test out the new kJQ features on the interactive examples page on our Factor House docs. Ultimately, this release is designed to streamline your workflows, reduce debugging time, and empower you to unlock the full potential of your data.