CQRS is a useful pattern to reason about the activities of a specific domain. But it definitely comes with a steep learning curve. Reads, Writes, DDD, Event Sourcing, Eventual Consistency, why should we care? This is a cross-post of my original article on my own space: CQRS: Why? And all…

How to run SQL queries over Kafka in seconds? With PySpark, we get a system to run ad-hoc analysis locally and any SQL queries to explore our data quickly. — PySpark is a Python interface to write Apache Spark applications to use it in command line. The Apache Spark platform is built to crunch big datasets in a distributed way. Combining PySpark with Kafka, we get a system that can process real-time data from Kafka in seconds using PySpark commands…

We’ve overhauled how we explore data and how we manage Kafka ACLs and Service Accounts. — User-Friendly Filters We have completely revamped our filtering UX on the Consume page to align more with your day-to-day use cases. Now, you can search your topics by combining any of the following filtering mechanisms. Simple Filters Search for any text in your messages, either in your Key or Value, using basic operators like…

It’s simple: never say “It’s simple” (or equivalent). — I’m a big fan of Writing Well by William Zinsser. I love writing, but I’m sad because I’m not writing enough. I have plenty of drafts on various platforms, but from ‘draft’ to ‘production’ is a big step, always asking a lot of energy (if not a ChatGPT thing). The…

It’s dangerous to go alone, take this! — We build trust not during times of success but during challenging times. Building a company is a unique, exciting challenge, but it can be profoundly annoying and not interesting sometimes. In the end, someone must always be accountable for everything, and that’s probably the co-founders often in the C-suite: CEO…

It’s way too easy and wrong to consider all engineers look alike, do the same job, and can be replaced by another engineer. — With over 15 years working in engineering, I‘ve seen many things and worked with various types of profiles across various companies, cultures, and positions. I’m not talking about IC leveling, Senior, Staff, etc. I’m talking about what’s behind us as humans: our mindset, how we think, what we are looking…

What happens when you combine Conduktor and Redpanda? See how we are working together to give the best experience to developers and infrastructure teams. Simplicity and cost-efficiency. — Conduktor & Redpanda Conduktor's mission is to provide simple, flexible, and powerful tooling for Kafka developers and infrastructure. Our product is composed of two parts: a UI dedicated to developers efficiency and Conduktor Gateway, a Kafka proxy to embrace DevSecOps needs in organizations (enforcing best practices, end-to-end encryption auditability, multi-tenancy, seamless failover, and…

AWS provides an exceptional managed Kafka Streaming Platform while Conduktor provides the best tooling to work with Kafka! Let’s see how they work together. — Deploying Apache Kafka on AWS is very easy, thanks to AWS MSK (Amazon Managed Streaming for Apache Kafka — Amazon MSK). You get a production-ready, self-managed Kafka cluster in a few clicks on which you can run your applications, create data streams and manage connectors.

From designing topics and optimizing performance to handling tombstones and tuning producers. Gain essential knowledge and best practices to build robust and efficient Kafka applications. — Apache Kafka has become the go-to choice for building distributed streaming platforms and real-time data pipelines. As a developer working with Kafka, it’s crucial to understand some core concepts and best practices to ensure applications work well and data processing pipelines are efficient and reliable.