Managed Kafka on AWS

Kafka®-compatible brokers by Selfhost.dev, with the data kept in S3.

Brokers compatible with Apache Kafka® that run on AWS and keep topic data in S3 instead of on their own disks, built on AutoMQ. Pick one broker or three across availability zones, connect any Kafka client with SASL/SCRAM and watch throughput, partitions and S3 traffic in the console, with the price shown before you create.

Kafka 3.9.1 protocol / Topic data in S3 / 1 broker or 3 across zones / Pay-as-you-go
Powered by AWS

The reliable home for your data.

Amazon Web Services (AWS)
Dedicated instances, run for you

Dedicated PostgreSQL, MySQL, Redis, ClickHouse, OpenSearch and Kafka on AWS, with BYOC into your own account. Multi-AZ on all six, PITR on Postgres and MySQL.

In our AWS account or your own with BYOC. You get AWS uptime and performance; we handle the ops.

How it works

Brokers without the storage problem.
Take one out and watch.

Producers write to the brokers, the brokers keep topic data in S3 and consumer groups read it back. Take a broker out of a three-broker cluster: it is fenced, a replacement joins and the data never moves, because it was never on that broker's disk.

topic: orders, 3 producers, 3 consumer groups

    What you get

    Built into the platform.
    Streaming next to your databases.

    Every cluster you create in the console comes with all of this, on the same account and the same bill as your PostgreSQL, ClickHouse and OpenSearch.

    Kafka 3.9.1 protocol

    Brokers speak the Kafka 3.9.1 protocol, so your existing producers, consumers and tools connect the way they connect to any cluster. The engine underneath is AutoMQ.

    Topic data in S3

    Brokers keep topic data in S3 rather than on their own disks. Durability comes from S3, so topics run with a replication factor of 1 by design.

    One broker or three across zones

    Start with a single broker or create a three-broker KRaft cluster with one broker per availability zone. When a broker stops responding it is fenced out and a replacement is provisioned automatically.

    SASL/SCRAM, ACLs and TLS

    Every client authenticates with SASL/SCRAM-SHA-256 as a user you create, with a role that maps to Kafka ACLs. A new instance gets TLS by default: each broker serves its own publicly trusted certificate.

    Schema Registry

    A Confluent-compatible registry for Avro, Protobuf and JSON Schema on port 8081, on one broker or three, with a publicly trusted certificate. Turn it on at create or later; it signs in with the cluster's own Kafka user.

    Metrics, logs and backups

    Messages and bytes in and out, produce and fetch requests, partitions, S3 traffic and offline partitions, with alert rules on them. Tail the broker log and open a point-in-time cluster snapshot with consumer-group lag. Backups archive topics with their settings, ACLs and users.

    Connect in a minute

    Any Kafka client.
    Three settings and a user.

    Copy the bootstrap server and a user from the console's Connection tab. The same settings work for the Kafka command-line tools, librdkafka clients such as confluent-kafka and kafkajs.

    SASL_PLAINTEXT or SASL_SSL

    A new instance gets TLS by default and switches to SASL_SSL once its certificate lands, a few minutes after the broker starts; the Connection tab always shows the protocol to use. Until it lands, SCRAM still protects the password, as it does on instances created without TLS.

    # client.properties, as the Connection tab gives it
    bootstrap.servers=YOUR_BROKER:9092
    # SASL_PLAINTEXT until the certificate lands
    security.protocol=SASL_SSL
    sasl.mechanism=SCRAM-SHA-256
    sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="YOUR_USER" password="YOUR_PASSWORD";
    
    # create a topic: replication factor 1, the data lives in S3
    kafka-topics.sh --bootstrap-server YOUR_BROKER:9092 \
      --command-config client.properties \
      --create --topic events --partitions 3 --replication-factor 1

    Before you start

    Six things we would
    want to know first.

    S3 changes how a broker works, so some of what you know about Kafka clusters does not carry over. Here it is up front.

    1. 1 Topics run with a replication factor of 1 by design: durability comes from S3 and the broker refuses a higher factor.
    2. 2 Backups archive topics, their settings, ACLs and users, not messages. A restore brings the topics back empty.
    3. 3 The shape is fixed at create: one broker or three, on the machine size you pick. The broker volume can grow.
    4. 4 TLS is set at create and cannot be switched afterwards. New instances get it by default since 28 October 2026; an instance created before then keeps the setting it was created with.
    5. 5 The Schema Registry needs a machine with 8 GB of RAM or more. It can be turned on or off after create and its schemas survive turning it off.
    6. 6 Topic data in S3 is billed on usage, storage and requests, outside the estimate the console shows before you create.

    Frequently asked questions

    Is this Apache Kafka?
    It is Kafka-compatible. The brokers run AutoMQ, which speaks the Kafka 3.9.1 protocol, so Kafka producers, consumers and tools connect as usual. The difference is where topic data lives: in S3 instead of on broker disks.
    Why keep topic data in S3?
    Brokers stop being the place your data lives. Durability comes from S3, so topics run with a replication factor of 1 and a failed broker is fenced and replaced rather than rebuilt from its peers.
    How do I create a cluster?
    In the console's create wizard: pick Kafka, choose one broker or three, the region and the machine size, add a Schema Registry if you want one, then create. TLS is on by default. The cluster appears with its bootstrap server, users and metrics; the Connection tab hands you a ready client.properties.
    What does it cost?
    Pay-as-you-go against prepaid credits, like every engine on Selfhost.dev. The broker machines and their volumes are metered by the hour and the create wizard shows that estimate before anything is created. Topic data lives in S3, so its storage and requests are billed on usage, outside the estimate. There are no tiers or subscriptions.
    One broker or three?
    One broker suits development and streams that can wait out a restart. For production, three brokers run one per availability zone as a KRaft quorum; a failed broker is replaced automatically. The shape is chosen at create.
    How do clients authenticate?
    With SASL/SCRAM-SHA-256 as a user you create, whose role maps to Kafka ACLs. A new instance gets TLS by default: each broker's publicly trusted certificate lands a few minutes after the broker starts and from then on clients connect with SASL_SSL, using the system truststore. Until it lands they connect with SASL_PLAINTEXT and SCRAM still protects the password. The Connection tab always shows the protocol to use.
    Is there a Schema Registry?
    Yes, on one broker or three: a Confluent-compatible registry for Avro, Protobuf and JSON Schema on port 8081 with a publicly trusted certificate, so serializers that verify certificates work with it. It signs in with the cluster's own Kafka user. Turn it on at create or later from the Configuration tab; its schemas survive turning it off. It runs its own Java process next to the broker, so it needs a machine with 8 GB of RAM or more. The console does not offer a Kafka UI yet.
    Can I manage it from my AI editor?
    Yes. Since version 0.12.0 the Selfhost.dev MCP server creates Kafka from Claude Code, Cursor or any MCP client: one broker or three across availability zones, with TLS on by default and an optional Schema Registry.

    Streaming without the disk problem.
    Priced before you press create.

    Create a Kafka cluster

    $5 in free credit to start, no card needed.