CloudKarafka are managed Apache Kafka servers in the cloud. Kafka is a distributed publish-subscribe messaging systems that is designed to be fast, scalable, and durable. It's an open-source message broker written in Scala and Java that can support a large number of consumers, and retain large amounts of data with very little overhead.
CloudKarafka automates every part of setup, running and scaling of Apache Kafka. We have support staff available 24/7 to help in any event or with any questions. CloudKarafka let developers focus on the core part of their applications, instead of managing and maintaining servers.
To get started with CloudKarafka, you need to sign up for an account and create a CloudKarafka instance.
Create an account and login to the control panel and press Create+ to create a new instance. Choose a name for the instance and the datacenter to host the instance. To get started you need to sign up for a customer plan. What plan you want to use depend of your needs. We offer five different plans for different needs. You can try CloudKarafka for free with the plan Developer Duck.
All plans are billed by the second, you can try out even the largest instance types for mere pennies. As soon as you delete the instance you won't be charged for it anymore. Billing occurs at the end of each month, and you are only charged for the time an instance has been available to you.
The instance is immediately provisioned after creation and you can view the instance details at the details page.
When you have signed up and created your instance it is time to Get started with CloudKarafka.
Our shared instances are on a multi tenant Kafka server. You share the server with other users. To get started with your shared instance you need a URL, username and password. This can be found on the details pages for your instance. Once you have that information you can have a look at any of the demo projects we have created to get you up and running quick and simple. The example can be found in menu at the top of this page, the examples are working and you can download it, set the URL, username and password and run.
A secure connection has to be set up between you clients and your CloudKarafka server. If you host your servers in the same cloud as your CloudKarafka server you can peer the VPCs, otherwise you can use SASL/SCRAM which will authenticate you using username and password and give you an encrypted connection.
Press on the details page for your instance and open up the VPC Peering tab. CloudKarafka will request to set up a VPC connection to you as soon as you have saved your VPC peering details. After that you need to accept the VPC peering connection request from us. The request can be accepted from the Amazon VPC console at https://console.aws.amazon.com/vpc/. Please note that the subnet given must be the same as your VPC subnet.
Kafka MGMT is a user-friendly Management Interface for Apache Kafka. The interface let you monitor and handle your Apache Kafka server from a web browser, in a much simpler way than previous versions.
Once your instance is provided to you, you are able to view numbers of consumers, create and delete topics, view broker information, create and delete users and you can view topic size and total topic message count - among other things.
There are a some concepts that is good to be familiar with before you get started with your first CloudKarafka instance.
Librdkafka is a C library implementation of the Apache Kafka protocol, containing both Producer and Consumer support. Most modern client libraries use librdkafka as the underlying implementation and are just a thin wrapper around it. You can read more about librdkafka here https://github.com/edenhill/librdkafka.
We use SASL SCRAM for authentication for our Apache Kafka cluster so it is important that librdkafka has the SSL and SASL SCRAM feature enabled to work with CloudKarafka.
You need to have OpenSSL / libcrypto installed for SSL and SASL SCRAM to work properly. Since Apple has deprecated their OpenSSL bundle, the easiest way is to install it via Homebrew
brew install openssl librdkafka
On Debian and Ubuntu, install librdkafka from the Confluent APT repositories, see instructions here and then install librdkafka:
apt-get install libssl-dev librdkafka-dev
Don't hesitate to contact us at firstname.lastname@example.org if you got any questions!