This guide will cover how to run Schema Registry on your server in
AWS using the hosted Kafka Cluster at Cloudkarafka.
You need a server running Ubuntu in your AWS account that you can access with ssh. To run Schema Registry without memory issues the server needs to have at least 1Gb of memory.
Create the Kafka cluster at cloudkarafka.com, make sure to select a subnet that doesn’t conflict with the subnet that your machines (in you account) is using.
See this Guide on how to set up VPC Peering connections Guide: VPC Peering
Schema Registry is part of the Confluent Platform and not available as standalone. So we'll go ahead and download the latest version of the Confluent Platform which is version 5.5.1. Download the tarball and extract into /opt
wget https://packages.confluent.io/archive/5.5/confluent-5.5.1-2.12.tar.gz tar -xzvf confluent-5.5.1-2.12.tar.gz -C /opt
cd /opt/confluent-5.5.1 vim etc/schema-registry/schema-registry.properties
# /opt/confluent-5.5.1/etc/schema-registry/schema-registry.properties listeners=http://0.0.0.0:8081 kafkastore.bootstrap.servers=PLAINTEXT://10.56.72.161:9092,PLAINTEXT://10.56.72.51:9092,PLAINTEXT://10.56.72.225:9092 kafkastore.topic=_schemas debug=true
Run Schema Registry as a Systemd service for better reliability.
# /etc/systemd/system/schemaregistry.service [Unit] Description=Schema Registry [Service] Type=simple PIDFile=/var/run/schemaregistry.pid User=ubuntu Group=ubuntu ExecStart=/opt/confluent-5.5.1/bin/schema-registry-start /opt/confluent-5.5.1/etc/schema-registry/schema-registry.properties ExecStop=/opt/confluent-5.5.1/bin/schema-registry-stop Restart=on-failure SyslogIdentifier=schemaregistry [Install] WantedBy=multi-user.target
Now enable the service and start it
sudo systemctl enable schemaregistry sudo systemctl start schemaregistry
And now the service will start automatically every time the server is rebooted.
To check the status of the service
sudo systemctl status schemaregistry
Instead of having Schema Registry listen to http://0.0.0.0:8081 you can change that to http://127.0.0.1:8081 and you can put nginx in front. This allows you to use an encrypted connection and it also adds the possibility to use a custom path, for example, different port, subdomain or a custom path to reach Schema Registry. You can also add some security to the endpoint by configuring nginx to check for basic auth header which will then force the user to use username and password to access the http service. More about that here: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/