Setting up Graylog in Azure

To get started with installing Graylog, do the following:

Create a VM using the following:

SSH into the server and follow this guide to get Graylog installed.

To set up public access, set the following variables in Graylog config file (make sure to include the leading slash in external_uri):

http_bind_address = PRIVATE_IP
http_external_uri = http://<PUBLIC_IP>/

Once fully installed, set up an Apache reverse proxy:

sudo apt-get install apache2 -y
sudo a2enmod proxy_http proxy_ajp rewrite deflate headers proxy_balancer proxy_connect proxy_html ssl lbmethod_byrequests slotmem_shm proxy

Edit /etc/apache2/sites-enabled/000-default.conf:

ProxyPass "/"  "http://PRIVATE_IP:9000/"
ProxyPassReverse "/"  "http://PRIVATE_IP:9000/"

Then restart both servers:

sudo systemctl restart graylog-server.service
sudo systemctl restart apache2

To verify installation, access at <IP_ADDRESS> to verify the installation. If you see the Graylog login screen, you’ve successfully set up the server.

Finish by setting the SSH networking rule to a trusted IP to improve security.

Set up HTTPS using Let’s Encrypt

To set up HTTPS using Let’s Encrypt, use the Certbot directions.

Once that’s done, make sure to change http_external_uri in the Graylog config file and restart Graylog.

Send Kubernetes Logs to Graylog

First, SSH into the server and configure Elasticsearch (/etc/elasticsearch/elasticsearch.yml) to bind to the private IP of the VM:

network.host: PRIVATE_IP

Restart Elasticsearch, then configure Graylog to listen to the new Elasticsearch host:

elasticsearch_hosts = http://PUBLIC_IP:9200

Restart Graylog, then open the firewall to allow for port 9200 to be accessible by the cluster IP. Confirm access by trying to hit port 9200.

Next, set up an input in Graylog.

After that, create the RBAC role for the cluster (fluentd-rbac.yml):

Then create the daemonset, changing the container environment variables as needed (fluentd-daemonset.yml):

Deploy both of these out:

Then check the logs of the daemonset to confirm correct connection:

Sending Azure App Service Logs to Graylog

To get started, create the following:

For the App Service in place, enable “Diagnostic logs” and send them to the Event Hub.

TO BE CONTINUED

Increase Heap Size

To increase the heap size, edit /etc/default/graylog-server, then restart graylog-server.service.

References

https://mherman.org/blog/logging-in-kubernetes-with-elasticsearch-Kibana-fluentd/#fluentd