Application Demo: Mastodon

This application needs a lot of services to be able to run. For production a lot of fine-tuning and configuration settings need to be adapted. A lightweight alternative could be Takahē.

This tutorial explains how to run Mastodon on APPUiO Cloud.

If you aren’t familiar with issuing commands on a terminal session, we recommend familiarizing yourself with it before continuing this tutorial.

Requirements

To follow this guide, please make sure that you have the following tools installed:

oc: You can download the OpenShift command directly from APPUiO Cloud, selecting the help menu (marked as a question mark) and selecting the "Command line tools" entry

About the Application

Your home feed should be filled with what matters to you most, not what a corporation thinks you should see. Radically different social media, back in the hands of the people.

Step 1: Create a Project

All the following steps are currently only working on the Exoscale APPUiO Zone because of the Application Catalog service availability.

Follow these steps to login to APPUiO Cloud on your terminal:

oc login --server=https://api.${zone}.appuio.cloud:6443

You can find the exact URL of your chosen zone in the APPUiO Cloud Portal.

This command displays a URL on your terminal:

You must obtain an API token by visiting
https://oauth-openshift.apps.${zone}.appuio.cloud/oauth/token/request

Click on the link above and open it in your browser.
Click "Display token" and copy the login command shown as "Log in with this token"

Paste the oc login command on the terminal:

oc login --token=sha256~_xxxxxx_xxxxxxxxxxxxxxxxxxxxxx-xxxxxxxxxx-X \
    --server=https://api.${zone}.appuio.cloud:6443

Create a new project called "[YOUR_USERNAME]-application-demo"
```
oc new-project "[YOUR_USERNAME]-application-demo"
```

Step 2: Deploy the application

To deploy the application we will use standard Kubernetes objects. Save the example YAML Kubernetes resources into a file and apply it with oc apply -f <myfile.yaml>.

First, we need a database and a cache:

Database ordering from the VSHN Application Catalog

apiVersion: exoscale.appcat.vshn.io/v1
kind: ExoscalePostgreSQL
metadata:
  name: example-app
spec:
  writeConnectionSecretToRef:
    name: postgresql-creds
---
apiVersion: exoscale.appcat.vshn.io/v1
kind: ExoscaleRedis
metadata:
  name: example-app
spec:
  writeConnectionSecretToRef:
    name: redis-creds

This will create a PosgtreSQL and a Redis DBaaS instance with default settings. See the AppCat docs for PostgreSQL and the AppCat docs for Redis for more information.

And we need some other supporting services for Mastodon to be able to work:

Supporting services

apiVersion: apps/v1
kind: Deployment
metadata:
  name: inbucket
  labels:
    app.kubernetes.io/name: inbucket
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: inbucket
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app.kubernetes.io/name: inbucket
    spec:
      containers:
        - name: inbucket
          image: docker.io/inbucket/inbucket:latest
          imagePullPolicy: Always
          env:
            - name: INBUCKET_MAILBOXNAMING
              value: full
          ports:
            - containerPort: 9000
              name: http
              protocol: TCP
            - containerPort: 2500
              name: smtp
              protocol: TCP
          resources:
            limits: {}
            requests: {}
---
apiVersion: v1
kind: Service
metadata:
  name: inbucket-web
  labels:
    app.kubernetes.io/name: inbucket
spec:
  type: ClusterIP
  sessionAffinity: None
  ports:
    - name: web
      port: 8080
      protocol: TCP
      targetPort: http
  selector:
    app.kubernetes.io/name: inbucket
---
apiVersion: v1
kind: Service
metadata:
  name: inbucket-smtp
  labels:
    app.kubernetes.io/name: inbucket
spec:
  type: ClusterIP
  sessionAffinity: None
  ports:
    - name: smtp
      port: 2500
      protocol: TCP
      targetPort: 2500
  selector:
    app.kubernetes.io/name: inbucket
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-tunnel
  labels:
    app.kubernetes.io/name: redis-tunnel
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: redis-tunnel
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app.kubernetes.io/name: redis-tunnel
    spec:
      containers:
        - name: redis-stunnel
          image: ghcr.io/appuio/stunnel-docker:latest
          imagePullPolicy: Always
          env:
            - name: ACCEPT
              value: "0.0.0.0:6379"
            - name: CONNECT_HOST
              valueFrom:
                secretKeyRef:
                  name: redis-creds
                  key: REDIS_HOST
            - name: CONNECT_PORT
              valueFrom:
                secretKeyRef:
                  name: redis-creds
                  key: REDIS_PORT
            - name: CLIENT
              value: "yes"
          ports:
            - name: redis
              containerPort: 6379
              protocol: TCP
          resources:
            limits: {}
            requests: {}
---
apiVersion: v1
kind: Service
metadata:
  name: redis-tunnel
  labels:
    app.kubernetes.io/name: redis-tunnel
spec:
  type: ClusterIP
  sessionAffinity: None
  ports:
    - name: redis
      port: 6379
      protocol: TCP
      targetPort: redis
  selector:
    app.kubernetes.io/name: redis-tunnel

This will create an Inbucket mail catcher to catch mails sent by Mastodon and a Redis tunnel to proxy unencrypted Redis connections (Mastodon currently only supports unencrypted connections, see GitHub Issue #19824).

Before we can deploy the application, a few parameters need to be replaced:
→ All the secret values in the mastodon-secrets secret (Replace CHANGEMESECRET with a random string, for example generated with pwgen 32. All values need to be different.)
→ Replace CHANGEMEURL in the environment variables LOCAL_DOMAIN,WEB_DOMAIN and MASTODON_STREAMING_API_BASE_URL of the Deployment with [YOUR_USERNAME]-application-demo
→ Replace CHANGEMEURL in Ingress object with [YOUR_USERNAME]-application-demo

Application deployment with all other needed resources

apiVersion: v1
kind: Secret
metadata:
  name: mastodon-secrets
stringData:
  SECRET_KEY_BASE: CHANGEMESECRET
  OTP_SECRET: CHANGEMESECRET
  VAPID_PRIVATE_KEY: CHANGEMESECRET
  VAPID_PUBLIC_KEY: CHANGEMESECRET
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mastodon
  labels:
    app.kubernetes.io/name: demo-app
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: demo-app
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app.kubernetes.io/name: demo-app
    spec:
      containers:
        - name: mastodon-web
          image: docker.io/bitnami/mastodon:latest
          imagePullPolicy: Always
          envFrom:
            - secretRef:
                name: mastodon-secrets
          env:
            - name: MASTODON_MODE
              value: web
            - name: MASTODON_HTTPS_ENABLED
              value: "true"
            - name: LOCAL_DOMAIN
              value: "CHANGEMEURL.apps.exoscale-ch-gva-2-0.appuio.cloud"
            - name: WEB_DOMAIN
              value: "CHANGEMEURL.apps.exoscale-ch-gva-2-0.appuio.cloud"
            - name: MASTODON_STREAMING_API_BASE_URL
              value: "wss://CHANGEMEURL.apps.exoscale-ch-gva-2-0.appuio.cloud/streaming"
            - name: MASTODON_DATABASE_HOST
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_HOST
            - name: MASTODON_DATABASE_PORT_NUMBER
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_PORT
            - name: MASTODON_DATABASE_NAME
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_DB
            - name: MASTODON_DATABASE_USERNAME
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_USER
            - name: MASTODON_DATABASE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_PASSWORD
            - name: MASTODON_REDIS_HOST
              value: redis-tunnel
            - name: MASTODON_REDIS_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: redis-creds
                  key: REDIS_PASSWORD
            - name: MASTODON_ELASTICSEARCH_ENABLED
              value: "false"
            - name: SMTP_SERVER
              value: inbucket-smtp
            - name: SMTP_PORT
              value: "2500"
            - name: SMTP_FROM_ADDRESS
              value: mastodon-demo-app@appuio.cloud
          ports:
            - name: web
              containerPort: 3000
              protocol: TCP
          resources:
            limits: {}
            requests: {}
          volumeMounts:
            - name: app-data
              mountPath: /bitnami/mastodon
        - name: mastodon-sidekiq
          image: docker.io/bitnami/mastodon:latest
          imagePullPolicy: Always
          envFrom:
            - secretRef:
                name: mastodon-secrets
          env:
            - name: MASTODON_MODE
              value: sidekiq
            - name: MASTODON_WEB_HOST
              value: localhost
            - name: MASTODON_WEB_PORT_NUMBER
              value: "3000"
            - name: MASTODON_DATABASE_HOST
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_HOST
            - name: MASTODON_DATABASE_PORT_NUMBER
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_PORT
            - name: MASTODON_DATABASE_NAME
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_DB
            - name: MASTODON_DATABASE_USERNAME
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_USER
            - name: MASTODON_DATABASE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_PASSWORD
            - name: MASTODON_REDIS_HOST
              value: redis-tunnel
            - name: MASTODON_REDIS_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: redis-creds
                  key: REDIS_PASSWORD
            - name: MASTODON_ELASTICSEARCH_ENABLED
              value: "false"
            - name: SMTP_SERVER
              value: inbucket-smtp
            - name: SMTP_PORT
              value: "2500"
            - name: SMTP_FROM_ADDRESS
              value: mastodon-demo-app@appuio.cloud
          resources:
            limits: {}
            requests: {}
          volumeMounts:
            - name: app-data
              mountPath: /bitnami/mastodon
      volumes:
        - name: app-data
          persistentVolumeClaim:
            claimName: app-data
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mastodon-streaming
  labels:
    app.kubernetes.io/name: demo-app-streaming
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: demo-app-streaming
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app.kubernetes.io/name: demo-app-streaming
    spec:
      containers:
        - name: mastodon-streaming
          image: docker.io/bitnami/mastodon:latest
          imagePullPolicy: Always
          envFrom:
            - secretRef:
                name: mastodon-secrets
          env:
            - name: MASTODON_MODE
              value: streaming
            - name: MASTODON_WEB_HOST
              value: example-app
            - name: MASTODON_WEB_PORT_NUMBER
              value: "443"
            - name: MASTODON_DATABASE_HOST
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_HOST
            - name: MASTODON_DATABASE_PORT_NUMBER
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_PORT
            - name: MASTODON_DATABASE_NAME
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_DB
            - name: MASTODON_DATABASE_USERNAME
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_USER
            - name: MASTODON_DATABASE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgresql-creds
                  key: POSTGRESQL_PASSWORD
            - name: MASTODON_REDIS_HOST
              value: redis-tunnel
            - name: MASTODON_REDIS_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: redis-creds
                  key: REDIS_PASSWORD
            - name: MASTODON_ELASTICSEARCH_ENABLED
              value: "false"
            - name: DB_SSLMODE
              value: "true"
            - name: NODE_EXTRA_CA_CERTS
              value: "/usr/local/share/ca-certificates/dbaas-postgresql-ca.crt"
          ports:
            - name: web
              containerPort: 4000
              protocol: TCP
          resources:
            limits: {}
            requests: {}
          volumeMounts:
            - name: db-ca
              mountPath: /usr/local/share/ca-certificates
      volumes:
        - name: db-ca
          secret:
            secretName: postgresql-creds
            optional: false
            items:
              - key: ca.crt
                path: dbaas-postgresql-ca.crt
---
apiVersion: v1
kind: Service
metadata:
  name: mastodon-streaming
spec:
  type: ClusterIP
  sessionAffinity: None
  ports:
    - name: web
      port: 4000
      protocol: TCP
      targetPort: web
  selector:
    app.kubernetes.io/name: demo-app-streaming
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mastodon
  annotations:
    route.openshift.io/termination: "edge"
spec:
  rules:
  - host: CHANGEMEURL.apps.exoscale-ch-gva-2-0.appuio.cloud
    http:
      paths:
      - backend:
          service:
            name: example-app
            port:
              number: 443
        path: /
        pathType: Prefix
  - host: CHANGEMEURL.apps.exoscale-ch-gva-2-0.appuio.cloud
    http:
      paths:
      - backend:
          service:
            name: mastodon-streaming
            port:
              number: 4000
        path: /streaming
        pathType: Prefix
---
apiVersion: v1
kind: Service
metadata:
  name: example-app
  labels:
    app.kubernetes.io/name: demo-app
spec:
  type: ClusterIP
  sessionAffinity: None
  ports:
    - name: web
      port: 443
      protocol: TCP
      targetPort: web
  selector:
    app.kubernetes.io/name: demo-app
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: app-data
  labels:
    app.kubernetes.io/name: demo-app
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: "1Gi"

Now wait until your pods appears with the status "Running":
```
oc get pods --watch
```

You can now login to your Mastodon instance with the default credentials by the image under https://[YOUR_USERNAME]-application-demo.apps.exoscale-ch-gva-2-0.appuio.cloud:

Username: user@example.com
Password: bitnami1

To see what mails Mastodon sends, you can access Inbucket by doing a port-forward:

oc port-forward svc/inbucket-web 8080:8080

And then open a browser at localhost:8080.

This example configuration isn’t meant for a production ready service.

What’s next?

For a production ready service, we recommend the following parts to be implemented and configured:

While the database is already being backed up because it’s a managed service, you should still backup your persistent volume with K8up.
Add some monitoring for your application:
- Monitor HTTP(S) Endpoints Using Blackbox Exporter
- Monitor PVC Disk Usage
Use your own URL with Let’s Encrypt.
Choose appropriate sizing:
- Persistent storage volume size
- Application Catalog DBaaS plan
Maybe you want to use an RWX storage class which allows you to scale your application by running multiple Pods.
Configure proper requests and limits for your Pod.
Use a pinned image version and set the imagePullPolicy to IfNotPresent.
Keep your app up-to-date! Install patches and upgrades as they get available. One way to achieve that, is to use a GitOps style deployment, either push or pull, and leverage the mighty Renovate Bot to keep your image references clean.
Using a Helm Chart for production deployments or Kustomize setup for different stages can be an advantage.

Especially for this example application:

Change the default username and password immediately.
Review the documentation of the used image to learn more about the configuration and possibilities.
Mastodon needs a proper mail sending configuration in production, we recommend using a managed mail sending service for that, for example Mailgun.

Mastodon isn’t particularly easy to run in production, some thoughts for improvements on this demo deployment:

Sidekick is configured as second container in the same Pod as the web application. This isn’t a good idea, it should run as a separate Pod which can be scaled accordingly, because Sidekick is where the hard work happens. For this to work there must be shared storage available, therefore choose an RWX volume for that.
As the Mastodon instance grows, a lot of tuning is needed. Some useful resources about that topic:
Follow the official documentation about the secrets to create better secrets.

Once you’re done evaluating this example application, cleanup again to not cause any unwanted costs:

oc delete project [YOUR_USERNAME]-application-demo

We’re happy to help you running your application. Contact us and let us know how we can help.