DEV Community

Chen Debra
Chen Debra

Posted on

Deploying Apache DolphinScheduler 3.1.9 Cluster with MySQL Instead of PostgreSQL? A Practical Docker Compose Guide

#mysql #apachedolphinscheduler #doris #postgressql

MySQL DS

Background

Today, we'll walk through one of the most requested deployment scenarios in the Apache DolphinScheduler community: deploying an Apache DolphinScheduler 3.1.9 cluster with MySQL as the metadata database using Docker Compose.

As many users know, the official Docker Compose deployment provided by DolphinScheduler uses PostgreSQL as the metadata repository by default. This is mainly because the GPLv2 license of MySQL is not fully compatible with Apache License 2.0, preventing the project from distributing an official MySQL-based package.

However, in real-world production environments, many organizations prefer MySQL due to its mature ecosystem, extensive tooling, operational familiarity, and widespread adoption across enterprises.

In this guide, we'll show how to adapt the official Docker Compose deployment and successfully run a DolphinScheduler cluster backed by MySQL metadata storage.

Pull Required Images 

docker pull apache/dolphinscheduler-master:3.1.9

docker pull apache/dolphinscheduler-worker:3.1.9

docker pull apache/dolphinscheduler-tools:3.1.9

docker pull apache/dolphinscheduler-api:3.1.9

docker pull apache/dolphinscheduler-alert-server:3.1.9

docker pull bitnami/zookeeper:3.7.1
Enter fullscreen mode Exit fullscreen mode

Download the MySQL JDBC Driver 

wget https://downloads.mysql.com/archives/get/p/3/file/mysql-connector-j-8.0.33.zip

unzip -q mysql-connector-j-8.0.33.zip

cp mysql-connector-j-8.0.33/mysql-connector-j-8.0.33.jar .
Enter fullscreen mode Exit fullscreen mode

Prepare Custom Images

Dockerfile for Master, Worker, API, and Alert Server 

# Based on the official DolphinScheduler image

ARG SERVICE=api

FROM apache/dolphinscheduler-${SERVICE}:3.1.9

# Copy the MySQL JDBC driver into the DolphinScheduler library directory

# DolphinScheduler loads JDBC drivers from the lib directory

COPY mysql-connector-j-8.0.33.jar /opt/dolphinscheduler/libs/
Enter fullscreen mode Exit fullscreen mode

Dockerfile for Tools 

# Based on the official DolphinScheduler image

ARG SERVICE=tools

FROM apache/dolphinscheduler-${SERVICE}:3.1.9

# Copy the MySQL JDBC driver into the DolphinScheduler tools library directory

# DolphinScheduler loads JDBC drivers from the lib directory

COPY mysql-connector-j-8.0.33.jar /opt/dolphinscheduler/tools/libs/
Enter fullscreen mode Exit fullscreen mode

Build Custom Images 

docker build --build-arg SERVICE=master -t apache/dolphinscheduler-master:3.1.9-mysql .

docker build --build-arg SERVICE=worker -t apache/dolphinscheduler-worker:3.1.9-mysql .

docker build --build-arg SERVICE=tools -t apache/dolphinscheduler-tools:3.1.9-mysql .

docker build --build-arg SERVICE=api -t apache/dolphinscheduler-api:3.1.9-mysql .

docker build --build-arg SERVICE=alert-server -t apache/dolphinscheduler-alert-server:3.1.9-mysql .
Enter fullscreen mode Exit fullscreen mode

Update docker-compose.yaml

Disable the PostgreSQL service and add a MySQL service as the metadata database.

# Comment out the PostgreSQL service

# dolphinscheduler-postgresql:
#   image: bitnami/postgresql:15.2.0
#   ...

# MySQL Metadata Database Service

dolphinscheduler-mysql:
  image: mysql:8.0
  container_name: dolphinscheduler-mysql

  profiles:
    - all
    - schema

  environment:
    MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-root}
    MYSQL_DATABASE: ${MYSQL_DATABASE:-dolphinscheduler}

  volumes:
    - dolphinscheduler-mysql:/var/lib/mysql

  ports:
    - "3306:3306"

  # Expose MySQL to allow workers on other servers to connect

  healthcheck:
    test:
      [
        "CMD",
        "mysqladmin",
        "ping",
        "-h",
        "localhost",
        "-u",
        "${MYSQL_USERNAME:-root}",
        "-p${MYSQL_PASSWORD:-root}"
      ]

    interval: 5s
    timeout: 60s
    retries: 120

  networks:
    - dolphinscheduler
Enter fullscreen mode Exit fullscreen mode

The remaining services (ZooKeeper, Schema Initializer, API, Alert Server, Master, Worker, Network, and Volume configurations) remain the same as the official Docker Compose deployment, with the following key modifications:

Schema Initializer Dependency 

depends_on:
  dolphinscheduler-mysql:
    condition: service_healthy
Enter fullscreen mode Exit fullscreen mode

Master JVM Configuration 

environment:
  JAVA_OPTS: >
    -server
    -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}
    -Xms8g
    -Xmx8g
    -Xmn4g
    -XX:+PrintGCDetails
    -Xloggc:gc.log
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:HeapDumpPath=dump.hprof
Enter fullscreen mode Exit fullscreen mode

Worker JVM Configuration 

environment:
  JAVA_OPTS: >
    -server
    -Duser.timezone=${SPRING_JACKSON_TIME_ZONE}
    -Xms8g
    -Xmx8g
    -Xmn4g
    -XX:+PrintGCDetails
    -Xloggc:gc.log
    -XX:+HeapDumpOnOutOfMemoryError
    -XX:HeapDumpPath=dump.hprof
Enter fullscreen mode Exit fullscreen mode

Volume Configuration 

volumes:
  # Comment out PostgreSQL volume

  # dolphinscheduler-postgresql:

  dolphinscheduler-mysql:
  dolphinscheduler-zookeeper:
  dolphinscheduler-worker-data:
  dolphinscheduler-logs:
  dolphinscheduler-shared-local:
Enter fullscreen mode Exit fullscreen mode

Update the .env File 

# Docker Hub Repository and Image Tag

HUB=apache
TAG=3.1.9

# MySQL Configuration

MYSQL_ROOT_PASSWORD=root
MYSQL_DATABASE=dolphinscheduler
MYSQL_USERNAME=root
MYSQL_PASSWORD=root

# DolphinScheduler Database Configuration

TZ=Asia/Shanghai

# Use MySQL as the metadata database

DATABASE=mysql

SPRING_JACKSON_TIME_ZONE=GMT+8

SPRING_DATASOURCE_URL=jdbc:mysql://dolphinscheduler-mysql:3306/${MYSQL_DATABASE}?useUnicode=true&characterEncoding=UTF-8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true

SPRING_DATASOURCE_USERNAME=${MYSQL_USERNAME}

SPRING_DATASOURCE_PASSWORD=${MYSQL_PASSWORD}

REGISTRY_ZOOKEEPER_CONNECT_STRING=dolphinscheduler-zookeeper:2181

MASTER_FETCH_COMMAND_NUM=10
Enter fullscreen mode Exit fullscreen mode

Initialize the Database 

docker compose --profile schema up -d
Enter fullscreen mode Exit fullscreen mode

Start the Entire Cluster 

docker compose --profile all up -d
Enter fullscreen mode Exit fullscreen mode

Start the Worker Service 

docker compose up -d dolphinscheduler-worker
Enter fullscreen mode Exit fullscreen mode

Start the Master Service 

docker compose up -d dolphinscheduler-master
Enter fullscreen mode Exit fullscreen mode

Start the Alert Service 

docker compose up -d dolphinscheduler-alert
Enter fullscreen mode Exit fullscreen mode

Start the API Service 

docker compose up -d dolphinscheduler-api
Enter fullscreen mode Exit fullscreen mode

Restart All Services 

docker compose --profile all restart
Enter fullscreen mode Exit fullscreen mode

Conclusion

Although Apache DolphinScheduler officially ships with a PostgreSQL-based Docker Compose deployment, many enterprises continue to standardize on MySQL for operational consistency and ecosystem compatibility.

By adding the MySQL JDBC driver, rebuilding the DolphinScheduler images, and adjusting the Docker Compose and environment configurations, you can quickly deploy a fully functional Apache DolphinScheduler 3.1.9 cluster powered by MySQL metadata storage.

This approach enables teams already invested in the MySQL ecosystem to integrate DolphinScheduler into their infrastructure with minimal friction while preserving the benefits of containerized deployment and cluster-based scheduling.

If your organization relies on MySQL as a strategic database platform, this solution provides a practical and production-friendly path to running Apache DolphinScheduler at scale.

Top comments (0)