Superset 4.0版本部署(docker部署及pip部署)

1.docker 部署

1.1 获取源码及初始化网络

git clone https://github.com/apache/superset.git
cd superset
git checkout 4.0
docker network create superset4 #初始化superset网络，方便后续与mysql与redis连接

1.2 拉取并处理mysql与redis镜像

#拉取mysql与redis镜像
docker pull mysql:8.0.18
docker pull redis:latest
 
#运行mysql与redis
docker run --name mysql -e MYSQL_ROOT_PASSWORD=root --network superset4 -p 3306:3306 -d mysql:8.0.18
docker run --name redis -p 6379:6379 --network superset4 -d redis:latest
 
#运行后在docker中将出现mysql与redis两个容器
 
 
 
#进入docker mysql容器，进exec中执行以下命令
mysql -u root -p
create database superset;

1.3 处理superset配置

找到superset代码中docker/.env-non-dev，找到database configurations，在其中配置mysql数据库的各项信息，redis数据库同理

其中DATA_BASE_HOST在docker容器中复制，步骤：点开mysql容器 → insepct → 往下拉到最后的IPAddress，redis同理

接下来在superset代码中找到requirements下develoment.txt，拉到最下，新增一行

pymysql==1.1.0

在superset根目录中的dockerfile文件配置文件路径，拉到最下方新增代码：

COPY --chown=superset ./docker/docker-bootstrap.sh /app/docker/
COPY --chown=superset ./docker/docker-init.sh /app/docker/
COPY --chown=superset ./docker/docker-ci.sh /app/docker/
COPY --chown=superset ./docker/.env-non-dev /app/docker/
COPY --chown=superset ./docker/pythonpath_dev/superset_config.py /app/docker/pythonpath_dev/
COPY --chown=superset:superset --chmod=755 ./docker/*.sh /app/docker/

1.4 构建镜像

#在dockerfile同级目录下运行该cmd命令，注意末尾点号
docker build -t superset4.0 .

1.5 错误处理

报错：Failed to fetch http://deb.debian.org/debian-security/pool/updates/main/p/perl/libperl5.36_5.36.0-7%2bdeb12u2_amd64.deb Hash Sum mismatch

这个错误是因为Debian软件源的包校验和不匹配。这通常是由于网络问题、镜像服务器同步问题或临时缓存问题导致的，在cursor询问了AI后，AI修改了dockerfile文件，进行了以下操作：

清除apt缓存、设置仅使用gz压缩方式、禁用HTTP管道、禁用HTTP缓存、设置可处理破损代理。

完整dockerfile文件如下：

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
 
######################################################################
# Node stage to deal with static asset construction
######################################################################
ARG PY_VER=3.9-slim-bookworm
 
# if BUILDPLATFORM is null, set it to 'amd64' (or leave as is otherwise).
ARG BUILDPLATFORM=${BUILDPLATFORM:-amd64}
FROM --platform=${BUILDPLATFORM} node:16-bookworm-slim AS superset-node
 
ARG NPM_BUILD_CMD="build"
 
RUN rm -rf /var/lib/apt/lists/* \
    && echo 'Acquire::CompressionTypes::Order:: "gz";' > /etc/apt/apt.conf.d/99compression-workaround \
    && echo 'Acquire::http::Pipeline-Depth "0";' > /etc/apt/apt.conf.d/99pipeline \
    && echo 'Acquire::http::No-Cache "true";' >> /etc/apt/apt.conf.d/99pipeline \
    && echo 'Acquire::BrokenProxy "true";' >> /etc/apt/apt.conf.d/99pipeline
 
RUN apt-get update -qq \
    && apt-get install -yqq --no-install-recommends \
        build-essential \
        python3
 
ENV BUILD_CMD=${NPM_BUILD_CMD} \
    PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
# NPM ci first, as to NOT invalidate previous steps except for when package.json changes
WORKDIR /app/superset-frontend
 
RUN --mount=type=bind,target=/frontend-mem-nag.sh,src=./docker/frontend-mem-nag.sh \
    /frontend-mem-nag.sh
 
RUN --mount=type=bind,target=./package.json,src=./superset-frontend/package.json \
    --mount=type=bind,target=./package-lock.json,src=./superset-frontend/package-lock.json \
    npm ci
 
COPY ./superset-frontend ./
# This seems to be the most expensive step
RUN npm run ${BUILD_CMD}
 
######################################################################
# Final lean image...
######################################################################
FROM python:${PY_VER} AS lean
 
WORKDIR /app
ENV LANG=C.UTF-8 \
    LC_ALL=C.UTF-8 \
    SUPERSET_ENV=production \
    FLASK_APP="superset.app:create_app()" \
    PYTHONPATH="/app/pythonpath" \
    SUPERSET_HOME="/app/superset_home" \
    SUPERSET_PORT=8088
 
RUN mkdir -p ${PYTHONPATH} superset/static superset-frontend apache_superset.egg-info requirements \
    && useradd --user-group -d ${SUPERSET_HOME} -m --no-log-init --shell /bin/bash superset \
    && rm -rf /var/lib/apt/lists/* \
    && echo 'Acquire::CompressionTypes::Order:: "gz";' > /etc/apt/apt.conf.d/99compression-workaround \
    && echo 'Acquire::http::Pipeline-Depth "0";' > /etc/apt/apt.conf.d/99pipeline \
    && echo 'Acquire::http::No-Cache "true";' >> /etc/apt/apt.conf.d/99pipeline \
    && echo 'Acquire::BrokenProxy "true";' >> /etc/apt/apt.conf.d/99pipeline \
    && apt-get update -qq && apt-get install -yqq --no-install-recommends \
        build-essential \
        curl \
        default-libmysqlclient-dev \
        libsasl2-dev \
        libsasl2-modules-gssapi-mit \
        libpq-dev \
        libecpg-dev \
        libldap2-dev \
    && touch superset/static/version_info.json \
    && chown -R superset:superset ./* \
    && rm -rf /var/lib/apt/lists/*
 
COPY --chown=superset:superset setup.py MANIFEST.in README.md ./
# setup.py uses the version information in package.json
COPY --chown=superset:superset superset-frontend/package.json superset-frontend/
RUN --mount=type=bind,target=./requirements/local.txt,src=./requirements/local.txt \
    --mount=type=bind,target=./requirements/development.txt,src=./requirements/development.txt \
    --mount=type=bind,target=./requirements/base.txt,src=./requirements/base.txt \
    --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements/local.txt
 
COPY --chown=superset:superset --from=superset-node /app/superset/static/assets superset/static/assets
## Lastly, let's install superset itself
COPY --chown=superset:superset superset superset
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -e . \
    && flask fab babel-compile --target superset/translations \
    && chown -R superset:superset superset/translations
 
COPY --chmod=755 ./docker/run-server.sh /usr/bin/
USER superset
 
HEALTHCHECK CMD curl -f "http://localhost:${SUPERSET_PORT}/health"
 
EXPOSE ${SUPERSET_PORT}
 
CMD ["/usr/bin/run-server.sh"]
 
######################################################################
# Dev image...
######################################################################
FROM lean AS dev
ARG GECKODRIVER_VERSION=v0.33.0 \
    FIREFOX_VERSION=117.0.1
 
USER root
 
RUN rm -rf /var/lib/apt/lists/* \
    && echo 'Acquire::CompressionTypes::Order:: "gz";' > /etc/apt/apt.conf.d/99compression-workaround \
    && echo 'Acquire::http::Pipeline-Depth "0";' > /etc/apt/apt.conf.d/99pipeline \
    && echo 'Acquire::http::No-Cache "true";' >> /etc/apt/apt.conf.d/99pipeline \
    && echo 'Acquire::BrokenProxy "true";' >> /etc/apt/apt.conf.d/99pipeline \
    && apt-get update -qq \
    && apt-get install -yqq --no-install-recommends \
        libnss3 \
        libdbus-glib-1-2 \
        libgtk-3-0 \
        libx11-xcb1 \
        libasound2 \
        libxtst6 \
        wget \
    # Install GeckoDriver WebDriver
    && wget -q https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin \
    # Install Firefox
    && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt \
    && ln -s /opt/firefox/firefox /usr/local/bin/firefox \
    && apt-get autoremove -yqq --purge wget && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*
# Cache everything for dev purposes...
RUN --mount=type=bind,target=./requirements/base.txt,src=./requirements/base.txt \
    --mount=type=bind,target=./requirements/docker.txt,src=./requirements/docker.txt \
    --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements/docker.txt
 
USER superset
######################################################################
# CI image...
######################################################################
FROM lean AS ci
 
COPY --chown=superset ./docker/docker-bootstrap.sh /app/docker/
COPY --chown=superset ./docker/docker-init.sh /app/docker/
COPY --chown=superset ./docker/docker-ci.sh /app/docker/
COPY --chown=superset ./docker/.env-non-dev /app/docker/
COPY --chown=superset ./docker/pythonpath_dev/superset_config.py /app/docker/pythonpath_dev/
COPY --chown=superset:superset --chmod=755 ./docker/*.sh /app/docker/
 
 
CMD ["/app/docker/docker-ci.sh"]

1.6 运行容器

docker run -v /etc/localtime:/etc/localtime:ro -d -p 8088:8088 --env-file=./docker/.env-non-dev --network superset4 --name superset superset4.0
#如要修改端口号，需要修改docker/.env-non-dev文件对应内容

2.pip部署

2.1 anaconda虚拟环境

打开anaconda prompt，输入命令

conda create -n superset python==3.11  #创建虚拟环境
conda activate superset                #激活虚拟环境

2.2 更新pip，并安装pillow和wheel库，安装python-geohash库

python -m pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple/
 
pip install pillow -i https://pypi.tuna.tsinghua.edu.cn/simple/
 
pip install wheel -i https://pypi.tuna.tsinghua.edu.cn/simple/

pip install python-geohash

2.3 处理依赖冲突

pip install markupsafe==2.0.1
pip install WTForms==2.3.3
pip install marshmallow==3.20

2.4 安装superset

pip install apache-superset==4.0.0 -i https://pypi.tuna.tsinghua.edu.cn/simple/

2.5 设置flask_app(每次启动都需要设置)

set FLASK_APP=superset

2.6 初始化数据库、创建管理员账户和初始化权限

superset db upgrade
说明:修改config.py文件，定位到SQLALCHEMY_DATABASE_URI，将其改为本地mysql的路径
 
 
superset fab create-admin
superset init

2.7 运行

superset run -p 8088

2.8 报错处理

#报错内容
A Default SECRET KEY was detected, please use superset config.py to override it.Use a strong complex alphanumeric string and use a tool to help you generatea sufficiently random sequence,ex:openssl rand -base64 42

解决方法：

点开superset容器 →file →app → superset → config.py →修改第181行 → 保存 → 重启容器

#将CHANGE_ME_SECRET_KEY改为自己生成的密钥
SECRET_KEY = os.environ.get("SUPERSET_SECRET_KEY") or CHANGE_ME_SECRET_KEY

密钥生成的cmd命令：

openssl rand -base64 42