You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "divinerapier (via GitHub)" <gi...@apache.org> on 2026/04/04 10:26:11 UTC
[PR] [feature](build) Add local-dev single-container FE+BE Dockerfile [doris]
divinerapier opened a new pull request, #62120:
URL: https://github.com/apache/doris/pull/62120
### What problem does this PR solve?
Problem Summary: Provide a Docker image recipe for running FE and BE in one container for local application development, with relaxed BE host checks and documented JDBC ports.
### Release note
Provide a Docker image recipe for running FE and BE in one container for local application development.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [x] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [x] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](build) Add local-dev single-container FE+BE Dockerfile [doris]
Posted by "divinerapier (via GitHub)" <gi...@apache.org>.
divinerapier commented on PR #62120:
URL: https://github.com/apache/doris/pull/62120#issuecomment-4188640599
@Thearas PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](build) Add local-dev single-container FE+BE Dockerfile [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #62120:
URL: https://github.com/apache/doris/pull/62120#issuecomment-4186901767
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](build) Add local-dev single-container FE+BE Dockerfile [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #62120:
URL: https://github.com/apache/doris/pull/62120#issuecomment-4303292761
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](build) Add local-dev single-container FE+BE Dockerfile [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #62120:
URL: https://github.com/apache/doris/pull/62120#discussion_r3129882109
##########
docker/runtime/local-dev/docker-entrypoint.sh:
##########
@@ -0,0 +1,178 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -eo pipefail
+
+# Prefer eth0/docker IP over loopback so FE/BE agree on published addresses (see log: HostInfo host=192.168.x.x).
+pick_primary_ipv4() {
+ local _tok
+ for _tok in $(hostname -I 2>/dev/null); do
+ [[ "${_tok}" == "127.0.0.1" ]] && continue
+ [[ -n "${_tok}" ]] && echo "${_tok}" && return
+ done
+ echo "127.0.0.1"
+}
+
+_DEFAULT_CONTAINER_IP="$(pick_primary_ipv4)"
+
+readonly DORIS_HOME="/opt/apache-doris"
+readonly FE_HOME="${DORIS_HOME}/fe"
+readonly BE_HOME="${DORIS_HOME}/be"
+readonly MYSQL_HOST="${MYSQL_HOST:-127.0.0.1}"
+readonly MYSQL_PORT="${MYSQL_PORT:-9030}"
+readonly FE_HTTP_PORT="${FE_HTTP_PORT:-8030}"
+readonly BE_HEARTBEAT_PORT="${BE_HEARTBEAT_PORT:-9050}"
+readonly BE_IP="${BE_IP:-${_DEFAULT_CONTAINER_IP}}"
+readonly MAX_WAIT_SECONDS="${MAX_WAIT_SECONDS:-300}"
+
+log() {
+ echo "$(date -Iseconds) [local-dev] $*"
+}
+
+_cleaned=0
+cleanup() {
+ [[ "${_cleaned}" -eq 1 ]] && return
+ _cleaned=1
+ log "Shutting down (stop BE then FE)..."
+ "${BE_HOME}/bin/stop_be.sh" 2>/dev/null || true
+ sleep 2
+ "${FE_HOME}/bin/stop_fe.sh" 2>/dev/null || true
+}
+
+on_signal() {
+ cleanup
+ exit 143
+}
+trap on_signal SIGTERM
+trap 'cleanup; exit 130' SIGINT
+trap cleanup EXIT
+
+export SKIP_CHECK_ULIMIT="${SKIP_CHECK_ULIMIT:-true}"
+
+mysql_root() {
+ mysql -uroot -h"${MYSQL_HOST}" -P"${MYSQL_PORT}" --protocol=tcp --connect-timeout=5 "$@"
+}
+
+first_fe_bootstrap() {
+ [[ ! -d "${FE_HOME}/doris-meta/image" ]]
+}
+
+priority_networks_value() {
+ if [[ -n "${PRIORITY_NETWORKS:-}" ]]; then
+ echo "${PRIORITY_NETWORKS}"
+ elif [[ "${BE_IP}" == "127.0.0.1" ]]; then
+ echo "127.0.0.1/24"
+ else
+ echo "${BE_IP%.*}.0/24"
+ fi
+}
+
+append_priority_networks() {
+ local f="$1"
+ local pn
+ pn="$(priority_networks_value)"
+ if [[ -f "$f" ]] && ! grep -qE '^[[:space:]]*priority_networks[[:space:]]*=' "$f"; then
+ log "Appending priority_networks = ${pn} to $(basename "$(dirname "$f")")/conf/$(basename "$f")"
+ echo "priority_networks = ${pn}" >>"$f"
+ fi
+}
+
+fe_http_up() {
+ curl -sf --max-time 2 "http://127.0.0.1:${FE_HTTP_PORT}/" &>/dev/null
+}
+
+wait_for_fe() {
+ local i
+ for ((i = 0; i < MAX_WAIT_SECONDS; i++)); do
+ # SELECT 1 can go through Nereids and require a BE (Doris 4.x+), while no BE is running yet — deadlock.
+ if mysql_root -e "SHOW FRONTENDS" &>/dev/null; then
+ log "FE query port is ready (SHOW FRONTENDS)."
+ return 0
+ fi
+ if ((i % 20 == 0)); then
+ if fe_http_up; then
+ log "Waiting for FE MySQL port ${MYSQL_PORT} (HTTP ${FE_HTTP_PORT} is up)... (${i}/${MAX_WAIT_SECONDS}s)"
+ else
+ log "Waiting for FE on ${MYSQL_HOST}:${MYSQL_PORT}... (${i}/${MAX_WAIT_SECONDS}s)"
+ fi
+ fi
+ sleep 1
+ done
+ log "ERROR: FE did not become ready in time."
+ return 1
+}
+
+backend_registered() {
+ mysql_root -N -e "SHOW BACKENDS" 2>/dev/null | grep -w "${BE_IP}" | grep -w "${BE_HEARTBEAT_PORT}" &>/dev/null
+}
+
+register_backend() {
+ if backend_registered; then
+ log "BE ${BE_IP}:${BE_HEARTBEAT_PORT} already registered."
+ return 0
+ fi
+ local i
+ for ((i = 0; i < MAX_WAIT_SECONDS; i++)); do
+ if mysql_root -e "ALTER SYSTEM ADD BACKEND '${BE_IP}:${BE_HEARTBEAT_PORT}'" &>/dev/null; then
Review Comment:
The README encourages reusing named volumes for `fe/doris-meta` and `be/storage`, but this code always falls back to `ALTER SYSTEM ADD BACKEND` whenever the current container IP is different from the one already stored in FE metadata. On a normal `docker rm && docker run` flow Docker usually assigns a new IP, so `backend_registered()` returns false and we create a second backend instead of reusing the persisted one. The existing `docker/runtime/be/resource/init_be.sh` intentionally skips registration when `be/storage/data` already exists for exactly this reason.
With persisted BE storage this can strand the old backend record/replicas and break the recreated cluster. Please gate registration on first BE bootstrap (or otherwise reconcile the stored backend identity) instead of always registering a new backend by transient container IP.
##########
docker/runtime/local-dev/docker-entrypoint.sh:
##########
@@ -0,0 +1,178 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -eo pipefail
+
+# Prefer eth0/docker IP over loopback so FE/BE agree on published addresses (see log: HostInfo host=192.168.x.x).
+pick_primary_ipv4() {
+ local _tok
+ for _tok in $(hostname -I 2>/dev/null); do
+ [[ "${_tok}" == "127.0.0.1" ]] && continue
+ [[ -n "${_tok}" ]] && echo "${_tok}" && return
+ done
+ echo "127.0.0.1"
+}
+
+_DEFAULT_CONTAINER_IP="$(pick_primary_ipv4)"
+
+readonly DORIS_HOME="/opt/apache-doris"
+readonly FE_HOME="${DORIS_HOME}/fe"
+readonly BE_HOME="${DORIS_HOME}/be"
+readonly MYSQL_HOST="${MYSQL_HOST:-127.0.0.1}"
+readonly MYSQL_PORT="${MYSQL_PORT:-9030}"
+readonly FE_HTTP_PORT="${FE_HTTP_PORT:-8030}"
+readonly BE_HEARTBEAT_PORT="${BE_HEARTBEAT_PORT:-9050}"
+readonly BE_IP="${BE_IP:-${_DEFAULT_CONTAINER_IP}}"
+readonly MAX_WAIT_SECONDS="${MAX_WAIT_SECONDS:-300}"
+
+log() {
+ echo "$(date -Iseconds) [local-dev] $*"
+}
+
+_cleaned=0
+cleanup() {
+ [[ "${_cleaned}" -eq 1 ]] && return
+ _cleaned=1
+ log "Shutting down (stop BE then FE)..."
+ "${BE_HOME}/bin/stop_be.sh" 2>/dev/null || true
+ sleep 2
+ "${FE_HOME}/bin/stop_fe.sh" 2>/dev/null || true
+}
+
+on_signal() {
+ cleanup
+ exit 143
+}
+trap on_signal SIGTERM
+trap 'cleanup; exit 130' SIGINT
+trap cleanup EXIT
+
+export SKIP_CHECK_ULIMIT="${SKIP_CHECK_ULIMIT:-true}"
+
+mysql_root() {
+ mysql -uroot -h"${MYSQL_HOST}" -P"${MYSQL_PORT}" --protocol=tcp --connect-timeout=5 "$@"
+}
+
+first_fe_bootstrap() {
+ [[ ! -d "${FE_HOME}/doris-meta/image" ]]
+}
+
+priority_networks_value() {
+ if [[ -n "${PRIORITY_NETWORKS:-}" ]]; then
+ echo "${PRIORITY_NETWORKS}"
+ elif [[ "${BE_IP}" == "127.0.0.1" ]]; then
+ echo "127.0.0.1/24"
+ else
+ echo "${BE_IP%.*}.0/24"
+ fi
+}
+
+append_priority_networks() {
+ local f="$1"
+ local pn
+ pn="$(priority_networks_value)"
+ if [[ -f "$f" ]] && ! grep -qE '^[[:space:]]*priority_networks[[:space:]]*=' "$f"; then
+ log "Appending priority_networks = ${pn} to $(basename "$(dirname "$f")")/conf/$(basename "$f")"
+ echo "priority_networks = ${pn}" >>"$f"
+ fi
+}
+
+fe_http_up() {
+ curl -sf --max-time 2 "http://127.0.0.1:${FE_HTTP_PORT}/" &>/dev/null
+}
+
+wait_for_fe() {
+ local i
+ for ((i = 0; i < MAX_WAIT_SECONDS; i++)); do
+ # SELECT 1 can go through Nereids and require a BE (Doris 4.x+), while no BE is running yet — deadlock.
+ if mysql_root -e "SHOW FRONTENDS" &>/dev/null; then
+ log "FE query port is ready (SHOW FRONTENDS)."
+ return 0
+ fi
+ if ((i % 20 == 0)); then
+ if fe_http_up; then
+ log "Waiting for FE MySQL port ${MYSQL_PORT} (HTTP ${FE_HTTP_PORT} is up)... (${i}/${MAX_WAIT_SECONDS}s)"
+ else
+ log "Waiting for FE on ${MYSQL_HOST}:${MYSQL_PORT}... (${i}/${MAX_WAIT_SECONDS}s)"
+ fi
+ fi
+ sleep 1
+ done
+ log "ERROR: FE did not become ready in time."
+ return 1
+}
+
+backend_registered() {
+ mysql_root -N -e "SHOW BACKENDS" 2>/dev/null | grep -w "${BE_IP}" | grep -w "${BE_HEARTBEAT_PORT}" &>/dev/null
+}
+
+register_backend() {
+ if backend_registered; then
+ log "BE ${BE_IP}:${BE_HEARTBEAT_PORT} already registered."
+ return 0
+ fi
+ local i
+ for ((i = 0; i < MAX_WAIT_SECONDS; i++)); do
+ if mysql_root -e "ALTER SYSTEM ADD BACKEND '${BE_IP}:${BE_HEARTBEAT_PORT}'" &>/dev/null; then
+ log "Registered BE ${BE_IP}:${BE_HEARTBEAT_PORT}."
+ return 0
+ fi
+ if ((i % 20 == 0)); then
+ log "Retrying ADD BACKEND... (${i}/${MAX_WAIT_SECONDS}s)"
+ fi
+ sleep 1
+ backend_registered && return 0
+ done
+ log "ERROR: Failed to register BE."
+ return 1
+}
+
+bootstrap_users() {
+ log "Ensuring admin user exists (empty password) and ADMIN_PRIV; root left as cluster default."
+ set +e
+ mysql_root --comments <<'EOSQL'
+CREATE USER IF NOT EXISTS 'admin'@'%' IDENTIFIED BY '';
+GRANT ADMIN_PRIV ON *.*.* TO 'admin'@'%';
+EOSQL
+ local st=$?
+ set -e
+ if [[ $st -ne 0 ]]; then
+ log "WARN: admin bootstrap SQL returned $st (empty password may be rejected on this version)."
+ fi
+}
+
+if first_fe_bootstrap; then
+ append_priority_networks "${BE_HOME}/conf/be.conf"
+ append_priority_networks "${FE_HOME}/conf/fe.conf"
+fi
+
+log "Container primary IPv4: ${_DEFAULT_CONTAINER_IP}; BE_IP=${BE_IP} (ALTER SYSTEM ADD BACKEND); MySQL host=${MYSQL_HOST}"
+
+export JAVA_TOOL_OPTIONS='-XX:-UseContainerSupport'
+
+log "Starting FE..."
+"${FE_HOME}/bin/start_fe.sh" --console &
+
+wait_for_fe || exit 1
+register_backend || exit 1
+bootstrap_users
+
+log "Starting BE (console, SKIP_CHECK_ULIMIT=${SKIP_CHECK_ULIMIT})..."
+"${BE_HOME}/bin/start_be.sh" --console &
+be_pid=$!
+wait "${be_pid}" || true
Review Comment:
Masking the `wait` status here makes PID 1 exit 0 when BE dies during startup or runtime. Because the EXIT trap then stops FE as well, Docker/Testcontainers will observe a clean exit even though the cluster never became usable.
```suggestion
wait "${be_pid}"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](build) Add local-dev single-container FE+BE Dockerfile [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #62120:
URL: https://github.com/apache/doris/pull/62120#issuecomment-6305442002
We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org