You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/08/24 20:41:59 UTC
[GitHub] [incubator-pinot] suvodeep-pyne opened a new pull request #5914: [TE] Added Docker Images for a distributed ThirdEye setup
suvodeep-pyne opened a new pull request #5914:
URL: https://github.com/apache/incubator-pinot/pull/5914
Added Docker Images for a distributed ThirdEye setup. Changes:
- Created docker images for frontend and backend servers using `thirdeye` as the base image.
- No breaking changes
- Update readme
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] suvodeep-pyne commented on a change in pull request #5914: [TE] Updated ThirdEye docker launch script to accept MODE as a second arg
Posted by GitBox <gi...@apache.org>.
suvodeep-pyne commented on a change in pull request #5914:
URL: https://github.com/apache/incubator-pinot/pull/5914#discussion_r475956456
##########
File path: docker/images/pinot-thirdeye/bin/start-thirdeye.sh
##########
@@ -18,32 +18,64 @@
# under the License.
#
+# Script Usage
+# ---------------------------------------------
+# ./start-thirdeye.sh ${CONFIG_DIR} ${MODE}
+#
+# - CONFIG_DIR: path to the thirdeye configuration director
+# - MODE: Choices: {frontend, backend, * }
+# frontend: Start the frontend server only
+# backend: Start the backend server only
+# For any other value, defaults to starting all services with an h2 db.
+#
+
if [[ "$#" -gt 0 ]]
then
CONFIG_DIR="./config/$1"
else
CONFIG_DIR="./config/default"
fi
-echo "Starting H2 database server"
-java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.Server -tcp -baseDir "${CONFIG_DIR}/.." &
-sleep 1
-echo "Creating ThirdEye database schema"
-java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.RunScript -user "sa" -password "sa" -url "jdbc:h2:tcp:localhost/h2db" -script "zip:./bin/thirdeye-pinot.jar!/schema/create-schema.sql"
+function start_server {
+ class_ref=$1
+ config_dir=$2
+ java -Dlog4j.configurationFile=log4j2.xml -cp "./bin/thirdeye-pinot.jar" ${class_ref} "${config_dir}"
+}
-if [ -f "${CONFIG_DIR}/bootstrap.sql" ]; then
- echo "Running database bootstrap script ${CONFIG_DIR}/bootstrap.sql"
- java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.RunScript -user "sa" -password "sa" -url "jdbc:h2:tcp:localhost/h2db" -script "${CONFIG_DIR}/bootstrap.sql"
-fi
+function start_frontend {
+ echo "Running Thirdeye frontend config: ${CONFIG_DIR}"
+ start_server org.apache.pinot.thirdeye.dashboard.ThirdEyeDashboardApplication "${CONFIG_DIR}"
+}
+
+function start_backend {
+ echo "Running Thirdeye backend config: ${CONFIG_DIR}"
+ start_server org.apache.pinot.thirdeye.anomaly.ThirdEyeAnomalyApplication "${CONFIG_DIR}"
+}
+
+function start_all {
Review comment:
The h2 launch doesn't seem to be a background job in the first place. I'm not sure whether the current setup launches the db server in a separate process and how it works. Ideally, starting multiple services in a single docker itself has some issues. So, I am not refactoring that part which I'm not able to test. I think launching a MySQL DB instead of H2 should just simplify things a lot.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] apucher commented on a change in pull request #5914: [TE] Updated ThirdEye docker launch script to accept MODE as a second arg
Posted by GitBox <gi...@apache.org>.
apucher commented on a change in pull request #5914:
URL: https://github.com/apache/incubator-pinot/pull/5914#discussion_r475938967
##########
File path: docker/images/pinot-thirdeye/bin/start-thirdeye.sh
##########
@@ -18,32 +18,64 @@
# under the License.
#
+# Script Usage
+# ---------------------------------------------
+# ./start-thirdeye.sh ${CONFIG_DIR} ${MODE}
+#
+# - CONFIG_DIR: path to the thirdeye configuration director
+# - MODE: Choices: {frontend, backend, * }
+# frontend: Start the frontend server only
+# backend: Start the backend server only
+# For any other value, defaults to starting all services with an h2 db.
+#
+
if [[ "$#" -gt 0 ]]
then
CONFIG_DIR="./config/$1"
else
CONFIG_DIR="./config/default"
fi
-echo "Starting H2 database server"
-java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.Server -tcp -baseDir "${CONFIG_DIR}/.." &
-sleep 1
-echo "Creating ThirdEye database schema"
-java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.RunScript -user "sa" -password "sa" -url "jdbc:h2:tcp:localhost/h2db" -script "zip:./bin/thirdeye-pinot.jar!/schema/create-schema.sql"
+function start_server {
+ class_ref=$1
+ config_dir=$2
+ java -Dlog4j.configurationFile=log4j2.xml -cp "./bin/thirdeye-pinot.jar" ${class_ref} "${config_dir}"
+}
-if [ -f "${CONFIG_DIR}/bootstrap.sql" ]; then
- echo "Running database bootstrap script ${CONFIG_DIR}/bootstrap.sql"
- java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.RunScript -user "sa" -password "sa" -url "jdbc:h2:tcp:localhost/h2db" -script "${CONFIG_DIR}/bootstrap.sql"
-fi
+function start_frontend {
+ echo "Running Thirdeye frontend config: ${CONFIG_DIR}"
+ start_server org.apache.pinot.thirdeye.dashboard.ThirdEyeDashboardApplication "${CONFIG_DIR}"
+}
+
+function start_backend {
+ echo "Running Thirdeye backend config: ${CONFIG_DIR}"
+ start_server org.apache.pinot.thirdeye.anomaly.ThirdEyeAnomalyApplication "${CONFIG_DIR}"
+}
+
+function start_all {
Review comment:
maybe add a **start_database** function for h2 specifically, so it can be started just like front and backend. imo otherwise **start_all** has an "unexpected" side-effect of bringing up a db
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] suvodeep-pyne commented on a change in pull request #5914: [TE] Updated ThirdEye docker launch script to accept MODE as a second arg
Posted by GitBox <gi...@apache.org>.
suvodeep-pyne commented on a change in pull request #5914:
URL: https://github.com/apache/incubator-pinot/pull/5914#discussion_r475983399
##########
File path: docker/images/pinot-thirdeye/bin/start-thirdeye.sh
##########
@@ -18,32 +18,64 @@
# under the License.
#
+# Script Usage
+# ---------------------------------------------
+# ./start-thirdeye.sh ${CONFIG_DIR} ${MODE}
+#
+# - CONFIG_DIR: path to the thirdeye configuration director
+# - MODE: Choices: {frontend, backend, * }
+# frontend: Start the frontend server only
+# backend: Start the backend server only
+# For any other value, defaults to starting all services with an h2 db.
+#
+
if [[ "$#" -gt 0 ]]
then
CONFIG_DIR="./config/$1"
else
CONFIG_DIR="./config/default"
fi
-echo "Starting H2 database server"
-java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.Server -tcp -baseDir "${CONFIG_DIR}/.." &
-sleep 1
-echo "Creating ThirdEye database schema"
-java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.RunScript -user "sa" -password "sa" -url "jdbc:h2:tcp:localhost/h2db" -script "zip:./bin/thirdeye-pinot.jar!/schema/create-schema.sql"
+function start_server {
+ class_ref=$1
+ config_dir=$2
+ java -Dlog4j.configurationFile=log4j2.xml -cp "./bin/thirdeye-pinot.jar" ${class_ref} "${config_dir}"
+}
-if [ -f "${CONFIG_DIR}/bootstrap.sql" ]; then
- echo "Running database bootstrap script ${CONFIG_DIR}/bootstrap.sql"
- java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.RunScript -user "sa" -password "sa" -url "jdbc:h2:tcp:localhost/h2db" -script "${CONFIG_DIR}/bootstrap.sql"
-fi
+function start_frontend {
+ echo "Running Thirdeye frontend config: ${CONFIG_DIR}"
+ start_server org.apache.pinot.thirdeye.dashboard.ThirdEyeDashboardApplication "${CONFIG_DIR}"
+}
+
+function start_backend {
+ echo "Running Thirdeye backend config: ${CONFIG_DIR}"
+ start_server org.apache.pinot.thirdeye.anomaly.ThirdEyeAnomalyApplication "${CONFIG_DIR}"
+}
+
+function start_all {
Review comment:
I see. My bad. Let me update this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 merged pull request #5914: [TE] Updated ThirdEye docker launch script to accept MODE as a second arg
Posted by GitBox <gi...@apache.org>.
fx19880617 merged pull request #5914:
URL: https://github.com/apache/incubator-pinot/pull/5914
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] apucher commented on a change in pull request #5914: [TE] Updated ThirdEye docker launch script to accept MODE as a second arg
Posted by GitBox <gi...@apache.org>.
apucher commented on a change in pull request #5914:
URL: https://github.com/apache/incubator-pinot/pull/5914#discussion_r475973847
##########
File path: docker/images/pinot-thirdeye/bin/start-thirdeye.sh
##########
@@ -18,32 +18,64 @@
# under the License.
#
+# Script Usage
+# ---------------------------------------------
+# ./start-thirdeye.sh ${CONFIG_DIR} ${MODE}
+#
+# - CONFIG_DIR: path to the thirdeye configuration director
+# - MODE: Choices: {frontend, backend, * }
+# frontend: Start the frontend server only
+# backend: Start the backend server only
+# For any other value, defaults to starting all services with an h2 db.
+#
+
if [[ "$#" -gt 0 ]]
then
CONFIG_DIR="./config/$1"
else
CONFIG_DIR="./config/default"
fi
-echo "Starting H2 database server"
-java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.Server -tcp -baseDir "${CONFIG_DIR}/.." &
-sleep 1
-echo "Creating ThirdEye database schema"
-java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.RunScript -user "sa" -password "sa" -url "jdbc:h2:tcp:localhost/h2db" -script "zip:./bin/thirdeye-pinot.jar!/schema/create-schema.sql"
+function start_server {
+ class_ref=$1
+ config_dir=$2
+ java -Dlog4j.configurationFile=log4j2.xml -cp "./bin/thirdeye-pinot.jar" ${class_ref} "${config_dir}"
+}
-if [ -f "${CONFIG_DIR}/bootstrap.sql" ]; then
- echo "Running database bootstrap script ${CONFIG_DIR}/bootstrap.sql"
- java -cp "./bin/thirdeye-pinot.jar" org.h2.tools.RunScript -user "sa" -password "sa" -url "jdbc:h2:tcp:localhost/h2db" -script "${CONFIG_DIR}/bootstrap.sql"
-fi
+function start_frontend {
+ echo "Running Thirdeye frontend config: ${CONFIG_DIR}"
+ start_server org.apache.pinot.thirdeye.dashboard.ThirdEyeDashboardApplication "${CONFIG_DIR}"
+}
+
+function start_backend {
+ echo "Running Thirdeye backend config: ${CONFIG_DIR}"
+ start_server org.apache.pinot.thirdeye.anomaly.ThirdEyeAnomalyApplication "${CONFIG_DIR}"
+}
+
+function start_all {
Review comment:
H2 *does* start in the background on line 58. The other calls on line 62 and 66 create the schema and bootstrap some metadata, respectively.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] suvodeep-pyne commented on pull request #5914: [TE] Updated ThirdEye docker launch script to accept MODE as a second arg
Posted by GitBox <gi...@apache.org>.
suvodeep-pyne commented on pull request #5914:
URL: https://github.com/apache/incubator-pinot/pull/5914#issuecomment-679415847
Thanks @apucher !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org