You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/03/07 00:12:19 UTC

[GitHub] [incubator-pinot] apucher opened a new pull request #5125: pinot-quickstart companion container for thirdeye

apucher opened a new pull request #5125: pinot-quickstart companion container for thirdeye
URL: https://github.com/apache/incubator-pinot/pull/5125
 
 
   Expands upon @fx19880617 pinot-thirdeye docker images to enable fully functional startup of ThirdEye as single companion container. The image supports dashboarding, live anomaly detection, and optional data ingestion from pinot. We integrate a dedicated H2 database instance and both, a thirdeye-dashboard process and a thirdeye-controller process. Furthermore, we support multiple startup modes:
   
   * **default** - ThirdEye with repo default configs
   * **ephemeral** - ThirdEye with mock data generated on the fly (pinot not required)
   * **pinot-thirdeye** - ThirdEye with auto-ingestion from pinot-quickstart
   
   Docker image build: `docker/images/pinot-thirdeye/docker-build.sh`
   
   Docker container launch: `docker run --network=pinot-demo --name thirdeye -p 1426:1426 -p 1427:1427 -d thirdeye:latest pinot-quickstart`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] apucher commented on a change in pull request #5125: pinot-quickstart companion container for thirdeye

Posted by GitBox <gi...@apache.org>.
apucher commented on a change in pull request #5125: pinot-quickstart companion container for thirdeye
URL: https://github.com/apache/incubator-pinot/pull/5125#discussion_r389213784
 
 

 ##########
 File path: docker/images/pinot-thirdeye/README.md
 ##########
 @@ -0,0 +1,114 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# docker-pinot
+This is a docker image of [Apache Thirdeye](https://github.com/apache/incubator-pinot/tree/master/thirdeye).
+
+## How to build a docker image
+
+There is a docker build script which will build a given Git repo/branch and tag the image.
+
+Usage:
+
+```SHELL
+./docker-build.sh [Docker Tag] [Git Branch] [Pinot Git URL]
+```
+
+This script will check out Pinot Repo `[Pinot Git URL]` on branch `[Git Branch]` and build the docker image for that.
+
+The docker image is tagged as `[Docker Tag]`.
+
+`Docker Tag`: Name and tag your docker image. Default is `thirdeye:latest`.
+
+`Git Branch`: The Pinot branch to build. Default is `master`.
+
+`Pinot Git URL`: The Pinot Git Repo to build, users can set it to their own fork. Please note that, the URL is `https://` based, not `git://`. Default is the Apache Repo: `https://github.com/apache/incubator-pinot.git`.
+
+* Example of building and tagging a snapshot on your own fork:
+```SHELL
+./docker-build.sh thirdeye:latest master https://github.com/apache/incubator-pinot.git
+```
+
+## How to publish a docker image
+
+Script `docker-push.sh` publishes a given docker image to your docker registry.
+
+In order to push to your own repo, the image needs to be explicitly tagged with the repo name.
+
+* Example of publishing a image to [apachepinot/thirdeye](https://cloud.docker.com/u/apachepinot/repository/docker/apachepinot/thirdeye) dockerHub repo.
+
+```SHELL
+docker tag thirdeye:latest apachepinot/thirdeye:latest
+./docker-push.sh apachepinot/thirdeye:latest
+```
+
+Script `docker-build-and-push.sh` builds and publishes this docker image to your docker registry after build.
+
+* Example of building and publishing a image to [apachepinot/thirdeye](https://cloud.docker.com/u/apachepinot/repository/docker/apachepinot/thirdeye) dockerHub repo.
+
+```SHELL
+./docker-build-and-push.sh apachepinot/thirdeye:latest master https://github.com/apache/incubator-pinot.git
+```
+
+## How to Run it
+
+The entry point of docker image is `start-thirdeye.sh` script.
+
+* Create an isolated bridge network in docker.
+
+```SHELL
+docker network create -d bridge pinot-demo
+```
+
+* Start Pinot Hybrid Quickstart
+
+```SHELL
+docker run \
+    --network=pinot-demo \
+    --name pinot-quickstart \
+    -p 9000:9000 \
+    -d apachepinot/pinot:latest QuickStart \
+    -type hybrid
+```
+
+* Start ThirdEye backend
 
 Review comment:
   good catch. updated.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io commented on issue #5125: pinot-quickstart companion container for thirdeye

Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #5125: pinot-quickstart companion container for thirdeye
URL: https://github.com/apache/incubator-pinot/pull/5125#issuecomment-596035424
 
 
   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5125?src=pr&el=h1) Report
   > Merging [#5125](https://codecov.io/gh/apache/incubator-pinot/pull/5125?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/91850cd68ec671028f31031c0fde7b32d519e99e?src=pr&el=desc) will **increase** coverage by `8.22%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/5125/graphs/tree.svg?width=650&token=4ibza2ugkz&height=150&src=pr)](https://codecov.io/gh/apache/incubator-pinot/pull/5125?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #5125      +/-   ##
   ============================================
   + Coverage     49.29%   57.52%   +8.22%     
     Complexity       12       12              
   ============================================
     Files          1183     1183              
     Lines         62377    62369       -8     
     Branches       9139     9136       -3     
   ============================================
   + Hits          30748    35877    +5129     
   + Misses        29196    23848    -5348     
   - Partials       2433     2644     +211
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/5125?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [.../java/org/apache/pinot/spi/data/TimeFieldSpec.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvZGF0YS9UaW1lRmllbGRTcGVjLmphdmE=) | `92.59% <0%> (-1.24%)` | `0% <0%> (ø)` | |
   | [...pql/parsers/PinotQuery2BrokerRequestConverter.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wcWwvcGFyc2Vycy9QaW5vdFF1ZXJ5MkJyb2tlclJlcXVlc3RDb252ZXJ0ZXIuamF2YQ==) | `88.07% <0%> (+0.45%)` | `0% <0%> (ø)` | :arrow_down: |
   | [...ment/creator/impl/SegmentColumnarIndexCreator.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2NyZWF0b3IvaW1wbC9TZWdtZW50Q29sdW1uYXJJbmRleENyZWF0b3IuamF2YQ==) | `88.25% <0%> (+0.75%)` | `0% <0%> (ø)` | :arrow_down: |
   | [.../helix/core/realtime/SegmentCompletionManager.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL3JlYWx0aW1lL1NlZ21lbnRDb21wbGV0aW9uTWFuYWdlci5qYXZh) | `70.63% <0%> (+0.85%)` | `0% <0%> (ø)` | :arrow_down: |
   | [.../FixedByteSingleColumnSingleValueReaderWriter.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9pby9yZWFkZXJ3cml0ZXIvaW1wbC9GaXhlZEJ5dGVTaW5nbGVDb2x1bW5TaW5nbGVWYWx1ZVJlYWRlcldyaXRlci5qYXZh) | `98.94% <0%> (+1.05%)` | `0% <0%> (ø)` | :arrow_down: |
   | [...t/common/response/broker/BrokerResponseNative.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vcmVzcG9uc2UvYnJva2VyL0Jyb2tlclJlc3BvbnNlTmF0aXZlLmphdmE=) | `90.52% <0%> (+1.05%)` | `0% <0%> (ø)` | :arrow_down: |
   | [...not/broker/broker/helix/ClusterChangeMediator.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0NsdXN0ZXJDaGFuZ2VNZWRpYXRvci5qYXZh) | `74.72% <0%> (+1.09%)` | `0% <0%> (ø)` | :arrow_down: |
   | [...e/io/writer/impl/MutableOffHeapByteArrayStore.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9pby93cml0ZXIvaW1wbC9NdXRhYmxlT2ZmSGVhcEJ5dGVBcnJheVN0b3JlLmphdmE=) | `85.71% <0%> (+1.09%)` | `0% <0%> (ø)` | :arrow_down: |
   | [.../pinot/broker/broker/helix/HelixBrokerStarter.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0hlbGl4QnJva2VyU3RhcnRlci5qYXZh) | `71.97% <0%> (+1.27%)` | `0% <0%> (ø)` | :arrow_down: |
   | [...g/apache/pinot/common/request/InstanceRequest.java](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vcmVxdWVzdC9JbnN0YW5jZVJlcXVlc3QuamF2YQ==) | `28.83% <0%> (+1.37%)` | `0% <0%> (ø)` | :arrow_down: |
   | ... and [298 more](https://codecov.io/gh/apache/incubator-pinot/pull/5125/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5125?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5125?src=pr&el=footer). Last update [91850cd...a48149d](https://codecov.io/gh/apache/incubator-pinot/pull/5125?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #5125: pinot-quickstart companion container for thirdeye

Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #5125: pinot-quickstart companion container for thirdeye
URL: https://github.com/apache/incubator-pinot/pull/5125#discussion_r389201947
 
 

 ##########
 File path: docker/images/pinot-thirdeye/README.md
 ##########
 @@ -0,0 +1,114 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# docker-pinot
+This is a docker image of [Apache Thirdeye](https://github.com/apache/incubator-pinot/tree/master/thirdeye).
+
+## How to build a docker image
+
+There is a docker build script which will build a given Git repo/branch and tag the image.
+
+Usage:
+
+```SHELL
+./docker-build.sh [Docker Tag] [Git Branch] [Pinot Git URL]
+```
+
+This script will check out Pinot Repo `[Pinot Git URL]` on branch `[Git Branch]` and build the docker image for that.
+
+The docker image is tagged as `[Docker Tag]`.
+
+`Docker Tag`: Name and tag your docker image. Default is `thirdeye:latest`.
+
+`Git Branch`: The Pinot branch to build. Default is `master`.
+
+`Pinot Git URL`: The Pinot Git Repo to build, users can set it to their own fork. Please note that, the URL is `https://` based, not `git://`. Default is the Apache Repo: `https://github.com/apache/incubator-pinot.git`.
+
+* Example of building and tagging a snapshot on your own fork:
+```SHELL
+./docker-build.sh thirdeye:latest master https://github.com/apache/incubator-pinot.git
+```
+
+## How to publish a docker image
+
+Script `docker-push.sh` publishes a given docker image to your docker registry.
+
+In order to push to your own repo, the image needs to be explicitly tagged with the repo name.
+
+* Example of publishing a image to [apachepinot/thirdeye](https://cloud.docker.com/u/apachepinot/repository/docker/apachepinot/thirdeye) dockerHub repo.
+
+```SHELL
+docker tag thirdeye:latest apachepinot/thirdeye:latest
+./docker-push.sh apachepinot/thirdeye:latest
+```
+
+Script `docker-build-and-push.sh` builds and publishes this docker image to your docker registry after build.
+
+* Example of building and publishing a image to [apachepinot/thirdeye](https://cloud.docker.com/u/apachepinot/repository/docker/apachepinot/thirdeye) dockerHub repo.
+
+```SHELL
+./docker-build-and-push.sh apachepinot/thirdeye:latest master https://github.com/apache/incubator-pinot.git
+```
+
+## How to Run it
+
+The entry point of docker image is `start-thirdeye.sh` script.
+
+* Create an isolated bridge network in docker.
+
+```SHELL
+docker network create -d bridge pinot-demo
+```
+
+* Start Pinot Hybrid Quickstart
+
+```SHELL
+docker run \
+    --network=pinot-demo \
+    --name pinot-quickstart \
+    -p 9000:9000 \
+    -d apachepinot/pinot:latest QuickStart \
+    -type hybrid
+```
+
+* Start ThirdEye backend
 
 Review comment:
   Update this ?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] apucher merged pull request #5125: pinot-quickstart companion container for thirdeye

Posted by GitBox <gi...@apache.org>.
apucher merged pull request #5125: pinot-quickstart companion container for thirdeye
URL: https://github.com/apache/incubator-pinot/pull/5125
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org