You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by si...@apache.org on 2020/10/28 05:07:01 UTC
[incubator-pinot] branch hotfixtextindex updated (a122728 ->
16bbbe2)
This is an automated email from the ASF dual-hosted git repository.
siddteotia pushed a change to branch hotfixtextindex
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
from a122728 [TE] Added support for BigQuery as data source (#5868)
new a5ce72f Use docBase from leaf context for multi-segment search
new 16bbbe2 Modify .travis_install.sh
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.travis/.travis_install.sh | 13 ++++++++++---
.travis/.travis_nightly_build.sh | 2 +-
.../segment/index/readers/text/LuceneDocIdCollector.java | 6 +++++-
3 files changed, 16 insertions(+), 5 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[incubator-pinot] 02/02: Modify .travis_install.sh
Posted by si...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
siddteotia pushed a commit to branch hotfixtextindex
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 16bbbe2d99a55e63939e210a45431c832a12baf3
Author: Jack Li(Analytics Engineering) <jl...@jlli-mn1.linkedin.biz>
AuthorDate: Mon Oct 26 22:34:33 2020 -0700
Modify .travis_install.sh
---
.travis/.travis_install.sh | 13 ++++++++++---
.travis/.travis_nightly_build.sh | 2 +-
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/.travis/.travis_install.sh b/.travis/.travis_install.sh
index 5b676cc..15576d5 100755
--- a/.travis/.travis_install.sh
+++ b/.travis/.travis_install.sh
@@ -49,17 +49,24 @@ java -version
if [ $noThirdEyeChange -ne 0 ]; then
echo "Full Pinot build"
echo "No ThirdEye changes"
+
+# echo "Fixing git setup for $TRAVIS_BRANCH"
+# git checkout ${TRAVIS_BRANCH}
+# git branch -u origin/${TRAVIS_BRANCH}
+# git config branch.${TRAVIS_BRANCH}.remote origin
+# git config branch.${TRAVIS_BRANCH}.merge refs/heads/${TRAVIS_BRANCH}
+
if [ "$TRAVIS_JDK_VERSION" != 'oraclejdk8' ]; then
# JDK 11 prints more logs exceeding Travis limits.
- mvn clean install -B -DskipTests=true -Pbin-dist -Dmaven.javadoc.skip=true ${DEPLOY_BUILD_OPTS} ${KAFKA_BUILD_OPTS} > /tmp/mvn_build_log
+ mvn clean install -B -DscmBranch="${TRAVIS_BRANCH}" -DskipTests=true -Pbin-dist -Dmaven.javadoc.skip=true ${KAFKA_BUILD_OPTS} > /tmp/mvn_build_log
if [ $? -eq 0 ]; then
exit 0
else
- tail -1000 /tmp/mvn_build_log
+ cat /tmp/mvn_build_log
exit 1
fi
else
- mvn clean install -B -DskipTests=true -Pbin-dist -Dmaven.javadoc.skip=true ${DEPLOY_BUILD_OPTS} ${KAFKA_BUILD_OPTS} || exit $?
+ mvn clean install -B -DscmBranch="${TRAVIS_BRANCH}" -DskipTests=true -Pbin-dist -Dmaven.javadoc.skip=true ${KAFKA_BUILD_OPTS} || exit $?
fi
fi
diff --git a/.travis/.travis_nightly_build.sh b/.travis/.travis_nightly_build.sh
index 602a4bf..27f9447 100755
--- a/.travis/.travis_nightly_build.sh
+++ b/.travis/.travis_nightly_build.sh
@@ -27,5 +27,5 @@ if [ -n "${DEPLOY_BUILD_OPTS}" ]; then
mvn versions:commit -q -B
# Deploy to bintray
- mvn deploy -s .travis/.ci.settings.xml -DskipTests -q -DretryFailedDeploymentCount=5 -DaltDeploymentRepository=bintray-linkedin-maven::default::'https://api.bintray.com/maven/linkedin/maven/pinot/;publish=1;override=1'
+ mvn deploy -s .travis/.ci.settings.xml -DscmBranch="${TRAVIS_BRANCH}" -DskipTests -q -DretryFailedDeploymentCount=5 -DaltDeploymentRepository=bintray-linkedin-maven::default::'https://api.bintray.com/maven/linkedin/maven/pinot/;publish=1;override=1'
fi
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[incubator-pinot] 01/02: Use docBase from leaf context for
multi-segment search
Posted by si...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
siddteotia pushed a commit to branch hotfixtextindex
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit a5ce72fdf9b06a8585463265e5c56829f4879ed7
Author: Siddharth Teotia <st...@steotia-mn1.linkedin.biz>
AuthorDate: Mon Oct 26 09:39:37 2020 -0700
Use docBase from leaf context for multi-segment search
---
.../pinot/core/segment/index/readers/text/LuceneDocIdCollector.java | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/text/LuceneDocIdCollector.java b/pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/text/LuceneDocIdCollector.java
index b24667b..723dd00 100644
--- a/pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/text/LuceneDocIdCollector.java
+++ b/pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/text/LuceneDocIdCollector.java
@@ -67,7 +67,11 @@ public class LuceneDocIdCollector implements Collector {
@Override
public void collect(int doc) throws IOException {
- _docIds.add(_docIdTranslator.getPinotDocId(doc));
+ // even though we merge lucene sub-indexes, there could still be cases
+ // that can lead to multiple sub-indexes. For search on multiple
+ // sub-indexes, we need to use compute the absolute lucene docID across
+ // sub-indexes because that's how the lookup table in docIdTranslator is built
+ _docIds.add(_docIdTranslator.getPinotDocId(context.docBase + doc));
}
};
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org