You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by al...@apache.org on 2019/08/22 01:34:10 UTC
[kudu] branch master updated (b9c429c -> 0f38821)
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git.
from b9c429c [tablet] Fixed the bug of DeltaTracker::CountDeletedRows
new 98b6faa docs: correct invalid link about schema_design
new 0f38821 [thirdparty] introduce chrony
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/index.adoc | 4 +--
docs/known_issues.adoc | 2 +-
docs/kudu_impala_integration.adoc | 4 +--
docs/prior_release_notes.adoc | 4 +--
thirdparty/build-definitions.sh | 40 ++++++++++++++++++++++++++++
thirdparty/build-thirdparty.sh | 5 ++++
thirdparty/download-thirdparty.sh | 7 +++++
thirdparty/patches/chrony-no-superuser.patch | 31 +++++++++++++++++++++
thirdparty/vars.sh | 3 +++
9 files changed, 93 insertions(+), 7 deletions(-)
create mode 100644 thirdparty/patches/chrony-no-superuser.patch
[kudu] 01/02: docs: correct invalid link about schema_design
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 98b6faae52c2bc787443f242e238a2e43d57ad81
Author: Xiaokai Wang <xi...@live.com>
AuthorDate: Sat Aug 17 00:13:02 2019 +0800
docs: correct invalid link about schema_design
Change-Id: I41e8671f518bb898dd357dc5024a7d8ab335b401
Reviewed-on: http://gerrit.cloudera.org:8080/14028
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <gr...@apache.org>
---
docs/index.adoc | 4 ++--
docs/known_issues.adoc | 2 +-
docs/kudu_impala_integration.adoc | 4 ++--
docs/prior_release_notes.adoc | 4 ++--
4 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/docs/index.adoc b/docs/index.adoc
index 7a301ec..f693b6e 100644
--- a/docs/index.adoc
+++ b/docs/index.adoc
@@ -82,7 +82,7 @@ Flexible Partitioning::
pre-split tables by hash or range into a predefined number of tablets, in order
to distribute writes and queries evenly across your cluster. You can partition by
any number of primary key columns, by any number of hashes, and an optional list of
- split rows. See link:schema_design.html[Schema Design].
+ split rows. See <<schema_design.adoc#schema_design,Schema Design>>.
Parallel Scan::
To achieve the highest possible performance on modern hardware, the Kudu client
used by Impala parallelizes scans across multiple tablets.
@@ -112,7 +112,7 @@ pattern-based compression can be orders of magnitude more efficient than
compressing mixed data types, which are used in row-based solutions. Combined
with the efficiencies of reading data from columns, compression allows you to
fulfill your query while reading even fewer blocks from disk. See
-link:schema_design.html#encoding[Data Compression]
+<<schema_design.adoc#encoding,Data Compression>>
.Table
diff --git a/docs/known_issues.adoc b/docs/known_issues.adoc
index 2c9d9bf..083654b 100644
--- a/docs/known_issues.adoc
+++ b/docs/known_issues.adoc
@@ -90,7 +90,7 @@
* Tables must be manually pre-split into tablets using simple or compound primary
keys. Automatic splitting is not yet possible. Range partitions may be added
or dropped after a table has been created. See
- link:schema_design.html[Schema Design] for more information.
+ <<schema_design.adoc#schema_design,Schema Design>> for more information.
* Data in existing tables cannot currently be automatically repartitioned. As a workaround,
create a new table with the new partitioning and insert the contents of the old
diff --git a/docs/kudu_impala_integration.adoc b/docs/kudu_impala_integration.adoc
index fe69cb5..6a5e49c 100755
--- a/docs/kudu_impala_integration.adoc
+++ b/docs/kudu_impala_integration.adoc
@@ -288,7 +288,7 @@ key columns. Each tablet is served by at least one tablet server. Ideally, a tab
should be split into tablets that are distributed across a number of tablet servers
to maximize parallel operations. The details of the partitioning schema you use
will depend entirely on the type of data you store and how you access it. For a full
-discussion of schema design in Kudu, see link:schema_design.html[Schema Design].
+discussion of schema design in Kudu, see <<schema_design.adoc#schema_design,Schema Design>>.
Kudu currently has no mechanism for splitting or merging tablets after the table has
been created. You must provide a partition schema for your table when you create it.
@@ -497,7 +497,7 @@ which address scenarios like the following:
need to be added or removed, such as the introduction or elimination of a product
type.
-See link:schema_design.html[Schema Design] for the caveats of non-covering partitions.
+See <<schema_design.adoc#schema_design,Schema Design>> for the caveats of non-covering partitions.
This example creates a tablet per year (5 tablets total), for storing log data.
The table only accepts data from 2012 to 2016. Keys outside of these
diff --git a/docs/prior_release_notes.adoc b/docs/prior_release_notes.adoc
index 16fda00..34d3d56 100644
--- a/docs/prior_release_notes.adoc
+++ b/docs/prior_release_notes.adoc
@@ -902,7 +902,7 @@ Apache Kudu 1.7.1 is a bug-fix release which fixes critical issues in Kudu 1.7.0
calculations where the imprecise representation and rounding behavior of float and
double make those types impractical. The decimal type is also useful for integers
larger than int64 and cases with fractional values in a primary key.
- See link:schema_design.html#decimal[Decimal Type] for more details.
+ See <<schema_design.adoc#decimal,Decimal Type>> for more details.
* The strategy Kudu uses for automatically healing tablets which have lost a
replica due to server or disk failures has been improved. The new re-replication
@@ -2620,7 +2620,7 @@ changes between 0.8.0 and 0.9.0</a>+++.
- Default table partitioning has been removed. All tables must now be created
with explicit partitioning. Existing tables are unaffected. See the
- link:schema_design.html#no_default_partitioning[schema design guide] for more
+ <<schema_design.adoc#no_default_partitioning,schema design guide>> for more
details.
[[rn_0.9.0_new_features]]
[kudu] 02/02: [thirdparty] introduce chrony
Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 0f38821c4867b86044be22dbceb80c7bc04b5d33
Author: Alexey Serbin <al...@apache.org>
AuthorDate: Wed Jul 24 14:36:07 2019 -0700
[thirdparty] introduce chrony
Added chrony into the thirdparty: a set of follow-up patches use the
chronyd NTP implementation and its chronyc CLI to provide the
functionality of reference NTP servers for test scenarios involving
Kudu built-in NTP client.
One nice feature that chronyd has is the ability to run in server-only
mode, i.e. not driving the system clock: 'man chronyd', the '-x' option.
Also, it's possible to chronyd NTP server using the system clock
as a source instead of a physical device (GPS, oscillator, etc.).
In addition, it's possible to manually set the reference time for
chrony NTP server running in server-only local mode. The chrony suite
has 'chronyc' CLI tool to send control commands to chronyd NTP daemon.
As of now, the latest version 3.5 is used with not-yet-upstream patch
to drop the requirements of being run under superuser UID when driving
the system clock is not required (e.g., when running in server-only
mode, see '-x' command-line flag).
Change-Id: I71ed12311b10979af8a12094881b6b8b47ef8008
Reviewed-on: http://gerrit.cloudera.org:8080/13915
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Grant Henke <gr...@apache.org>
---
thirdparty/build-definitions.sh | 40 ++++++++++++++++++++++++++++
thirdparty/build-thirdparty.sh | 5 ++++
thirdparty/download-thirdparty.sh | 7 +++++
thirdparty/patches/chrony-no-superuser.patch | 31 +++++++++++++++++++++
thirdparty/vars.sh | 3 +++
5 files changed, 86 insertions(+)
diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh
index f125ea5..8dcad99 100644
--- a/thirdparty/build-definitions.sh
+++ b/thirdparty/build-definitions.sh
@@ -1004,3 +1004,43 @@ build_yaml() {
popd
done
}
+
+build_chrony() {
+ CHRONY_BDIR=$TP_BUILD_DIR/$CHRONY_NAME$MODE_SUFFIX
+ mkdir -p $CHRONY_BDIR
+ pushd $CHRONY_BDIR
+
+ # The configure script for chrony doesn't follow the common policy of
+ # the autogen tools (probably, it's manually written from scratch).
+ # It's not possible to configure and build chrony in a separate directory;
+ # it's necessary to do so in the source directory itself.
+ rsync -av --delete $CHRONY_SOURCE/ .
+
+ # In the scope of using chrony in Kudu test framework, it's better to have
+ # leaner binaries for chronyd and chronyc, stripping off everything but
+ # essential functionality.
+ CFLAGS="$EXTRA_CFLAGS" \
+ CXXFLAGS="$EXTRA_CXXFLAGS" \
+ LDFLAGS="$EXTRA_LDFLAGS" \
+ LIBS="$EXTRA_LIBS" \
+ ./configure \
+ --prefix=$PREFIX \
+ --sysconfdir=$PREFIX/etc \
+ --localstatedir=$PREFIX/var \
+ --enable-debug \
+ --disable-ipv6 \
+ --disable-pps \
+ --disable-privdrop \
+ --disable-readline \
+ --without-editline \
+ --without-nettle \
+ --without-nss \
+ --without-tomcrypt \
+ --without-libcap \
+ --without-seccomp \
+ --disable-forcednsretry \
+ --disable-sechash
+
+ make -j$PARALLEL $EXTRA_MAKEFLAGS install
+ popd
+}
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index 169300d..81d36d6 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -102,6 +102,7 @@ else
"hive") F_HIVE=1 ;;
"sentry") F_SENTRY=1 ;;
"yaml") F_YAML=1 ;;
+ "chrony") F_CHRONY=1 ;;
*) echo "Unknown module: $arg"; exit 1 ;;
esac
done
@@ -247,6 +248,10 @@ if [ -n "$F_COMMON" -o -n "$F_BISON" ]; then
build_bison
fi
+if [ -n "$F_COMMON" -o -n "$F_CHRONY" ]; then
+ build_chrony
+fi
+
# Install Hadoop, Hive, and Sentry by symlinking their source directories (which
# are pre-built) into $PREFIX/opt.
if [ -n "$F_COMMON" -o -n "$F_HADOOP" ]; then
diff --git a/thirdparty/download-thirdparty.sh b/thirdparty/download-thirdparty.sh
index ee481c3..7160c5d 100755
--- a/thirdparty/download-thirdparty.sh
+++ b/thirdparty/download-thirdparty.sh
@@ -442,5 +442,12 @@ fetch_and_patch \
$YAML_SOURCE \
$YAML_PATCHLEVEL
+CHRONY_PATCHLEVEL=1
+fetch_and_patch \
+ $CHRONY_NAME.tar.gz \
+ $CHRONY_SOURCE \
+ $CHRONY_PATCHLEVEL \
+ "patch -p1 < $TP_DIR/patches/chrony-no-superuser.patch"
+
echo "---------------"
echo "Thirdparty dependencies downloaded successfully"
diff --git a/thirdparty/patches/chrony-no-superuser.patch b/thirdparty/patches/chrony-no-superuser.patch
new file mode 100644
index 0000000..937f5d9
--- /dev/null
+++ b/thirdparty/patches/chrony-no-superuser.patch
@@ -0,0 +1,31 @@
+commit ecdaaf222a126ec9aeecb0479e2337f875a9618f
+Author: Alexey Serbin <al...@apache.org>
+Date: Tue Aug 20 16:47:02 2019 -0700
+
+ main: allow to run under regular user in server-only mode
+
+ This patch allows for running chronyd under non-superuser account when
+ no clock control is requested (i.e. when '-d' option is specified).
+
+ The motivation for this change is to run chronyd in server-only mode
+ (i.e. not driving system's clock) in various development and testing
+ environments. Of course, it's assumed non-standard ports are used
+ for the NTP and the command endpoints in such a case, otherwise
+ chronyd would fail to bind to the standard NTP (123) and
+ command (323) ports since these are privileged ones.
+
+diff --git a/main.c b/main.c
+index fafca65..d94a3b5 100644
+--- a/main.c
++++ b/main.c
+@@ -501,8 +501,9 @@ int main
+ }
+ }
+
+- if (getuid() && !client_only)
++ if (getuid() && !(client_only || !clock_control)) {
+ LOG_FATAL("Not superuser");
++ }
+
+ /* Turn into a daemon */
+ if (!nofork) {
diff --git a/thirdparty/vars.sh b/thirdparty/vars.sh
index 3bbcaa1..1c386c0 100644
--- a/thirdparty/vars.sh
+++ b/thirdparty/vars.sh
@@ -244,3 +244,6 @@ YAML_VERSION=0.6.2
YAML_NAME=yaml-cpp-yaml-cpp-$YAML_VERSION
YAML_SOURCE=$TP_SOURCE_DIR/$YAML_NAME
+CHRONY_VERSION=3.5
+CHRONY_NAME=chrony-$CHRONY_VERSION
+CHRONY_SOURCE=$TP_SOURCE_DIR/$CHRONY_NAME