You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by ad...@apache.org on 2019/04/10 00:07:07 UTC
[kudu] branch master updated (66485e1 -> 36146d5)
This is an automated email from the ASF dual-hosted git repository.
adar pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git.
from 66485e1 sentry_authz_provider-test: fix logical conflict
new 216dc37 KUDU-2449: "Tablet server addition" best practices
new 424f1dc sentry: populate TablePrivilegePBs
new 49d3438 remove schema copies from RowBlock and RowBuilder
new 36146d5 tool: perf tablet_scan action
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/administration.adoc | 33 ++++
src/kudu/codegen/codegen-test.cc | 14 +-
src/kudu/common/generic_iterators-test.cc | 6 +-
src/kudu/common/generic_iterators.cc | 6 +-
src/kudu/common/row.h | 38 +++--
src/kudu/common/row_changelist-test.cc | 6 +-
src/kudu/common/rowblock.cc | 14 +-
src/kudu/common/rowblock.h | 62 ++++----
src/kudu/common/schema-test.cc | 6 +-
src/kudu/common/wire_protocol-test.cc | 8 +-
src/kudu/common/wire_protocol.cc | 6 +-
src/kudu/integration-tests/linked_list-test-util.h | 2 +-
src/kudu/master/authz_provider.h | 16 ++
src/kudu/master/default_authz_provider.h | 19 +++
src/kudu/master/sentry_authz_provider-test.cc | 167 ++++++++++++++++++---
src/kudu/master/sentry_authz_provider.cc | 77 ++++++++++
src/kudu/master/sentry_authz_provider.h | 17 ++-
src/kudu/master/sys_catalog.cc | 2 +-
src/kudu/tablet/cfile_set-test.cc | 8 +-
src/kudu/tablet/compaction-test.cc | 14 +-
src/kudu/tablet/compaction.cc | 8 +-
src/kudu/tablet/delta_compaction.cc | 2 +-
src/kudu/tablet/deltafile-test.cc | 2 +-
src/kudu/tablet/diskrowset-test-base.h | 12 +-
src/kudu/tablet/diskrowset-test.cc | 28 ++--
src/kudu/tablet/diskrowset.cc | 2 +-
src/kudu/tablet/memrowset-test.cc | 16 +-
src/kudu/tablet/mt-rowset_delta_compaction-test.cc | 2 +-
src/kudu/tablet/mt-tablet-test.cc | 6 +-
src/kudu/tablet/tablet-decoder-eval-test.cc | 21 +--
src/kudu/tablet/tablet-pushdown-test.cc | 4 +-
src/kudu/tablet/tablet-test-base.h | 2 +-
src/kudu/tablet/tablet-test-util.h | 8 +-
src/kudu/tablet/tablet-test.cc | 6 +-
src/kudu/tablet/tablet_random_access-test.cc | 2 +-
src/kudu/tools/kudu-tool-test.cc | 25 ++-
src/kudu/tools/tool_action_perf.cc | 128 +++++++++++++++-
src/kudu/tserver/tablet_server-test-base.cc | 4 +-
src/kudu/tserver/tablet_service.cc | 4 +-
39 files changed, 626 insertions(+), 177 deletions(-)
[kudu] 01/04: KUDU-2449: "Tablet server addition" best practices
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 216dc377c92c983b4693d16ef880f9ec0ebb1d8d
Author: Mitch Barnett <mb...@cloudera.com>
AuthorDate: Tue Apr 9 11:24:18 2019 -0700
KUDU-2449: "Tablet server addition" best practices
This section will detail the best practices around adding new tablet
servers to a cluster, specifically around node utilization. The most
common use-case for adding a tablet server is to reduce load/utilization
per node, or add additional capacity to the cluster. In either case, the
newly added tablet server instances won't be utilized unless new tablets are
created, or if tablets need to be re-homed, which is not the usual expectation
(I added new tablet servers, Kudu should detect that and use rebalance
automatically).
Change-Id: I7e44b4a84a8ca8048b8dc930cc1ee228daef6cf9
Reviewed-on: http://gerrit.cloudera.org:8080/12969
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <gr...@apache.org>
---
docs/administration.adoc | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/docs/administration.adoc b/docs/administration.adoc
index 2794b40..d4cfe35 100644
--- a/docs/administration.adoc
+++ b/docs/administration.adoc
@@ -797,6 +797,39 @@ INVALIDATE METADATA;
.. Verify updating the metadata worked by running a simple `SELECT` query on a
Kudu-backed Impala table.
+[[adding_tablet_servers]]
+=== Best Practices When Adding New Tablet Servers
+
+A common workflow when administering a Kudu cluster is adding additional tablet
+server instances, in an effort to increase storage capacity, decrease load or
+utilization on individual hosts, increase compute power, etc.
+
+By default, any newly added tablet servers will not be utilized immediately
+after their addition to the cluster. Instead, newly added tablet servers will
+only be utilized when new tablets are created or when existing tablets need to
+be replicated, which can lead to imbalanced nodes.
+
+To add additional tablet servers to an existing cluster, the
+following steps can be taken to ensure tablets are uniformly distributed
+across the cluster:
+
+1. Ensure that Kudu is installed on the new machines being added to the
+cluster, and that the new instances have been
+link:https://kudu.apache.org/docs/configuration.html#_configuring_tablet_servers[correctly configured]
+to point to the pre-existing cluster. Then, start up the new tablet server instances.
+2. Verify that the new instances check in with the Kudu Master(s)
+successfully. A quick method for veryifying they've successfully checked in
+with the existing Master instances is to view the Kudu Master WebUI,
+specifically the `/tablet-servers` section, and validate that the newly
+added instances are registered, and heartbeating.
+3. Once the tablet server(s) are successfully online and healthy, follow
+the steps to run the
+link:https://kudu.apache.org/docs/administration.html#rebalancer_tool[rebalancing tool]
+which will spread existing tablets to the newly added tablet server
+nodes.
+4. After the balancer has completed, or even during its execution, you can
+check on the health of the cluster using the `ksck` command-line utility.
+
[[ksck]]
=== Checking Cluster Health with `ksck`
[kudu] 02/04: sentry: populate TablePrivilegePBs
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 424f1dcf41ca121406b46507356db7ca44ffd2a5
Author: Andrew Wong <aw...@apache.org>
AuthorDate: Wed Apr 3 17:19:39 2019 -0700
sentry: populate TablePrivilegePBs
This patch allows the authz provider to populate TablePrivilegePBs,
which will be used as authorization metadata for authz tokens. A
follow-up patch will integrate this functionality into the
GetTableSchema endpoint.
Change-Id: Icb6c533a04d2749e0ee68546dac61011432cba2f
Reviewed-on: http://gerrit.cloudera.org:8080/12941
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Kudu Jenkins
Reviewed-by: Hao Hao <ha...@cloudera.com>
---
src/kudu/master/authz_provider.h | 16 +++
src/kudu/master/default_authz_provider.h | 19 +++
src/kudu/master/sentry_authz_provider-test.cc | 167 ++++++++++++++++++++++----
src/kudu/master/sentry_authz_provider.cc | 77 ++++++++++++
src/kudu/master/sentry_authz_provider.h | 17 ++-
5 files changed, 271 insertions(+), 25 deletions(-)
diff --git a/src/kudu/master/authz_provider.h b/src/kudu/master/authz_provider.h
index d45f03f..4231fde 100644
--- a/src/kudu/master/authz_provider.h
+++ b/src/kudu/master/authz_provider.h
@@ -24,6 +24,13 @@
#include "kudu/util/status.h"
namespace kudu {
+
+class SchemaPB;
+
+namespace security {
+class TablePrivilegePB;
+} // namespace security
+
namespace master {
// An interface for handling authorizations on Kudu operations.
@@ -71,6 +78,15 @@ class AuthzProvider {
virtual Status AuthorizeGetTableMetadata(const std::string& table_name,
const std::string& user) WARN_UNUSED_RESULT = 0;
+ // Populates the privilege fields of 'pb' with the table-specific privileges
+ // for the given user, using 'schema_pb' for metadata (e.g. column IDs). This
+ // does not populate the table ID field of 'pb' -- only the privilege fields;
+ // as such, it is expected that the table ID field is already set.
+ virtual Status FillTablePrivilegePB(const std::string& table_name,
+ const std::string& user,
+ const SchemaPB& schema_pb,
+ security::TablePrivilegePB* pb) WARN_UNUSED_RESULT = 0;
+
virtual ~AuthzProvider() {}
// Checks if the given user is trusted and thus can be exempted from
diff --git a/src/kudu/master/default_authz_provider.h b/src/kudu/master/default_authz_provider.h
index 6c512f6..516dab1 100644
--- a/src/kudu/master/default_authz_provider.h
+++ b/src/kudu/master/default_authz_provider.h
@@ -19,10 +19,16 @@
#include <string>
+#include <glog/logging.h>
+
#include "kudu/master/authz_provider.h"
+#include "kudu/security/token.pb.h"
#include "kudu/util/status.h"
namespace kudu {
+
+class SchemaPB;
+
namespace master {
// Default AuthzProvider which always authorizes any operations.
@@ -54,6 +60,19 @@ class DefaultAuthzProvider : public AuthzProvider {
const std::string& /*user*/) override WARN_UNUSED_RESULT {
return Status::OK();
}
+
+ Status FillTablePrivilegePB(const std::string& /*table_name*/,
+ const std::string& /*user*/,
+ const SchemaPB& /*schema_pb*/,
+ security::TablePrivilegePB* pb) override WARN_UNUSED_RESULT {
+ DCHECK(pb);
+ DCHECK(pb->has_table_id());
+ pb->set_delete_privilege(true);
+ pb->set_insert_privilege(true);
+ pb->set_scan_privilege(true);
+ pb->set_update_privilege(true);
+ return Status::OK();
+ }
};
} // namespace master
diff --git a/src/kudu/master/sentry_authz_provider-test.cc b/src/kudu/master/sentry_authz_provider-test.cc
index bb2df0a..853998d 100644
--- a/src/kudu/master/sentry_authz_provider-test.cc
+++ b/src/kudu/master/sentry_authz_provider-test.cc
@@ -21,17 +21,25 @@
#include <memory>
#include <ostream>
#include <string>
+#include <unordered_map>
+#include <unordered_set>
#include <vector>
#include <gflags/gflags_declare.h>
#include <glog/logging.h>
+#include <google/protobuf/stubs/port.h>
+#include <google/protobuf/util/message_differencer.h>
#include <gtest/gtest.h>
+#include "kudu/common/common.pb.h"
+#include "kudu/common/schema.h"
+#include "kudu/common/wire_protocol.h"
#include "kudu/gutil/macros.h"
#include "kudu/gutil/map-util.h"
#include "kudu/gutil/ref_counted.h"
#include "kudu/gutil/strings/substitute.h"
#include "kudu/master/sentry_authz_provider-test-base.h"
+#include "kudu/security/token.pb.h"
#include "kudu/sentry/mini_sentry.h"
#include "kudu/sentry/sentry-test-base.h"
#include "kudu/sentry/sentry_action.h"
@@ -41,6 +49,7 @@
#include "kudu/util/hdr_histogram.h"
#include "kudu/util/metrics.h"
#include "kudu/util/net/net_util.h"
+#include "kudu/util/pb_util.h"
#include "kudu/util/random.h"
#include "kudu/util/random_util.h"
#include "kudu/util/status.h"
@@ -60,17 +69,22 @@ METRIC_DECLARE_counter(sentry_client_reconnections_succeeded);
METRIC_DECLARE_counter(sentry_client_reconnections_failed);
METRIC_DECLARE_histogram(sentry_client_task_execution_time_us);
+using kudu::pb_util::SecureDebugString;
+using kudu::security::ColumnPrivilegePB;
+using kudu::security::TablePrivilegePB;
using kudu::sentry::AuthorizableScopesSet;
using kudu::sentry::SentryAction;
using kudu::sentry::SentryActionsSet;
using kudu::sentry::SentryTestBase;
using kudu::sentry::SentryAuthorizableScope;
+using google::protobuf::util::MessageDifferencer;
using sentry::TSentryAuthorizable;
using sentry::TSentryGrantOption;
using sentry::TSentryPrivilege;
using std::string;
-using std::tuple;
using std::unique_ptr;
+using std::unordered_map;
+using std::unordered_set;
using std::vector;
using strings::Substitute;
@@ -206,6 +220,23 @@ class SentryAuthzProviderTest : public SentryTestBase {
namespace {
+const SentryActionsSet kAllActions({
+ SentryAction::ALL,
+ SentryAction::METADATA,
+ SentryAction::SELECT,
+ SentryAction::INSERT,
+ SentryAction::UPDATE,
+ SentryAction::DELETE,
+ SentryAction::ALTER,
+ SentryAction::CREATE,
+ SentryAction::DROP,
+ SentryAction::OWNER,
+});
+
+} // anonymous namespace
+
+namespace {
+
// Indicates different invalid privilege response types to be injected.
enum class InvalidPrivilege {
// No error is injected.
@@ -233,30 +264,15 @@ enum class InvalidPrivilege {
FLIPPED_FIELD,
};
-const SentryActionsSet kAllActions({
- SentryAction::ALL,
- SentryAction::METADATA,
- SentryAction::SELECT,
- SentryAction::INSERT,
- SentryAction::UPDATE,
- SentryAction::DELETE,
- SentryAction::ALTER,
- SentryAction::CREATE,
- SentryAction::DROP,
- SentryAction::OWNER,
-});
-
constexpr const char* kDb = "db";
constexpr const char* kTable = "table";
constexpr const char* kColumn = "column";
} // anonymous namespace
-class SentryAuthzProviderFilterResponsesTest :
- public SentryAuthzProviderTest,
- public ::testing::WithParamInterface<SentryAuthorizableScope::Scope> {
+class SentryAuthzProviderFilterPrivilegesTest : public SentryAuthzProviderTest {
public:
- SentryAuthzProviderFilterResponsesTest()
+ SentryAuthzProviderFilterPrivilegesTest()
: prng_(SeedRandom()) {}
void SetUp() override {
@@ -277,7 +293,7 @@ class SentryAuthzProviderFilterResponsesTest :
const SentryAuthorizableScope& scope, const SentryAction& action,
InvalidPrivilege invalid_privilege = InvalidPrivilege::NONE) {
DCHECK(!full_authorizable.server.empty() && !full_authorizable.db.empty() &&
- !full_authorizable.table.empty() && !full_authorizable.column.empty());
+ !full_authorizable.table.empty() && !full_authorizable.column.empty());
TSentryPrivilege privilege;
privilege.__set_action(invalid_privilege == InvalidPrivilege::INCORRECT_ACTION ?
"foobar" : ActionToString(action.action()));
@@ -342,10 +358,117 @@ class SentryAuthzProviderFilterResponsesTest :
mutable Random prng_;
};
+TEST_F(SentryAuthzProviderFilterPrivilegesTest, TestTablePrivilegePBParsing) {
+ constexpr int kNumColumns = 10;
+ SchemaBuilder schema_builder;
+ schema_builder.AddKeyColumn("col0", DataType::INT32);
+ vector<string> column_names = { "col0" };
+ for (int i = 1; i < kNumColumns; i++) {
+ const string col = Substitute("col$0", i);
+ schema_builder.AddColumn(ColumnSchema(col, DataType::INT32),
+ /*is_key=*/false);
+ column_names.emplace_back(col);
+ }
+ SchemaPB schema_pb;
+ ASSERT_OK(SchemaToPB(schema_builder.Build(), &schema_pb));
+ unordered_map<string, ColumnId> col_name_to_id;
+ for (const auto& col_pb : schema_pb.columns()) {
+ EmplaceOrDie(&col_name_to_id, col_pb.name(), ColumnId(col_pb.id()));
+ }
+
+ // First, grant some privileges at the table authorizable scope or higher.
+ Random prng(SeedRandom());
+ vector<SentryAuthorizableScope::Scope> scope_to_grant_that_implies_table =
+ SelectRandomSubset<vector<SentryAuthorizableScope::Scope>,
+ SentryAuthorizableScope::Scope, Random>({ SentryAuthorizableScope::SERVER,
+ SentryAuthorizableScope::DATABASE,
+ SentryAuthorizableScope::TABLE }, 0, &prng);
+ unordered_map<SentryAuthorizableScope::Scope, SentryActionsSet, std::hash<int>>
+ granted_privileges;
+ SentryActionsSet table_privileges;
+ TSentryAuthorizable table_authorizable;
+ table_authorizable.__set_server(FLAGS_server_name);
+ table_authorizable.__set_db(kDb);
+ table_authorizable.__set_table(kTable);
+ table_authorizable.__set_column(column_names[0]);
+ for (const auto& granted_scope : scope_to_grant_that_implies_table) {
+ for (const auto& action : SelectRandomSubset<SentryActionsSet, SentryAction::Action, Random>(
+ kAllActions, 0, &prng)) {
+ // Grant the privilege to the user.
+ TSentryPrivilege table_privilege = CreatePrivilege(table_authorizable,
+ SentryAuthorizableScope(granted_scope), SentryAction(action));
+ ASSERT_OK(AlterRoleGrantPrivilege(sentry_client_.get(), kRoleName, table_privilege));
+
+ // All of the privileges imply the table-level action.
+ InsertIfNotPresent(&table_privileges, action);
+ }
+ }
+
+ // Grant some privileges at the column scope.
+ vector<string> columns_to_grant =
+ SelectRandomSubset<vector<string>, string, Random>(column_names, 0, &prng);
+ unordered_set<ColumnId> scannable_columns;
+ for (const auto& column_name : columns_to_grant) {
+ for (const auto& action : SelectRandomSubset<SentryActionsSet, SentryAction::Action, Random>(
+ kAllActions, 0, &prng)) {
+ // Grant the privilege to the user.
+ TSentryPrivilege column_privilege =
+ GetColumnPrivilege(kDb, kTable, column_name, ActionToString(action));
+ ASSERT_OK(AlterRoleGrantPrivilege(sentry_client_.get(), kRoleName, column_privilege));
+
+ if (SentryAction(action).Implies(SentryAction(SentryAction::SELECT))) {
+ InsertIfNotPresent(&scannable_columns, FindOrDie(col_name_to_id, column_name));
+ }
+ }
+ }
+
+ // Make sure that any implied privileges make their way to the token.
+ const string kTableId = "table-id";
+ TablePrivilegePB expected_pb;
+ expected_pb.set_table_id(kTableId);
+ for (const auto& granted_table_action : table_privileges) {
+ if (SentryAction(granted_table_action).Implies(SentryAction(SentryAction::INSERT))) {
+ expected_pb.set_insert_privilege(true);
+ }
+ if (SentryAction(granted_table_action).Implies(SentryAction(SentryAction::UPDATE))) {
+ expected_pb.set_update_privilege(true);
+ }
+ if (SentryAction(granted_table_action).Implies(SentryAction(SentryAction::DELETE))) {
+ expected_pb.set_delete_privilege(true);
+ }
+ if (SentryAction(granted_table_action).Implies(SentryAction(SentryAction::SELECT))) {
+ expected_pb.set_scan_privilege(true);
+ }
+ }
+
+ // If any of the table-level privileges imply privileges on scan, we
+ // shouldn't expect per-column scan privileges. Otherwise, we should expect
+ // the columns privileges that implied SELECT to have scan privileges.
+ if (!expected_pb.scan_privilege()) {
+ ColumnPrivilegePB scan_col_privilege;
+ scan_col_privilege.set_scan_privilege(true);
+ for (const auto& id : scannable_columns) {
+ InsertIfNotPresent(expected_pb.mutable_column_privileges(), id, scan_col_privilege);
+ }
+ }
+ // Validate the privileges went through.
+ TablePrivilegePB privilege_pb;
+ privilege_pb.set_table_id(kTableId);
+ ASSERT_OK(sentry_authz_provider_->FillTablePrivilegePB(Substitute("$0.$1", kDb, kTable),
+ kTestUser, schema_pb, &privilege_pb));
+ ASSERT_TRUE(MessageDifferencer::Equals(expected_pb, privilege_pb))
+ << Substitute("$0 vs $1", SecureDebugString(expected_pb), SecureDebugString(privilege_pb));
+}
+
+// Parameterized on the scope at which the privilege will be granted.
+class SentryAuthzProviderFilterPrivilegesScopeTest :
+ public SentryAuthzProviderFilterPrivilegesTest,
+ public ::testing::WithParamInterface<SentryAuthorizableScope::Scope> {};
+
// Attempst to grant privileges for various actions on a single scope of an
// authorizable, injecting various invalid privileges, and checking that Kudu
// ignores them.
-TEST_P(SentryAuthzProviderFilterResponsesTest, TestFilterInvalidResponses) {
+TEST_P(SentryAuthzProviderFilterPrivilegesScopeTest, TestFilterInvalidResponses) {
const string& table_ident = Substitute("$0.$1", full_authorizable_.db, full_authorizable_.table);
static constexpr InvalidPrivilege kInvalidPrivileges[] = {
InvalidPrivilege::INCORRECT_ACTION,
@@ -373,7 +496,7 @@ TEST_P(SentryAuthzProviderFilterResponsesTest, TestFilterInvalidResponses) {
}
// Grants privileges for various actions on a single scope of an authorizable.
-TEST_P(SentryAuthzProviderFilterResponsesTest, TestFilterValidResponses) {
+TEST_P(SentryAuthzProviderFilterPrivilegesScopeTest, TestFilterValidResponses) {
const string& table_ident = Substitute("$0.$1", full_authorizable_.db, full_authorizable_.table);
SentryAuthorizableScope granted_scope(GetParam());
// Send valid requests and verify that we can get it back through the
@@ -397,7 +520,7 @@ TEST_P(SentryAuthzProviderFilterResponsesTest, TestFilterValidResponses) {
}
}
-INSTANTIATE_TEST_CASE_P(GrantedScopes, SentryAuthzProviderFilterResponsesTest,
+INSTANTIATE_TEST_CASE_P(GrantedScopes, SentryAuthzProviderFilterPrivilegesScopeTest,
::testing::Values(SentryAuthorizableScope::SERVER,
SentryAuthorizableScope::DATABASE,
SentryAuthorizableScope::TABLE,
diff --git a/src/kudu/master/sentry_authz_provider.cc b/src/kudu/master/sentry_authz_provider.cc
index 187661a..dfb6895 100644
--- a/src/kudu/master/sentry_authz_provider.cc
+++ b/src/kudu/master/sentry_authz_provider.cc
@@ -21,6 +21,7 @@
#include <ostream>
#include <type_traits>
#include <unordered_map>
+#include <unordered_set>
#include <utility>
#include <vector>
@@ -28,11 +29,13 @@
#include <gflags/gflags.h>
#include <glog/logging.h>
+#include "kudu/common/common.pb.h"
#include "kudu/common/table_util.h"
#include "kudu/gutil/macros.h"
#include "kudu/gutil/map-util.h"
#include "kudu/gutil/strings/substitute.h"
#include "kudu/master/sentry_client_metrics.h"
+#include "kudu/security/token.pb.h"
#include "kudu/sentry/sentry_action.h"
#include "kudu/sentry/sentry_client.h"
#include "kudu/sentry/sentry_policy_service_types.h"
@@ -50,6 +53,7 @@ using sentry::TSentryGrantOption;
using sentry::TSentryPrivilege;
using std::string;
using std::unordered_map;
+using std::unordered_set;
using std::vector;
DEFINE_string(sentry_service_rpc_addresses, "",
@@ -123,6 +127,8 @@ using strings::Substitute;
namespace kudu {
+using security::ColumnPrivilegePB;
+using security::TablePrivilegePB;
using sentry::SentryAction;
using sentry::SentryAuthorizableScope;
using sentry::AuthorizableScopesSet;
@@ -541,6 +547,77 @@ Status SentryAuthzProvider::GetSentryPrivileges(SentryAuthorizableScope::Scope s
return Status::OK();
}
+Status SentryAuthzProvider::FillTablePrivilegePB(const string& table_name,
+ const string& user,
+ const SchemaPB& schema_pb,
+ TablePrivilegePB* pb) {
+ DCHECK(pb);
+ DCHECK(pb->has_table_id());
+ if (AuthzProvider::IsTrustedUser(user)) {
+ pb->set_delete_privilege(true);
+ pb->set_insert_privilege(true);
+ pb->set_scan_privilege(true);
+ pb->set_update_privilege(true);
+ return Status::OK();
+ }
+ static ColumnPrivilegePB scan_col_privilege;
+ scan_col_privilege.set_scan_privilege(true);
+
+ // Note: it might seem like we could cache these TablePrivilegePBs rather
+ // than parsing them from Sentry privileges every time. This is tricky
+ // because the column-level privileges depend on the input schema, which may
+ // be different upon subsequent calls to this function.
+ SentryPrivilegesBranch privileges;
+ RETURN_NOT_OK(GetSentryPrivileges(SentryAuthorizableScope::TABLE, table_name,
+ user, &privileges));
+ unordered_set<string> scannable_col_names;
+ static const SentryAuthorizableScope kTableScope(SentryAuthorizableScope::TABLE);
+ for (const auto& privilege : privileges.privileges) {
+ if (SentryAuthorizableScope(privilege.scope).Implies(kTableScope)) {
+ // Pull out any privileges at the table scope or higher.
+ if (ContainsKey(privilege.granted_privileges, SentryAction::ALL) ||
+ ContainsKey(privilege.granted_privileges, SentryAction::OWNER)) {
+ // Generate privilege with everything.
+ pb->set_delete_privilege(true);
+ pb->set_insert_privilege(true);
+ pb->set_scan_privilege(true);
+ pb->set_update_privilege(true);
+ return Status::OK();
+ }
+ if (ContainsKey(privilege.granted_privileges, SentryAction::DELETE)) {
+ pb->set_delete_privilege(true);
+ }
+ if (ContainsKey(privilege.granted_privileges, SentryAction::INSERT)) {
+ pb->set_insert_privilege(true);
+ }
+ if (ContainsKey(privilege.granted_privileges, SentryAction::SELECT)) {
+ pb->set_scan_privilege(true);
+ }
+ if (ContainsKey(privilege.granted_privileges, SentryAction::UPDATE)) {
+ pb->set_update_privilege(true);
+ }
+ } else if (!pb->scan_privilege() &&
+ (ContainsKey(privilege.granted_privileges, SentryAction::ALL) ||
+ ContainsKey(privilege.granted_privileges, SentryAction::OWNER) ||
+ ContainsKey(privilege.granted_privileges, SentryAction::SELECT))) {
+ // Pull out any scan privileges at the column scope.
+ DCHECK_EQ(SentryAuthorizableScope::COLUMN, privilege.scope);
+ DCHECK(!privilege.column_name.empty());
+ EmplaceIfNotPresent(&scannable_col_names, privilege.column_name);
+ }
+ }
+ // If we got any column-level scan privileges and we don't already have
+ // table-level scan privileges, set them now.
+ if (!pb->scan_privilege()) {
+ for (const auto& col : schema_pb.columns()) {
+ if (ContainsKey(scannable_col_names, col.name())) {
+ InsertIfNotPresent(pb->mutable_column_privileges(), col.id(), scan_col_privilege);
+ }
+ }
+ }
+ return Status::OK();
+}
+
Status SentryAuthzProvider::Authorize(SentryAuthorizableScope::Scope scope,
SentryAction::Action action,
const string& table_ident,
diff --git a/src/kudu/master/sentry_authz_provider.h b/src/kudu/master/sentry_authz_provider.h
index 4be72b3..1a1efa2 100644
--- a/src/kudu/master/sentry_authz_provider.h
+++ b/src/kudu/master/sentry_authz_provider.h
@@ -43,6 +43,12 @@ class TSentryPrivilege;
namespace kudu {
+class SchemaPB;
+
+namespace security {
+class TablePrivilegePB;
+} // namespace security
+
namespace master {
// Utility struct to facilitate evaluating the privileges of a given
@@ -150,12 +156,17 @@ class SentryAuthzProvider : public AuthzProvider {
Status AuthorizeGetTableMetadata(const std::string& table_name,
const std::string& user) override WARN_UNUSED_RESULT;
+ Status FillTablePrivilegePB(const std::string& table_name,
+ const std::string& user,
+ const SchemaPB& schema_pb,
+ security::TablePrivilegePB* pb) override WARN_UNUSED_RESULT;
+
private:
- friend class SentryAuthzProviderFilterResponsesTest;
+ friend class SentryAuthzProviderFilterPrivilegesTest;
FRIEND_TEST(SentryAuthzProviderStaticTest, TestPrivilegesWellFormed);
FRIEND_TEST(TestAuthzHierarchy, TestAuthorizableScope);
- FRIEND_TEST(SentryAuthzProviderFilterResponsesTest, TestFilterInvalidResponses);
- FRIEND_TEST(SentryAuthzProviderFilterResponsesTest, TestFilterValidResponses);
+ FRIEND_TEST(SentryAuthzProviderFilterPrivilegesScopeTest, TestFilterInvalidResponses);
+ FRIEND_TEST(SentryAuthzProviderFilterPrivilegesScopeTest, TestFilterValidResponses);
// Utility function to determine whether the given privilege is a well-formed
// possibly Kudu-related privilege describing a descendent or ancestor of the
[kudu] 04/04: tool: perf tablet_scan action
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 36146d56c302374f75d03b22a5076def76b24199
Author: Adar Dembo <ad...@cloudera.com>
AuthorDate: Fri Apr 5 10:35:34 2019 -0700
tool: perf tablet_scan action
I found this action useful for benchmarking raw tserver scan performance
(i.e. by removing the client from the picture). Not sure it's generally
useful, but if it is, burying it in the 'perf' mode makes sense.
Change-Id: Id4667dcb2d9a21d77149ebdae2e29b7fce8b460b
Reviewed-on: http://gerrit.cloudera.org:8080/12945
Tested-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
src/kudu/tools/kudu-tool-test.cc | 25 +++++++-
src/kudu/tools/tool_action_perf.cc | 128 ++++++++++++++++++++++++++++++++++---
2 files changed, 144 insertions(+), 9 deletions(-)
diff --git a/src/kudu/tools/kudu-tool-test.cc b/src/kudu/tools/kudu-tool-test.cc
index 7a3760c..50ff0d6 100644
--- a/src/kudu/tools/kudu-tool-test.cc
+++ b/src/kudu/tools/kudu-tool-test.cc
@@ -1027,7 +1027,8 @@ TEST_F(ToolTest, TestModeHelp) {
{
const vector<string> kPerfRegexes = {
"loadgen.*Run load generation with optional scan afterwards",
- "table_scan.*Show row count and scanning time cost of tablets in a table"
+ "table_scan.*Show row count and scanning time cost of tablets in a table",
+ "tablet_scan.*Show row count of a local tablet"
};
NO_FATALS(RunTestHelp("perf", kPerfRegexes));
}
@@ -2204,6 +2205,28 @@ TEST_F(ToolTest, TestPerfTableScan) {
NO_FATALS(RunScanTableCheck(kTableName, "", 1, 2000, {}, "perf table_scan"));
}
+TEST_F(ToolTest, TestPerfTabletScan) {
+ // Create a table.
+ const string& kTableName = "perf.tablet_scan";
+ NO_FATALS(RunLoadgen(1, { "--keep_auto_table=true" }, kTableName));
+
+ // Get the list of tablets.
+ vector<string> tablet_ids;
+ TServerDetails* ts = ts_map_[cluster_->tablet_server(0)->uuid()];
+ ASSERT_OK(ListRunningTabletIds(ts, MonoDelta::FromSeconds(30), &tablet_ids));
+
+ // Scan the tablets using the local tool.
+ cluster_->Shutdown();
+ for (const string& tid : tablet_ids) {
+ const string args =
+ Substitute("perf tablet_scan $0 --fs_wal_dir=$1 --fs_data_dirs=$2 --num_iters=2",
+ tid, cluster_->tablet_server(0)->wal_dir(),
+ JoinStrings(cluster_->tablet_server(0)->data_dirs(), ","));
+ NO_FATALS(RunActionStdoutNone(args));
+ NO_FATALS(RunActionStdoutNone(args + " --ordered_scan"));
+ }
+}
+
// Test 'kudu remote_replica copy' tool when the destination tablet server is online.
// 1. Test the copy tool when the destination replica is healthy
// 2. Test the copy tool when the destination replica is tombstoned
diff --git a/src/kudu/tools/tool_action_perf.cc b/src/kudu/tools/tool_action_perf.cc
index 85f96cf..f980d86 100644
--- a/src/kudu/tools/tool_action_perf.cc
+++ b/src/kudu/tools/tool_action_perf.cc
@@ -186,20 +186,41 @@
#include "kudu/client/schema.h"
#include "kudu/client/shared_ptr.h"
#include "kudu/client/write_op.h"
+#include "kudu/clock/clock.h"
+#include "kudu/clock/logical_clock.h"
#include "kudu/common/common.pb.h"
+#include "kudu/common/iterator.h"
#include "kudu/common/partial_row.h"
+#include "kudu/common/rowblock.h"
#include "kudu/common/schema.h"
+#include "kudu/common/timestamp.h"
#include "kudu/common/types.h"
+#include "kudu/consensus/consensus_meta.h"
+#include "kudu/consensus/consensus_meta_manager.h"
+#include "kudu/consensus/log.h"
+#include "kudu/consensus/log_anchor_registry.h"
+#include "kudu/consensus/raft_consensus.h"
+#include "kudu/fs/fs_manager.h"
#include "kudu/gutil/map-util.h"
+#include "kudu/gutil/ref_counted.h"
#include "kudu/gutil/stl_util.h"
#include "kudu/gutil/strings/strcat.h"
#include "kudu/gutil/strings/substitute.h"
+#include "kudu/rpc/result_tracker.h"
+#include "kudu/tablet/rowset.h"
+#include "kudu/tablet/tablet.h"
+#include "kudu/tablet/tablet_bootstrap.h"
+#include "kudu/tablet/tablet_metadata.h"
+#include "kudu/tablet/tablet_replica.h"
#include "kudu/tools/table_scanner.h"
#include "kudu/tools/tool_action.h"
#include "kudu/tools/tool_action_common.h"
#include "kudu/util/decimal_util.h"
+#include "kudu/util/env.h"
#include "kudu/util/flag_validators.h"
#include "kudu/util/int128.h"
+#include "kudu/util/logging.h"
+#include "kudu/util/memory/arena.h"
#include "kudu/util/oid_generator.h"
#include "kudu/util/random.h"
#include "kudu/util/status.h"
@@ -219,7 +240,16 @@ using kudu::client::KuduSchemaBuilder;
using kudu::client::KuduSession;
using kudu::client::KuduTable;
using kudu::client::KuduTableCreator;
-using kudu::client::sp::shared_ptr;
+using kudu::clock::Clock;
+using kudu::clock::LogicalClock;
+using kudu::consensus::ConsensusBootstrapInfo;
+using kudu::consensus::ConsensusMetadata;
+using kudu::consensus::ConsensusMetadataManager;
+using kudu::log::Log;
+using kudu::log::LogAnchorRegistry;
+using kudu::tablet::RowIteratorOptions;
+using kudu::tablet::Tablet;
+using kudu::tablet::TabletMetadata;
using std::accumulate;
using std::cerr;
using std::cout;
@@ -228,6 +258,7 @@ using std::lock_guard;
using std::mutex;
using std::numeric_limits;
using std::ostringstream;
+using std::shared_ptr;
using std::string;
using std::thread;
using std::unique_ptr;
@@ -257,11 +288,15 @@ DEFINE_bool(keep_auto_table, false,
"has no effect if using already existing table "
"(see the '--table_name' flag): neither the existing table "
"nor its data is ever dropped/deleted.");
+DEFINE_int32(num_iters, 1,
+ "Number of times to run the scan.");
DEFINE_uint64(num_rows_per_thread, 1000,
"Number of rows each thread generates and inserts; "
"0 means unlimited. All rows generated by a thread are inserted "
"in the context of the same session.");
DECLARE_int32(num_threads);
+DEFINE_bool(ordered_scan, false,
+ "Whether to run an ordered or unordered scan.");
DEFINE_bool(run_scan, false,
"Whether to run post-insertion scan to verify that the count of "
"the inserted rows matches the expected number. If enabled, "
@@ -469,14 +504,14 @@ Status GenerateRowData(Generator* gen, KuduPartialRow* row,
mutex cerr_lock;
void GeneratorThread(
- const shared_ptr<KuduClient>& client, const string& table_name,
+ const client::sp::shared_ptr<KuduClient>& client, const string& table_name,
size_t gen_idx, Status* status, uint64_t* row_count, uint64_t* err_count) {
const Generator::Mode gen_mode = FLAGS_use_random ? Generator::MODE_RAND
: Generator::MODE_SEQ;
const size_t flush_per_n_rows = FLAGS_flush_per_n_rows;
const uint64_t gen_seq_start = FLAGS_seq_start;
- shared_ptr<KuduSession> session(client->NewSession());
+ client::sp::shared_ptr<KuduSession> session(client->NewSession());
uint64_t idx = 0;
auto generator = [&]() -> Status {
@@ -491,7 +526,7 @@ void GeneratorThread(
: KuduSession::MANUAL_FLUSH));
const size_t num_rows_per_gen = FLAGS_num_rows_per_thread;
- shared_ptr<KuduTable> table;
+ client::sp::shared_ptr<KuduTable> table;
RETURN_NOT_OK(client->OpenTable(table_name, &table));
const size_t num_columns = table->schema().num_columns();
@@ -536,7 +571,7 @@ void GeneratorThread(
}
}
-Status GenerateInsertRows(const shared_ptr<KuduClient>& client,
+Status GenerateInsertRows(const client::sp::shared_ptr<KuduClient>& client,
const string& table_name,
uint64_t* total_row_count,
uint64_t* total_err_count) {
@@ -570,7 +605,7 @@ Status GenerateInsertRows(const shared_ptr<KuduClient>& client,
// Fetch all rows from the table with the specified name; iterate over them
// and output their total count.
-Status CountTableRows(const shared_ptr<KuduClient>& client,
+Status CountTableRows(const client::sp::shared_ptr<KuduClient>& client,
const string& table_name, uint64_t* count) {
TableScanner scanner(client, table_name);
scanner.SetReadMode(KuduScanner::ReadMode::READ_YOUR_WRITES);
@@ -583,7 +618,7 @@ Status CountTableRows(const shared_ptr<KuduClient>& client,
}
Status TestLoadGenerator(const RunnerContext& context) {
- shared_ptr<KuduClient> client;
+ client::sp::shared_ptr<KuduClient> client;
RETURN_NOT_OK(CreateKuduClient(context, &client));
string table_name;
@@ -686,7 +721,7 @@ Status TestLoadGenerator(const RunnerContext& context) {
return Status::OK();
}
-Status TableScan(const RunnerContext &context) {
+Status TableScan(const RunnerContext& context) {
client::sp::shared_ptr<KuduClient> client;
RETURN_NOT_OK(CreateKuduClient(context, &client));
@@ -698,6 +733,72 @@ Status TableScan(const RunnerContext &context) {
return scanner.StartScan();
}
+Status TabletScan(const RunnerContext& context) {
+ const string& tablet_id = FindOrDie(context.required_args, kTabletIdArg);
+
+ // Initialize just enough of a tserver to bootstrap the tablet. We must
+ // bootstrap so that our scan includes data from the WAL segments.
+ //
+ // Note: we need a read-write FsManager because bootstrapping will do
+ // destructive things (e.g. rename the tablet's WAL segment directory).
+ FsManager fs(Env::Default(), FsManagerOpts());
+ RETURN_NOT_OK(fs.Open());
+
+ scoped_refptr<TabletMetadata> tmeta;
+ RETURN_NOT_OK(TabletMetadata::Load(&fs, tablet_id, &tmeta));
+
+ scoped_refptr<ConsensusMetadataManager> cmeta_manager(
+ new ConsensusMetadataManager(&fs));
+ scoped_refptr<ConsensusMetadata> cmeta;
+ RETURN_NOT_OK(cmeta_manager->Load(tablet_id, &cmeta));
+
+ scoped_refptr<Clock> clock(
+ LogicalClock::CreateStartingAt(Timestamp::kInitialTimestamp));
+ RETURN_NOT_OK(clock->Init());
+
+ scoped_refptr<LogAnchorRegistry> registry(new LogAnchorRegistry());
+
+ // Bootstrap the tablet.
+ shared_ptr<Tablet> tablet;
+ scoped_refptr<Log> log;
+ ConsensusBootstrapInfo cbi;
+ RETURN_NOT_OK(tablet::BootstrapTablet(std::move(tmeta),
+ cmeta->CommittedConfig(),
+ std::move(clock),
+ /*mem_tracker=*/ nullptr,
+ /*result_tracker=*/ nullptr,
+ /*metric_registry=*/ nullptr,
+ /*tablet_replica=*/ nullptr,
+ &tablet,
+ &log,
+ std::move(registry),
+ &cbi));
+
+ // Tablet has been bootstrapped and opened. We can now scan it.
+ for (int i = 0; i < FLAGS_num_iters; i++) {
+ LOG_TIMING(INFO, Substitute("scanning tablet (iter $0)", i)) {
+ Schema projection = tablet->schema()->CopyWithoutColumnIds();
+ RowIteratorOptions opts;
+ opts.projection = &projection;
+ opts.order = FLAGS_ordered_scan ? ORDERED : UNORDERED;
+ unique_ptr<RowwiseIterator> iter;
+ RETURN_NOT_OK(tablet->NewRowIterator(std::move(opts), &iter));
+ RETURN_NOT_OK(iter->Init(nullptr));
+ Arena arena(1024);
+ RowBlock block(&projection, 100, &arena);
+ int64_t rows_scanned = 0;
+ while (iter->HasNext()) {
+ arena.Reset();
+ RETURN_NOT_OK(iter->NextBlock(&block));
+ rows_scanned += block.nrows();
+ KLOG_EVERY_N_SECS(INFO, 10) << "scanned " << rows_scanned << " rows";
+ }
+ LOG(INFO) << "scanned " << rows_scanned << " rows";
+ }
+ }
+ return Status::OK();
+}
+
} // anonymous namespace
unique_ptr<Mode> BuildPerfMode() {
@@ -761,10 +862,21 @@ unique_ptr<Mode> BuildPerfMode() {
.AddOptionalParameter("tablets")
.Build();
+ unique_ptr<Action> tablet_scan =
+ ActionBuilder("tablet_scan", &TabletScan)
+ .Description("Show row count of a local tablet")
+ .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
+ .AddOptionalParameter("fs_data_dirs")
+ .AddOptionalParameter("fs_metadata_dir")
+ .AddOptionalParameter("fs_wal_dir")
+ .AddOptionalParameter("num_iters")
+ .AddOptionalParameter("ordered_scan")
+ .Build();
return ModeBuilder("perf")
.Description("Measure the performance of a Kudu cluster")
.AddAction(std::move(loadgen))
.AddAction(std::move(table_scan))
+ .AddAction(std::move(tablet_scan))
.Build();
}
[kudu] 03/04: remove schema copies from RowBlock and RowBuilder
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 49d3438f0a5ac21d8875916ad437da71752f8ddd
Author: Adar Dembo <ad...@cloudera.com>
AuthorDate: Mon Apr 1 15:31:20 2019 -0700
remove schema copies from RowBlock and RowBuilder
AFAICT the schemas in these two always outlive the classes themselves, so
there's no reason to make schema copies. This is especially painful for the
MergeIterator where each sub-iterator has a RowBlock (and thus a full copy
of the schema).
Change-Id: Ie4d60640b2bac4f2b3077ddd696966e1f6658740
Reviewed-on: http://gerrit.cloudera.org:8080/12907
Tested-by: Kudu Jenkins
Reviewed-by: Mike Percy <mp...@apache.org>
---
src/kudu/codegen/codegen-test.cc | 14 ++---
src/kudu/common/generic_iterators-test.cc | 6 +--
src/kudu/common/generic_iterators.cc | 6 +--
src/kudu/common/row.h | 38 ++++++++-----
src/kudu/common/row_changelist-test.cc | 6 +--
src/kudu/common/rowblock.cc | 14 ++---
src/kudu/common/rowblock.h | 62 +++++++++++++---------
src/kudu/common/schema-test.cc | 6 +--
src/kudu/common/wire_protocol-test.cc | 8 +--
src/kudu/common/wire_protocol.cc | 6 +--
src/kudu/integration-tests/linked_list-test-util.h | 2 +-
src/kudu/master/sys_catalog.cc | 2 +-
src/kudu/tablet/cfile_set-test.cc | 8 +--
src/kudu/tablet/compaction-test.cc | 14 ++---
src/kudu/tablet/compaction.cc | 8 +--
src/kudu/tablet/delta_compaction.cc | 2 +-
src/kudu/tablet/deltafile-test.cc | 2 +-
src/kudu/tablet/diskrowset-test-base.h | 12 +++--
src/kudu/tablet/diskrowset-test.cc | 28 ++++++----
src/kudu/tablet/diskrowset.cc | 2 +-
src/kudu/tablet/memrowset-test.cc | 16 +++---
src/kudu/tablet/mt-rowset_delta_compaction-test.cc | 2 +-
src/kudu/tablet/mt-tablet-test.cc | 6 +--
src/kudu/tablet/tablet-decoder-eval-test.cc | 21 ++++----
src/kudu/tablet/tablet-pushdown-test.cc | 4 +-
src/kudu/tablet/tablet-test-base.h | 2 +-
src/kudu/tablet/tablet-test-util.h | 8 +--
src/kudu/tablet/tablet-test.cc | 6 +--
src/kudu/tablet/tablet_random_access-test.cc | 2 +-
src/kudu/tserver/tablet_server-test-base.cc | 4 +-
src/kudu/tserver/tablet_service.cc | 4 +-
31 files changed, 178 insertions(+), 143 deletions(-)
diff --git a/src/kudu/codegen/codegen-test.cc b/src/kudu/codegen/codegen-test.cc
index 2aace58..0db310b 100644
--- a/src/kudu/codegen/codegen-test.cc
+++ b/src/kudu/codegen/codegen-test.cc
@@ -86,7 +86,7 @@ class CodegenTest : public KuduTest {
defaults_ = SchemaBuilder(defaults_).Build(); // add IDs
test_rows_arena_.reset(new Arena(2 * 1024));
- RowBuilder rb(base_);
+ RowBuilder rb(&base_);
for (int i = 0; i < kNumTestRows; ++i) {
rb.AddUint64(i);
rb.AddInt32(random_.Next32());
@@ -168,14 +168,14 @@ const Slice kStrWValue = "WWWWW STRING DEFAULT WRITE";
void CheckRowBlocksEqual(const RowBlock* rb1, const RowBlock* rb2,
const string& name1, const string& name2) {
CHECK_EQ(rb1->nrows(), rb2->nrows());
- const Schema& schema = rb1->schema();
+ const Schema* schema = rb1->schema();
for (int i = 0; i < rb1->nrows(); ++i) {
RowBlockRow row1 = rb1->row(i);
RowBlockRow row2 = rb2->row(i);
- CHECK_EQ(schema.Compare(row1, row2), 0)
+ CHECK_EQ(schema->Compare(row1, row2), 0)
<< "Rows unequal (failed at row " << i << "):\n"
- << "\t(" << name1 << ") = " << schema.DebugRow(row1) << "\n"
- << "\t(" << name2 << ") = " << schema.DebugRow(row2);
+ << "\t(" << name1 << ") = " << schema->DebugRow(row1) << "\n"
+ << "\t(" << name2 << ") = " << schema->DebugRow(row2);
}
}
@@ -219,8 +219,8 @@ void CodegenTest::TestProjection(const Schema* proj) {
CHECK_EQ(with->base_schema(), &base_);
CHECK_EQ(with->projection(), proj);
- RowBlock rb_with(*proj, kNumTestRows, &projections_arena_);
- RowBlock rb_without(*proj, kNumTestRows, &projections_arena_);
+ RowBlock rb_with(proj, kNumTestRows, &projections_arena_);
+ RowBlock rb_without(proj, kNumTestRows, &projections_arena_);
projections_arena_.Reset();
ProjectTestRows<READ>(with.get(), &rb_with);
diff --git a/src/kudu/common/generic_iterators-test.cc b/src/kudu/common/generic_iterators-test.cc
index d002e83..10592d2 100644
--- a/src/kudu/common/generic_iterators-test.cc
+++ b/src/kudu/common/generic_iterators-test.cc
@@ -364,7 +364,7 @@ void TestMerge(const Schema& schema, const TestIntRangePredicate &predicate,
std::move(to_merge)));
ASSERT_OK(merger->Init(&spec));
- RowBlock dst(schema, 100, nullptr);
+ RowBlock dst(&schema, 100, nullptr);
size_t total_idx = 0;
auto expected_iter = expected.cbegin();
while (merger->HasNext()) {
@@ -452,7 +452,7 @@ TEST(TestMaterializingIterator, TestMaterializingPredicatePushdown) {
ASSERT_EQ(0, spec.predicates().size()) << "Iterator should have pushed down predicate";
Arena arena(1024);
- RowBlock dst(kIntSchema, 100, &arena);
+ RowBlock dst(&kIntSchema, 100, &arena);
ASSERT_OK(materializing->NextBlock(&dst));
ASSERT_EQ(dst.nrows(), 100);
@@ -499,7 +499,7 @@ TEST(TestPredicateEvaluatingIterator, TestPredicateEvaluation) {
<< "Predicate should be evaluated by the outer iterator";
Arena arena(1024);
- RowBlock dst(kIntSchema, 100, &arena);
+ RowBlock dst(&kIntSchema, 100, &arena);
ASSERT_OK(outer_iter->NextBlock(&dst));
ASSERT_EQ(dst.nrows(), 100);
diff --git a/src/kudu/common/generic_iterators.cc b/src/kudu/common/generic_iterators.cc
index 682c2e1..0230fc4 100644
--- a/src/kudu/common/generic_iterators.cc
+++ b/src/kudu/common/generic_iterators.cc
@@ -103,7 +103,7 @@ class MergeIterState {
explicit MergeIterState(unique_ptr<RowwiseIterator> iter) :
iter_(std::move(iter)),
arena_(1024),
- read_block_(iter_->schema(), kMergeRowBuffer, &arena_),
+ read_block_(&iter_->schema(), kMergeRowBuffer, &arena_),
next_row_idx_(0),
rows_advanced_(0),
rows_valid_(0)
@@ -387,7 +387,7 @@ Status MergeIterator::InitSubIterators(ScanSpec *spec) {
Status MergeIterator::NextBlock(RowBlock* dst) {
CHECK(initted_);
- DCHECK_SCHEMA_EQ(dst->schema(), schema());
+ DCHECK_SCHEMA_EQ(*dst->schema(), schema());
PrepareBatch(dst);
RETURN_NOT_OK(MaterializeBlock(dst));
@@ -1015,7 +1015,7 @@ Status PredicateEvaluatingIterator::NextBlock(RowBlock *dst) {
RETURN_NOT_OK(base_iter_->NextBlock(dst));
for (const auto& predicate : col_predicates_) {
- int32_t col_idx = dst->schema().find_column(predicate.column().name());
+ int32_t col_idx = dst->schema()->find_column(predicate.column().name());
if (col_idx == Schema::kColumnNotFound) {
return Status::InvalidArgument("Unknown column in predicate", predicate.ToString());
}
diff --git a/src/kudu/common/row.h b/src/kudu/common/row.h
index 8015cdd..45090fc 100644
--- a/src/kudu/common/row.h
+++ b/src/kudu/common/row.h
@@ -429,6 +429,9 @@ class ContiguousRowHelper {
template<class ContiguousRowType>
class ContiguousRowCell {
public:
+ // Constructs a new ContiguousRowCell.
+ //
+ // The 'row' object must outlive this ContiguousRowCell.
ContiguousRowCell(const ContiguousRowType* row, int idx)
: row_(row), col_idx_(idx) {
}
@@ -455,6 +458,9 @@ class ContiguousRow {
public:
typedef ContiguousRowCell<ContiguousRow> Cell;
+ // Constructs a new ContiguousRow.
+ //
+ // The 'schema' and 'row_data' objects must outlive this ContiguousRow.
explicit ContiguousRow(const Schema* schema, uint8_t *row_data = NULL)
: schema_(schema), row_data_(row_data) {
}
@@ -504,6 +510,9 @@ class ConstContiguousRow {
public:
typedef ContiguousRowCell<ConstContiguousRow> Cell;
+ // Constructs a new ConstContiguousRow.
+ //
+ // The 'row' object's schema and data must outlive this ConstContiguousRow.
explicit ConstContiguousRow(const ContiguousRow &row)
: schema_(row.schema_),
row_data_(row.row_data_) {
@@ -560,13 +569,16 @@ void ContiguousRowCell<ConstContiguousRow>::set_null(bool null) const;
// Utility class for building rows corresponding to a given schema.
// This is used only by tests.
-// TODO: move it into a test utility.
+// TODO(todd): move it into a test utility.
class RowBuilder {
public:
- explicit RowBuilder(const Schema& schema)
+ // Constructs a new RowBuilder.
+ //
+ // The 'schema' object must outlive this RowBuilder.
+ explicit RowBuilder(const Schema* schema)
: schema_(schema),
arena_(1024),
- bitmap_size_(ContiguousRowHelper::null_bitmap_size(schema)) {
+ bitmap_size_(ContiguousRowHelper::null_bitmap_size(*schema)) {
Reset();
}
@@ -579,12 +591,12 @@ class RowBuilder {
// (eg using CopyRowToArena()).
void Reset() {
arena_.Reset();
- size_t row_size = schema_.byte_size() + bitmap_size_;
+ size_t row_size = schema_->byte_size() + bitmap_size_;
buf_ = reinterpret_cast<uint8_t *>(arena_.AllocateBytes(row_size));
CHECK(buf_) << "could not allocate " << row_size << " bytes for row builder";
col_idx_ = 0;
byte_idx_ = 0;
- ContiguousRowHelper::InitNullsBitmap(schema_, buf_, bitmap_size_);
+ ContiguousRowHelper::InitNullsBitmap(*schema_, buf_, bitmap_size_);
}
void AddString(const Slice &slice) {
@@ -674,8 +686,8 @@ class RowBuilder {
}
void AddNull() {
- CHECK(schema_.column(col_idx_).is_nullable());
- BitmapSet(buf_ + schema_.byte_size(), col_idx_);
+ CHECK(schema_->column(col_idx_).is_nullable());
+ BitmapSet(buf_ + schema_->byte_size(), col_idx_);
Advance();
}
@@ -691,16 +703,16 @@ class RowBuilder {
// data is copied, it is not safe to Reset() before also calling
// CopyRowIndirectDataToArena.
const Slice data() const {
- CHECK_EQ(byte_idx_, schema_.byte_size());
+ CHECK_EQ(byte_idx_, schema_->byte_size());
return Slice(buf_, byte_idx_ + bitmap_size_);
}
- const Schema& schema() const {
+ const Schema* schema() const {
return schema_;
}
ConstContiguousRow row() const {
- return ConstContiguousRow(&schema_, data());
+ return ConstContiguousRow(schema_, data());
}
private:
@@ -724,17 +736,17 @@ class RowBuilder {
}
void CheckNextType(DataType type) {
- CHECK_EQ(schema_.column(col_idx_).type_info()->type(),
+ CHECK_EQ(schema_->column(col_idx_).type_info()->type(),
type);
}
void Advance() {
- int size = schema_.column(col_idx_).type_info()->size();
+ int size = schema_->column(col_idx_).type_info()->size();
byte_idx_ += size;
col_idx_++;
}
- const Schema schema_;
+ const Schema* schema_;
Arena arena_;
uint8_t *buf_;
diff --git a/src/kudu/common/row_changelist-test.cc b/src/kudu/common/row_changelist-test.cc
index 1797c89..c21ff7c 100644
--- a/src/kudu/common/row_changelist-test.cc
+++ b/src/kudu/common/row_changelist-test.cc
@@ -145,7 +145,7 @@ TEST_F(TestRowChangeList, TestReinserts) {
{
// Reinserts include indirect data, so it should be ok to make the RowBuilder a scoped var.
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
rb.AddString(Slice("hello"));
rb.AddString(Slice("world"));
rb.AddUint32(12345);
@@ -168,9 +168,9 @@ TEST_F(TestRowChangeList, TestReinserts) {
faststring buf2;
RowChangeListEncoder reinsert_2_enc(&buf2);
{
- RowBlock block(schema_, 1, nullptr);
+ RowBlock block(&schema_, 1, nullptr);
RowBlockRow dst_row = block.row(0);
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
rb.AddString(Slice("hello"));
rb.AddString(Slice("mundo"));
rb.AddUint32(54321);
diff --git a/src/kudu/common/rowblock.cc b/src/kudu/common/rowblock.cc
index df43eda..32cce4b 100644
--- a/src/kudu/common/rowblock.cc
+++ b/src/kudu/common/rowblock.cc
@@ -110,12 +110,12 @@ bool operator!=(const SelectionVector& a, const SelectionVector& b) {
//////////////////////////////
// RowBlock
//////////////////////////////
-RowBlock::RowBlock(const Schema &schema,
+RowBlock::RowBlock(const Schema* schema,
size_t nrows,
Arena *arena)
: schema_(schema),
- columns_data_(schema.num_columns()),
- column_null_bitmaps_(schema.num_columns()),
+ columns_data_(schema->num_columns()),
+ column_null_bitmaps_(schema->num_columns()),
row_capacity_(nrows),
nrows_(nrows),
arena_(arena),
@@ -123,8 +123,8 @@ RowBlock::RowBlock(const Schema &schema,
CHECK_GT(row_capacity_, 0);
size_t bitmap_size = BitmapSize(row_capacity_);
- for (size_t i = 0; i < schema.num_columns(); ++i) {
- const ColumnSchema& col_schema = schema.column(i);
+ for (size_t i = 0; i < schema->num_columns(); ++i) {
+ const ColumnSchema& col_schema = schema->column(i);
size_t col_size = row_capacity_ * col_schema.type_info()->size();
columns_data_[i] = new uint8_t[col_size];
@@ -135,10 +135,10 @@ RowBlock::RowBlock(const Schema &schema,
}
RowBlock::~RowBlock() {
- for (uint8_t *column_data : columns_data_) {
+ for (uint8_t* column_data : columns_data_) {
delete[] column_data;
}
- for (uint8_t *bitmap_data : column_null_bitmaps_) {
+ for (uint8_t* bitmap_data : column_null_bitmaps_) {
delete[] bitmap_data;
}
}
diff --git a/src/kudu/common/rowblock.h b/src/kudu/common/rowblock.h
index 2afd212..d2cfc82 100644
--- a/src/kudu/common/rowblock.h
+++ b/src/kudu/common/rowblock.h
@@ -149,6 +149,9 @@ bool operator!=(const SelectionVector& a, const SelectionVector& b);
// underlying selection vector can easily be updated batch-by-batch.
class SelectionVectorView {
public:
+ // Constructs a new SelectionVectorView.
+ //
+ // The 'sel_vec' object must outlive this SelectionVectorView.
explicit SelectionVectorView(SelectionVector *sel_vec)
: sel_vec_(sel_vec), row_offset_(0) {
}
@@ -192,9 +195,12 @@ class SelectionVectorView {
// of the latter doesn't mean you _should_.
class RowBlock {
public:
- RowBlock(const Schema &schema,
+ // Constructs a new RowBlock.
+ //
+ // The 'schema' and 'arena' objects must outlive this RowBlock.
+ RowBlock(const Schema* schema,
size_t nrows,
- Arena *arena);
+ Arena* arena);
~RowBlock();
// Resize the block to the given number of rows.
@@ -209,8 +215,8 @@ class RowBlock {
RowBlockRow row(size_t idx) const;
- const Schema &schema() const { return schema_; }
- Arena *arena() const { return arena_; }
+ const Schema* schema() const { return schema_; }
+ Arena* arena() const { return arena_; }
ColumnBlock column_block(size_t col_idx) const {
return column_block(col_idx, nrows_);
@@ -219,9 +225,9 @@ class RowBlock {
ColumnBlock column_block(size_t col_idx, size_t nrows) const {
DCHECK_LE(nrows, nrows_);
- const ColumnSchema& col_schema = schema_.column(col_idx);
- uint8_t *col_data = columns_data_[col_idx];
- uint8_t *nulls_bitmap = column_null_bitmaps_[col_idx];
+ const ColumnSchema& col_schema = schema_->column(col_idx);
+ uint8_t* col_data = columns_data_[col_idx];
+ uint8_t* nulls_bitmap = column_null_bitmaps_[col_idx];
return ColumnBlock(col_schema.type_info(), nulls_bitmap, col_data, nrows, arena_);
}
@@ -245,8 +251,8 @@ class RowBlock {
// from unit tests.
void ZeroMemory() {
size_t bitmap_size = BitmapSize(row_capacity_);
- for (size_t i = 0; i < schema_.num_columns(); ++i) {
- const ColumnSchema& col_schema = schema_.column(i);
+ for (size_t i = 0; i < schema_->num_columns(); ++i) {
+ const ColumnSchema& col_schema = schema_->column(i);
size_t col_size = col_schema.type_info()->size() * row_capacity_;
memset(columns_data_[i], '\0', col_size);
@@ -263,11 +269,11 @@ class RowBlock {
// as predicates or deletions make rows invalid, they are set to 0s.
// After a batch has completed, only those rows with associated true
// bits in the selection vector are valid results for the scan.
- SelectionVector *selection_vector() {
+ SelectionVector* selection_vector() {
return &sel_vec_;
}
- const SelectionVector *selection_vector() const {
+ const SelectionVector* selection_vector() const {
return &sel_vec_;
}
@@ -286,9 +292,9 @@ class RowBlock {
return block_size;
}
- Schema schema_;
- std::vector<uint8_t *> columns_data_;
- std::vector<uint8_t *> column_null_bitmaps_;
+ const Schema* schema_;
+ std::vector<uint8_t*> columns_data_;
+ std::vector<uint8_t*> column_null_bitmaps_;
// The maximum number of rows that can be stored in our allocated buffer.
size_t row_capacity_;
@@ -297,7 +303,7 @@ class RowBlock {
// nrows_ <= row_capacity_
size_t nrows_;
- Arena *arena_;
+ Arena* arena_;
// The bitmap indicating which rows are valid in this block.
// Deleted rows or rows which have failed to pass predicates will be zeroed
@@ -315,11 +321,15 @@ class RowBlockRow {
public:
typedef ColumnBlock::Cell Cell;
- explicit RowBlockRow(const RowBlock *row_block = NULL, size_t row_index = 0)
+ // Constructs a new RowBlockRow.
+ //
+ // The 'row_block' object must outlive this RowBlockRow.
+ explicit RowBlockRow(const RowBlock* row_block = nullptr,
+ size_t row_index = 0)
: row_block_(row_block), row_index_(row_index) {
}
- RowBlockRow *Reset(const RowBlock *row_block, size_t row_index) {
+ RowBlockRow* Reset(const RowBlock* row_block, size_t row_index) {
row_block_ = row_block;
row_index_ = row_index;
return this;
@@ -334,22 +344,22 @@ class RowBlockRow {
}
const Schema* schema() const {
- return &row_block_->schema();
+ return row_block_->schema();
}
bool is_null(size_t col_idx) const {
return column_block(col_idx).is_null(row_index_);
}
- uint8_t *mutable_cell_ptr(size_t col_idx) const {
+ uint8_t* mutable_cell_ptr(size_t col_idx) const {
return const_cast<uint8_t*>(cell_ptr(col_idx));
}
- const uint8_t *cell_ptr(size_t col_idx) const {
+ const uint8_t* cell_ptr(size_t col_idx) const {
return column_block(col_idx).cell_ptr(row_index_);
}
- const uint8_t *nullable_cell_ptr(size_t col_idx) const {
+ const uint8_t* nullable_cell_ptr(size_t col_idx) const {
return column_block(col_idx).nullable_cell_ptr(row_index_);
}
@@ -363,23 +373,23 @@ class RowBlockRow {
// Mark this row as unselected in the selection vector.
void SetRowUnselected() {
- // TODO: const-ness issues since this class holds a const RowBlock *.
+ // TODO(todd): const-ness issues since this class holds a const RowBlock*.
// hack around this for now
- SelectionVector *vec = const_cast<SelectionVector *>(row_block_->selection_vector());
+ SelectionVector* vec = const_cast<SelectionVector*>(row_block_->selection_vector());
vec->SetRowUnselected(row_index_);
}
#ifndef NDEBUG
void OverwriteWithPattern(StringPiece pattern) {
- const Schema& schema = row_block_->schema();
- for (size_t col = 0; col < schema.num_columns(); col++) {
+ const Schema* schema = row_block_->schema();
+ for (size_t col = 0; col < schema->num_columns(); col++) {
row_block_->column_block(col).OverwriteWithPattern(row_index_, pattern);
}
}
#endif
private:
- const RowBlock *row_block_;
+ const RowBlock* row_block_;
size_t row_index_;
};
diff --git a/src/kudu/common/schema-test.cc b/src/kudu/common/schema-test.cc
index 1c5e8c6..0d26bbb 100644
--- a/src/kudu/common/schema-test.cc
+++ b/src/kudu/common/schema-test.cc
@@ -504,7 +504,7 @@ TEST_F(TestSchema, TestRowOperations) {
Arena arena(1024);
- RowBuilder rb(schema);
+ RowBuilder rb(&schema);
rb.AddString(string("row_a_1"));
rb.AddString(string("row_a_2"));
rb.AddUint32(3);
@@ -662,9 +662,9 @@ TEST(TestKeyEncoder, BenchmarkSimpleKey) {
faststring fs;
Schema schema({ ColumnSchema("col1", STRING) }, 1);
- RowBuilder rb(schema);
+ RowBuilder rb(&schema);
rb.AddString(Slice("hello world"));
- ConstContiguousRow row(&rb.schema(), rb.data());
+ ConstContiguousRow row(rb.schema(), rb.data());
LOG_TIMING(INFO, "Encoding") {
for (int i = 0; i < 10000000; i++) {
diff --git a/src/kudu/common/wire_protocol-test.cc b/src/kudu/common/wire_protocol-test.cc
index 35aa7ea..4885b96 100644
--- a/src/kudu/common/wire_protocol-test.cc
+++ b/src/kudu/common/wire_protocol-test.cc
@@ -221,7 +221,7 @@ TEST_F(WireProtocolTest, TestBadSchema_DuplicateColumnName) {
// converted to and from protobuf.
TEST_F(WireProtocolTest, TestColumnarRowBlockToPB) {
Arena arena(1024);
- RowBlock block(schema_, 10, &arena);
+ RowBlock block(&schema_, 10, &arena);
FillRowBlockWithTestRows(&block);
// Convert to PB.
@@ -258,7 +258,7 @@ TEST_F(WireProtocolTest, TestColumnarRowBlockToPBWithPadding) {
ColumnSchema("col2", UNIXTIME_MICROS),
ColumnSchema("col3", INT32, true /* nullable */),
ColumnSchema("col4", UNIXTIME_MICROS, true /* nullable */)}, 1);
- RowBlock block(tablet_schema, kNumRows, &arena);
+ RowBlock block(&tablet_schema, kNumRows, &arena);
block.selection_vector()->SetAllTrue();
for (int i = 0; i < block.nrows(); i++) {
@@ -342,7 +342,7 @@ TEST_F(WireProtocolTest, TestColumnarRowBlockToPBWithPadding) {
TEST_F(WireProtocolTest, TestColumnarRowBlockToPBBenchmark) {
Arena arena(1024);
const int kNumTrials = AllowSlowTests() ? 100 : 10;
- RowBlock block(schema_, 10000 * kNumTrials, &arena);
+ RowBlock block(&schema_, 10000 * kNumTrials, &arena);
FillRowBlockWithTestRows(&block);
RowwiseRowBlockPB pb;
@@ -386,7 +386,7 @@ TEST_F(WireProtocolTest, TestInvalidRowBlock) {
TEST_F(WireProtocolTest, TestBlockWithNoColumns) {
Schema empty(std::vector<ColumnSchema>(), 0);
Arena arena(1024);
- RowBlock block(empty, 1000, &arena);
+ RowBlock block(&empty, 1000, &arena);
block.selection_vector()->SetAllTrue();
// Unselect 100 rows
for (int i = 0; i < 100; i++) {
diff --git a/src/kudu/common/wire_protocol.cc b/src/kudu/common/wire_protocol.cc
index ce2cf0b..e7781b7 100644
--- a/src/kudu/common/wire_protocol.cc
+++ b/src/kudu/common/wire_protocol.cc
@@ -891,10 +891,10 @@ void SerializeRowBlock(const RowBlock& block,
faststring* indirect_data,
bool pad_unixtime_micros_to_16_bytes) {
DCHECK_GT(block.nrows(), 0);
- const Schema& tablet_schema = block.schema();
+ const Schema* tablet_schema = block.schema();
if (projection_schema == nullptr) {
- projection_schema = &tablet_schema;
+ projection_schema = tablet_schema;
}
// Check whether we need to pad or if there are nullable columns, this will dictate whether
@@ -932,7 +932,7 @@ void SerializeRowBlock(const RowBlock& block,
size_t padding_so_far = 0;
for (int p_schema_idx = 0; p_schema_idx < projection_schema->num_columns(); p_schema_idx++) {
const ColumnSchema& col = projection_schema->column(p_schema_idx);
- t_schema_idx = tablet_schema.find_column(col.name());
+ t_schema_idx = tablet_schema->find_column(col.name());
DCHECK_NE(t_schema_idx, -1);
size_t column_offset = projection_schema->column_offset(p_schema_idx) + padding_so_far;
diff --git a/src/kudu/integration-tests/linked_list-test-util.h b/src/kudu/integration-tests/linked_list-test-util.h
index e51fc27..46a509f 100644
--- a/src/kudu/integration-tests/linked_list-test-util.h
+++ b/src/kudu/integration-tests/linked_list-test-util.h
@@ -685,7 +685,7 @@ Status LinkedListTester::VerifyLinkedListLocal(const tablet::Tablet* tablet,
RETURN_NOT_OK_PREPEND(iter->Init(nullptr), "Cannot initialize row iterator");
Arena arena(1024);
- RowBlock block(projection, 100, &arena);
+ RowBlock block(&projection, 100, &arena);
while (iter->HasNext()) {
RETURN_NOT_OK(iter->NextBlock(&block));
for (int i = 0; i < block.nrows(); i++) {
diff --git a/src/kudu/master/sys_catalog.cc b/src/kudu/master/sys_catalog.cc
index c9bc5ad..4918f11 100644
--- a/src/kudu/master/sys_catalog.cc
+++ b/src/kudu/master/sys_catalog.cc
@@ -627,7 +627,7 @@ Status SysCatalogTable::ProcessRows(
RETURN_NOT_OK(iter->Init(&spec));
Arena arena(32 * 1024);
- RowBlock block(iter->schema(), 512, &arena);
+ RowBlock block(&iter->schema(), 512, &arena);
while (iter->HasNext()) {
RETURN_NOT_OK(iter->NextBlock(&block));
const size_t nrows = block.nrows();
diff --git a/src/kudu/tablet/cfile_set-test.cc b/src/kudu/tablet/cfile_set-test.cc
index 20b9f22..33f21d3 100644
--- a/src/kudu/tablet/cfile_set-test.cc
+++ b/src/kudu/tablet/cfile_set-test.cc
@@ -94,7 +94,7 @@ class TestCFileSet : public KuduRowSetTest {
ASSERT_OK(rsw.Open());
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
for (int i = 0; i < nrows; i++) {
rb.Reset();
rb.AddInt32(i * 2);
@@ -193,7 +193,7 @@ class TestCFileSet : public KuduRowSetTest {
// Check that the range was respected on all the results.
Arena arena(1024);
- RowBlock block(schema_, 100, &arena);
+ RowBlock block(&schema_, 100, &arena);
while (iter->HasNext()) {
ASSERT_OK_FAST(iter->NextBlock(&block));
for (size_t i = 0; i < block.nrows(); i++) {
@@ -227,7 +227,7 @@ class TestCFileSet : public KuduRowSetTest {
ASSERT_OK(iter->Init(&spec));
// Check that the range was respected on all the results.
Arena arena(1024);
- RowBlock block(schema_, 100, &arena);
+ RowBlock block(&schema_, 100, &arena);
while (iter->HasNext()) {
ASSERT_OK_FAST(iter->NextBlock(&block));
for (size_t i = 0; i < block.nrows(); i++) {
@@ -284,7 +284,7 @@ TEST_F(TestCFileSet, TestPartiallyMaterialize) {
ASSERT_OK(iter->Init(nullptr));
Arena arena(4096);
- RowBlock block(schema_, 100, &arena);
+ RowBlock block(&schema_, 100, &arena);
rowid_t row_idx = 0;
while (iter->HasNext()) {
arena.Reset();
diff --git a/src/kudu/tablet/compaction-test.cc b/src/kudu/tablet/compaction-test.cc
index 6c83169..e5bafea 100644
--- a/src/kudu/tablet/compaction-test.cc
+++ b/src/kudu/tablet/compaction-test.cc
@@ -111,7 +111,7 @@ class TestCompaction : public KuduRowSetTest {
TestCompaction()
: KuduRowSetTest(CreateSchema()),
op_id_(consensus::MaximumOpId()),
- row_builder_(schema_),
+ row_builder_(&schema_),
arena_(32*1024),
clock_(clock::LogicalClock::CreateStartingAt(Timestamp::kInitialTimestamp)),
log_anchor_registry_(new log::LogAnchorRegistry()) {
@@ -160,9 +160,9 @@ class TestCompaction : public KuduRowSetTest {
int row_key,
int32_t val) {
BuildRow(row_key, val);
- if (!mrs->schema().Equals(row_builder_.schema())) {
+ if (!mrs->schema().Equals(*row_builder_.schema())) {
// The MemRowSet is not projecting the row, so must be done by the caller
- RowProjector projector(&row_builder_.schema(), &mrs->schema());
+ RowProjector projector(row_builder_.schema(), &mrs->schema());
uint8_t rowbuf[ContiguousRowHelper::row_size(mrs->schema())];
ContiguousRow dst_row(&mrs->schema(), rowbuf);
ASSERT_OK_FAST(projector.Init());
@@ -216,7 +216,8 @@ class TestCompaction : public KuduRowSetTest {
nullable_col_id, &new_val);
}
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema proj_key = schema_.CreateKeyProjection();
+ RowBuilder rb(&proj_key);
rb.AddString(Slice(keybuf));
RowSetKeyProbe probe(rb.row());
ProbeStats stats;
@@ -253,7 +254,8 @@ class TestCompaction : public KuduRowSetTest {
RowChangeListEncoder update(&update_buf);
update.SetToDelete();
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema proj_key = schema_.CreateKeyProjection();
+ RowBuilder rb(&proj_key);
rb.AddString(Slice(keybuf));
RowSetKeyProbe probe(rb.row());
ProbeStats stats;
@@ -775,7 +777,7 @@ TEST_F(TestCompaction, TestDuplicatedRowsRandomCompaction) {
}
- RowBlock block(schema_, kBaseNumRowSets * kNumRowsPerRowSet, &arena_);
+ RowBlock block(&schema_, kBaseNumRowSets * kNumRowsPerRowSet, &arena_);
// Go through the expected compaction input rows, flip the last undo into a redo and
// build the base. This will give us the final version that we'll expect the result
// of the real compaction to match.
diff --git a/src/kudu/tablet/compaction.cc b/src/kudu/tablet/compaction.cc
index 5f4510d..a076302 100644
--- a/src/kudu/tablet/compaction.cc
+++ b/src/kudu/tablet/compaction.cc
@@ -115,7 +115,7 @@ class MemRowSetCompactionInput : public CompactionInput {
// Realloc the internal block storage if we don't have enough space to
// copy the whole leaf node's worth of data into it.
if (PREDICT_FALSE(!row_block_ || num_in_block > row_block_->nrows())) {
- row_block_.reset(new RowBlock(iter_->schema(), num_in_block, nullptr));
+ row_block_.reset(new RowBlock(&iter_->schema(), num_in_block, nullptr));
}
arena_.Reset();
@@ -202,7 +202,7 @@ class DiskRowSetCompactionInput : public CompactionInput {
redo_delta_iter_(std::move(redo_delta_iter)),
undo_delta_iter_(std::move(undo_delta_iter)),
arena_(32 * 1024),
- block_(base_iter_->schema(), kRowsPerBlock, &arena_),
+ block_(&base_iter_->schema(), kRowsPerBlock, &arena_),
redo_mutation_block_(kRowsPerBlock, static_cast<Mutation *>(nullptr)),
undo_mutation_block_(kRowsPerBlock, static_cast<Mutation *>(nullptr)) {}
@@ -689,7 +689,7 @@ class MergeCompactionInput : public CompactionInput {
num_dup_rows_++;
if (row_idx == 0) {
duplicated_rows_.push_back(std::unique_ptr<RowBlock>(
- new RowBlock(*schema_, kDuplicatedRowsPerBlock, static_cast<Arena*>(nullptr))));
+ new RowBlock(schema_, kDuplicatedRowsPerBlock, static_cast<Arena*>(nullptr))));
}
return duplicated_rows_.back()->row(row_idx);
}
@@ -1101,7 +1101,7 @@ Status FlushCompactionInput(CompactionInput* input,
DCHECK(out->schema().has_column_ids());
- RowBlock block(out->schema(), kCompactionOutputBlockNumRows, nullptr);
+ RowBlock block(&out->schema(), kCompactionOutputBlockNumRows, nullptr);
while (input->HasMoreBlocks()) {
RETURN_NOT_OK(input->PrepareBlock(&rows));
diff --git a/src/kudu/tablet/delta_compaction.cc b/src/kudu/tablet/delta_compaction.cc
index dd208b0..d6dbb9e 100644
--- a/src/kudu/tablet/delta_compaction.cc
+++ b/src/kudu/tablet/delta_compaction.cc
@@ -132,7 +132,7 @@ Status MajorDeltaCompaction::FlushRowSetAndDeltas(const IOContext* io_context) {
RETURN_NOT_OK(delta_iter_->SeekToOrdinal(0));
Arena arena(32 * 1024);
- RowBlock block(partial_schema_, kRowsPerBlock, &arena);
+ RowBlock block(&partial_schema_, kRowsPerBlock, &arena);
DVLOG(1) << "Applying deltas and rewriting columns (" << partial_schema_.ToString() << ")";
DeltaStats redo_stats;
diff --git a/src/kudu/tablet/deltafile-test.cc b/src/kudu/tablet/deltafile-test.cc
index 65aefa7..02a72dc 100644
--- a/src/kudu/tablet/deltafile-test.cc
+++ b/src/kudu/tablet/deltafile-test.cc
@@ -176,7 +176,7 @@ class TestDeltaFile : public KuduTest {
ASSERT_OK(s);
ASSERT_OK(it->Init(nullptr));
- RowBlock block(schema_, 100, &arena_);
+ RowBlock block(&schema_, 100, &arena_);
// Iterate through the faked table, starting with batches that
// come before all of the updates, and extending a bit further
diff --git a/src/kudu/tablet/diskrowset-test-base.h b/src/kudu/tablet/diskrowset-test-base.h
index d26ee03..3c11c7c 100644
--- a/src/kudu/tablet/diskrowset-test-base.h
+++ b/src/kudu/tablet/diskrowset-test-base.h
@@ -114,7 +114,7 @@ class TestRowSet : public KuduRowSetTest {
CHECK_OK(writer->Open());
char buf[256];
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
for (int i = 0; i < n_rows; i++) {
CHECK_OK(writer->RollIfNecessary());
rb.Reset();
@@ -179,7 +179,8 @@ class TestRowSet : public KuduRowSetTest {
uint32_t row_idx,
const RowChangeList &mutation,
OperationResultPB* result) {
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema proj_key = schema_.CreateKeyProjection();
+ RowBuilder rb(&proj_key);
BuildRowKey(&rb, row_idx);
RowSetKeyProbe probe(rb.row());
@@ -192,7 +193,8 @@ class TestRowSet : public KuduRowSetTest {
}
Status CheckRowPresent(const DiskRowSet &rs, uint32_t row_idx, bool *present) {
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema proj_key = schema_.CreateKeyProjection();
+ RowBuilder rb(&proj_key);
BuildRowKey(&rb, row_idx);
RowSetKeyProbe probe(rb.row());
ProbeStats stats;
@@ -219,7 +221,7 @@ class TestRowSet : public KuduRowSetTest {
CHECK_OK(row_iter->Init(nullptr));
Arena arena(1024);
int batch_size = 10000;
- RowBlock dst(proj_val, batch_size, &arena);
+ RowBlock dst(&proj_val, batch_size, &arena);
int i = 0;
while (row_iter->HasNext()) {
@@ -283,7 +285,7 @@ class TestRowSet : public KuduRowSetTest {
int batch_size = 1000;
Arena arena(1024);
- RowBlock dst(schema, batch_size, &arena);
+ RowBlock dst(&schema, batch_size, &arena);
int i = 0;
int log_interval = expected_rows/20 / batch_size;
diff --git a/src/kudu/tablet/diskrowset-test.cc b/src/kudu/tablet/diskrowset-test.cc
index 01549cd..8f98be4 100644
--- a/src/kudu/tablet/diskrowset-test.cc
+++ b/src/kudu/tablet/diskrowset-test.cc
@@ -133,7 +133,8 @@ TEST_F(TestRowSet, TestRowSetRoundTrip) {
// 1. Check a key which comes before all keys in rowset
{
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema pk = schema_.CreateKeyProjection();
+ RowBuilder rb(&pk);
rb.AddString(Slice("h"));
RowSetKeyProbe probe(rb.row());
bool present;
@@ -143,7 +144,8 @@ TEST_F(TestRowSet, TestRowSetRoundTrip) {
// 2. Check a key which comes after all keys in rowset
{
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema pk = schema_.CreateKeyProjection();
+ RowBuilder rb(&pk);
rb.AddString(Slice("z"));
RowSetKeyProbe probe(rb.row());
bool present;
@@ -154,7 +156,8 @@ TEST_F(TestRowSet, TestRowSetRoundTrip) {
// 3. Check a key which is not present, but comes between present
// keys
{
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema pk = schema_.CreateKeyProjection();
+ RowBuilder rb(&pk);
rb.AddString(Slice("hello 00000000000049x"));
RowSetKeyProbe probe(rb.row());
bool present;
@@ -165,8 +168,9 @@ TEST_F(TestRowSet, TestRowSetRoundTrip) {
// 4. Check a key which is present
{
char buf[256];
- RowBuilder rb(schema_.CreateKeyProjection());
FormatKey(49, buf, sizeof(buf));
+ Schema pk = schema_.CreateKeyProjection();
+ RowBuilder rb(&pk);
rb.AddString(Slice(buf));
RowSetKeyProbe probe(rb.row());
bool present;
@@ -198,7 +202,8 @@ TEST_F(TestRowSet, TestRowSetUpdate) {
enc.SetToDelete();
Timestamp timestamp(0);
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema proj_key = schema_.CreateKeyProjection();
+ RowBuilder rb(&proj_key);
rb.AddString(Slice("hello 00000000000049x"));
RowSetKeyProbe probe(rb.row());
@@ -224,7 +229,8 @@ TEST_F(TestRowSet, TestErrorDuringUpdate) {
// Get a row that we expect to be in the rowset.
Timestamp timestamp(0);
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema proj_key = schema_.CreateKeyProjection();
+ RowBuilder rb(&proj_key);
rb.AddString(Slice("hello 000000000000050"));
RowSetKeyProbe probe(rb.row());
@@ -383,7 +389,7 @@ TEST_F(TestRowSet, TestFlushedUpdatesRespectMVCC) {
ASSERT_OK(drsw.Open());
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
rb.AddString(key_slice);
rb.AddUint32(1);
ASSERT_OK_FAST(WriteRow(rb.data(), &drsw));
@@ -410,7 +416,8 @@ TEST_F(TestRowSet, TestFlushedUpdatesRespectMVCC) {
tx.StartApplying();
update.Reset();
update.AddColumnUpdate(schema_.column(1), schema_.column_id(1), &i);
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema proj_key = schema_.CreateKeyProjection();
+ RowBuilder rb(&proj_key);
rb.AddString(key_slice);
RowSetKeyProbe probe(rb.row());
OperationResultPB result;
@@ -737,7 +744,7 @@ TEST_P(DiffScanRowSetTest, TestFuzz) {
BloomFilterSizing::BySizeAndFPRate(32 * 1024, 0.01f));
ASSERT_OK(drsw.Open());
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
for (int i = 0; i < 4; i++) {
rb.Reset();
rb.AddUint32(i);
@@ -833,7 +840,8 @@ TEST_P(DiffScanRowSetTest, TestFuzz) {
}
// Build the row key.
- RowBuilder rb(schema_.CreateKeyProjection());
+ Schema proj_key = schema_.CreateKeyProjection();
+ RowBuilder rb(&proj_key);
rb.AddUint32(row_idx);
RowSetKeyProbe probe(rb.row());
diff --git a/src/kudu/tablet/diskrowset.cc b/src/kudu/tablet/diskrowset.cc
index 5725e53..d6972cc 100644
--- a/src/kudu/tablet/diskrowset.cc
+++ b/src/kudu/tablet/diskrowset.cc
@@ -184,7 +184,7 @@ Status DiskRowSetWriter::InitAdHocIndexWriter() {
}
Status DiskRowSetWriter::AppendBlock(const RowBlock &block) {
- DCHECK_EQ(block.schema().num_columns(), schema_->num_columns());
+ DCHECK_EQ(block.schema()->num_columns(), schema_->num_columns());
CHECK(!finished_);
// If this is the very first block, encode the first key and save it as metadata
diff --git a/src/kudu/tablet/memrowset-test.cc b/src/kudu/tablet/memrowset-test.cc
index 4cd3b25..ff8ba4b 100644
--- a/src/kudu/tablet/memrowset-test.cc
+++ b/src/kudu/tablet/memrowset-test.cc
@@ -116,7 +116,7 @@ class TestMemRowSet : public KuduTest {
Status CheckRowPresent(const MemRowSet &mrs,
const string &key, bool *present) {
- RowBuilder rb(key_schema_);
+ RowBuilder rb(&key_schema_);
rb.AddString(Slice(key));
RowSetKeyProbe probe(rb.row());
ProbeStats stats;
@@ -125,7 +125,7 @@ class TestMemRowSet : public KuduTest {
}
Status InsertRows(MemRowSet *mrs, int num_rows) {
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
char keybuf[256];
for (uint32_t i = 0; i < num_rows; i++) {
rb.Reset();
@@ -140,7 +140,7 @@ class TestMemRowSet : public KuduTest {
Status InsertRow(MemRowSet *mrs, const string &key, uint32_t val) {
ScopedTransaction tx(&mvcc_, clock_->Now());
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
rb.AddString(key);
rb.AddUint32(val);
tx.StartApplying();
@@ -160,7 +160,7 @@ class TestMemRowSet : public KuduTest {
RowChangeListEncoder update(&mutation_buf_);
update.AddColumnUpdate(schema_.column(1), schema_.column_id(1), &new_val);
- RowBuilder rb(key_schema_);
+ RowBuilder rb(&key_schema_);
rb.AddString(Slice(key));
RowSetKeyProbe probe(rb.row());
ProbeStats stats;
@@ -183,7 +183,7 @@ class TestMemRowSet : public KuduTest {
RowChangeListEncoder update(&mutation_buf_);
update.SetToDelete();
- RowBuilder rb(key_schema_);
+ RowBuilder rb(&key_schema_);
rb.AddString(Slice(key));
RowSetKeyProbe probe(rb.row());
ProbeStats stats;
@@ -203,7 +203,7 @@ class TestMemRowSet : public KuduTest {
CHECK_OK(iter->Init(nullptr));
Arena arena(1024);
- RowBlock block(schema_, 100, &arena);
+ RowBlock block(&schema_, 100, &arena);
int fetched = 0;
while (iter->HasNext()) {
CHECK_OK(iter->NextBlock(&block));
@@ -298,7 +298,7 @@ TEST_F(TestMemRowSet, TestInsertAndIterateCompoundKey) {
ASSERT_OK(MemRowSet::Create(0, compound_key_schema, log_anchor_registry_.get(),
MemTracker::GetRootTracker(), &mrs));
- RowBuilder rb(compound_key_schema);
+ RowBuilder rb(&compound_key_schema);
{
ScopedTransaction tx(&mvcc_, clock_->Now());
tx.StartApplying();
@@ -542,7 +542,7 @@ TEST_F(TestMemRowSet, TestInsertionMVCC) {
{
ScopedTransaction tx(&mvcc_, clock_->Now());
tx.StartApplying();
- RowBuilder rb(schema_);
+ RowBuilder rb(&schema_);
char keybuf[256];
rb.Reset();
snprintf(keybuf, sizeof(keybuf), "tx%d", i);
diff --git a/src/kudu/tablet/mt-rowset_delta_compaction-test.cc b/src/kudu/tablet/mt-rowset_delta_compaction-test.cc
index a1501db..6af4d12 100644
--- a/src/kudu/tablet/mt-rowset_delta_compaction-test.cc
+++ b/src/kudu/tablet/mt-rowset_delta_compaction-test.cc
@@ -109,7 +109,7 @@ class TestMultiThreadedRowSetDeltaCompaction : public TestRowSet {
void ReadVerify(DiskRowSet *rs) {
Arena arena(1024);
- RowBlock dst(schema_, 1000, &arena);
+ RowBlock dst(&schema_, 1000, &arena);
RowIteratorOptions opts;
opts.projection = &schema_;
unique_ptr<RowwiseIterator> iter;
diff --git a/src/kudu/tablet/mt-tablet-test.cc b/src/kudu/tablet/mt-tablet-test.cc
index f5a15e9..2e3afaa 100644
--- a/src/kudu/tablet/mt-tablet-test.cc
+++ b/src/kudu/tablet/mt-tablet-test.cc
@@ -143,7 +143,7 @@ class MultiThreadedTabletTest : public TabletTestBase<SETUP> {
LocalTabletWriter writer(this->tablet().get(), &this->client_schema_);
Arena tmp_arena(1024);
- RowBlock block(schema_, 1, &tmp_arena);
+ RowBlock block(&schema_, 1, &tmp_arena);
faststring update_buf;
uint64_t updates_since_last_report = 0;
@@ -213,7 +213,7 @@ class MultiThreadedTabletTest : public TabletTestBase<SETUP> {
// trying to reference already-freed memrowset memory.
void SlowReaderThread(int /*tid*/) {
Arena arena(32*1024);
- RowBlock block(schema_, 1, &arena);
+ RowBlock block(&schema_, 1, &arena);
uint64_t max_rows = this->ClampRowCount(FLAGS_inserts_per_thread * FLAGS_num_insert_threads)
/ FLAGS_num_insert_threads;
@@ -248,7 +248,7 @@ class MultiThreadedTabletTest : public TabletTestBase<SETUP> {
Arena arena(1024); // unused, just scanning ints
static const int kBufInts = 1024*1024 / 8;
- RowBlock block(valcol_projection_, kBufInts, &arena);
+ RowBlock block(&valcol_projection_, kBufInts, &arena);
ColumnBlock column = block.column_block(0);
uint64_t count_since_report = 0;
diff --git a/src/kudu/tablet/tablet-decoder-eval-test.cc b/src/kudu/tablet/tablet-decoder-eval-test.cc
index eb4ff7c..3c5f43c 100644
--- a/src/kudu/tablet/tablet-decoder-eval-test.cc
+++ b/src/kudu/tablet/tablet-decoder-eval-test.cc
@@ -55,6 +55,7 @@ DEFINE_int32(decoder_eval_test_lower, 0, "Lower bound on the predicate [lower, u
DEFINE_int32(decoder_eval_test_upper, 50, "Upper bound on the predicate [lower, upper)");
DEFINE_int32(decoder_eval_test_strlen, 10, "Number of strings per cell");
+using std::string;
using std::unique_ptr;
using strings::Substitute;
@@ -136,7 +137,7 @@ public:
}
void FillTestTablet(size_t nrows, size_t cardinality, size_t strlen, int null_upper) {
- RowBuilder rb(client_schema_);
+ RowBuilder rb(&client_schema_);
LocalTabletWriter writer(tablet().get(), &client_schema_);
KuduPartialRow row(&client_schema_);
for (int64_t i = 0; i < nrows; i++) {
@@ -163,8 +164,8 @@ public:
ScanSpec spec;
// Generate the predicate.
- const std::string lower_string = LeftZeroPadded(lower_val, strlen);
- const std::string upper_string = LeftZeroPadded(upper_val, strlen);
+ const string lower_string = LeftZeroPadded(lower_val, strlen);
+ const string upper_string = LeftZeroPadded(upper_val, strlen);
Slice lower(lower_string);
Slice upper(upper_string);
auto string_pred = ColumnPredicate::Range(schema_.column(2), &lower, &upper);
@@ -211,7 +212,7 @@ public:
return (nrows / cardinality) * (upper - lower) + last_chunk_count;
}
- std::string LeftZeroPadded(size_t n, size_t strlen) {
+ string LeftZeroPadded(size_t n, size_t strlen) {
// Assumes the string representation of n is under strlen characters.
return StringPrintf(Substitute("%0$0$1", strlen, PRId64).c_str(), static_cast<int64_t>(n));
}
@@ -231,12 +232,12 @@ public:
ScanSpec spec;
// Generate the predicates [0, upper) AND [lower, cardinality).
- const std::string lower_string_a(LeftZeroPadded(0, strlen));
- const std::string upper_string_a(LeftZeroPadded(upper, strlen));
+ const string lower_string_a(LeftZeroPadded(0, strlen));
+ const string upper_string_a(LeftZeroPadded(upper, strlen));
Slice lower_a(lower_string_a);
Slice upper_a(upper_string_a);
- const std::string lower_string_b = LeftZeroPadded(lower, strlen);
- const std::string upper_string_b = LeftZeroPadded(cardinality, strlen);
+ const string lower_string_b = LeftZeroPadded(lower, strlen);
+ const string upper_string_b = LeftZeroPadded(cardinality, strlen);
Slice lower_b(lower_string_b);
Slice upper_b(upper_string_b);
@@ -260,9 +261,9 @@ public:
Arena ret_arena(1024);
size_t expected_count = ExpectedCount(nrows, cardinality, lower, upper);
Schema schema = iter->schema();
- RowBlock block(schema, 100, &ret_arena);
+ RowBlock block(&schema, 100, &ret_arena);
int fetched = 0;
- std::string column_str_a, column_str_b;
+ string column_str_a, column_str_b;
while (iter->HasNext()) {
ASSERT_OK(iter->NextBlock(&block));
for (size_t i = 0; i < block.nrows(); i++) {
diff --git a/src/kudu/tablet/tablet-pushdown-test.cc b/src/kudu/tablet/tablet-pushdown-test.cc
index c9bee93..caeb227 100644
--- a/src/kudu/tablet/tablet-pushdown-test.cc
+++ b/src/kudu/tablet/tablet-pushdown-test.cc
@@ -74,7 +74,7 @@ class TabletPushdownTest : public KuduTabletTest,
}
void FillTestTablet() {
- RowBuilder rb(client_schema_);
+ RowBuilder rb(&client_schema_);
nrows_ = 2100;
if (AllowSlowTests()) {
@@ -236,7 +236,7 @@ class TabletSparsePushdownTest : public KuduTabletTest {
void SetUp() override {
KuduTabletTest::SetUp();
- RowBuilder rb(client_schema_);
+ RowBuilder rb(&client_schema_);
LocalTabletWriter writer(tablet().get(), &client_schema_);
KuduPartialRow row(&client_schema_);
diff --git a/src/kudu/tablet/tablet-test-base.h b/src/kudu/tablet/tablet-test-base.h
index ef98ef1..4a86a2b 100644
--- a/src/kudu/tablet/tablet-test-base.h
+++ b/src/kudu/tablet/tablet-test-base.h
@@ -435,7 +435,7 @@ class TabletTestBase : public KuduTabletTest {
int batch_size = std::max<size_t>(1, std::min<size_t>(expected_row_count / 10,
4L * 1024 * 1024 / schema_.byte_size()));
Arena arena(32*1024);
- RowBlock block(schema_, batch_size, &arena);
+ RowBlock block(&schema_, batch_size, &arena);
bool check_for_dups = true;
if (expected_row_count > INT_MAX) {
diff --git a/src/kudu/tablet/tablet-test-util.h b/src/kudu/tablet/tablet-test-util.h
index 9571d4b..267a18c 100644
--- a/src/kudu/tablet/tablet-test-util.h
+++ b/src/kudu/tablet/tablet-test-util.h
@@ -199,7 +199,7 @@ static inline Status SilentIterateToStringList(RowwiseIterator* iter,
int* fetched) {
const Schema& schema = iter->schema();
Arena arena(1024);
- RowBlock block(schema, 100, &arena);
+ RowBlock block(&schema, 100, &arena);
*fetched = 0;
while (iter->HasNext()) {
RETURN_NOT_OK(iter->NextBlock(&block));
@@ -218,7 +218,7 @@ static inline Status IterateToStringList(RowwiseIterator* iter,
out->clear();
Schema schema = iter->schema();
Arena arena(1024);
- RowBlock block(schema, 100, &arena);
+ RowBlock block(&schema, 100, &arena);
int fetched = 0;
while (iter->HasNext() && fetched < limit) {
RETURN_NOT_OK(iter->NextBlock(&block));
@@ -331,7 +331,7 @@ static Status WriteRow(const Slice &row_slice, RowSetWriterClass *writer) {
const Schema &schema = writer->schema();
DCHECK_EQ(row_slice.size(), schema.byte_size());
- RowBlock block(schema, 1, nullptr);
+ RowBlock block(&schema, 1, nullptr);
ConstContiguousRow row(&schema, row_slice.data());
RowBlockRow dst_row = block.row(0);
RETURN_NOT_OK(CopyRow(row, &dst_row, static_cast<Arena*>(nullptr)));
@@ -672,7 +672,7 @@ void CreateRandomDeltas(const Schema& schema,
if (is_deleted) {
// The row is deleted; we must REINSERT it.
DCHECK(allow_reinserts);
- RowBuilder rb(schema);
+ RowBuilder rb(&schema);
for (int i = 0; i < schema.num_columns(); i++) {
rb.AddUint32(prng->Next());
}
diff --git a/src/kudu/tablet/tablet-test.cc b/src/kudu/tablet/tablet-test.cc
index 3ceb6c1..895a185 100644
--- a/src/kudu/tablet/tablet-test.cc
+++ b/src/kudu/tablet/tablet-test.cc
@@ -491,7 +491,7 @@ TYPED_TEST(TestTablet, TestRowIteratorSimple) {
ASSERT_TRUE(iter->HasNext());
- RowBlock block(this->schema_, 100, &this->arena_);
+ RowBlock block(&this->schema_, 100, &this->arena_);
// First call to CopyNextRows should fetch the whole memrowset.
ASSERT_OK_FAST(iter->NextBlock(&block));
@@ -564,7 +564,7 @@ TYPED_TEST(TestTablet, TestRowIteratorOrdered) {
// Iterate the tablet collecting rows.
vector<shared_ptr<faststring> > rows;
for (int i = 0; i < numBlocks; i++) {
- RowBlock block(this->schema_, rowsPerBlock, &this->arena_);
+ RowBlock block(&this->schema_, rowsPerBlock, &this->arena_);
ASSERT_TRUE(iter->HasNext());
ASSERT_OK(iter->NextBlock(&block));
ASSERT_EQ(rowsPerBlock, block.nrows()) << "unexpected number of rows returned";
@@ -667,7 +667,7 @@ TYPED_TEST(TestTablet, TestRowIteratorComplex) {
vector<bool> seen(max_rows, false);
int seen_count = 0;
- RowBlock block(schema, 100, &this->arena_);
+ RowBlock block(&schema, 100, &this->arena_);
while (iter->HasNext()) {
this->arena_.Reset();
ASSERT_OK(iter->NextBlock(&block));
diff --git a/src/kudu/tablet/tablet_random_access-test.cc b/src/kudu/tablet/tablet_random_access-test.cc
index d8fc442..0c87894 100644
--- a/src/kudu/tablet/tablet_random_access-test.cc
+++ b/src/kudu/tablet/tablet_random_access-test.cc
@@ -281,7 +281,7 @@ class TestRandomAccess : public KuduTabletTest {
int n_results = 0;
Arena arena(1024);
- RowBlock block(schema, 100, &arena);
+ RowBlock block(&schema, 100, &arena);
while (iter->HasNext()) {
arena.Reset();
CHECK_OK(iter->NextBlock(&block));
diff --git a/src/kudu/tserver/tablet_server-test-base.cc b/src/kudu/tserver/tablet_server-test-base.cc
index 524a11a..1bfd976 100644
--- a/src/kudu/tserver/tablet_server-test-base.cc
+++ b/src/kudu/tserver/tablet_server-test-base.cc
@@ -102,7 +102,7 @@ void TabletServerTestBase::SetUp() {
KuduTest::SetUp();
key_schema_ = schema_.CreateKeyProjection();
- rb_.reset(new RowBuilder(schema_));
+ rb_.reset(new RowBuilder(&schema_));
rpc::MessengerBuilder bld("Client");
ASSERT_OK(bld.Build(&client_messenger_));
@@ -411,7 +411,7 @@ void TabletServerTestBase::VerifyRows(const Schema& schema,
4*1024*1024 / schema.byte_size()));
Arena arena(32*1024);
- RowBlock block(schema, batch_size, &arena);
+ RowBlock block(&schema, batch_size, &arena);
int count = 0;
while (iter->HasNext()) {
diff --git a/src/kudu/tserver/tablet_service.cc b/src/kudu/tserver/tablet_service.cc
index d240b68..1335b5d 100644
--- a/src/kudu/tserver/tablet_service.cc
+++ b/src/kudu/tserver/tablet_service.cc
@@ -783,7 +783,7 @@ class ScanResultChecksummer : public ScanResultCollector {
const Schema* client_projection_schema = scanner->client_projection_schema();
if (!client_projection_schema) {
- client_projection_schema = &row_block.schema();
+ client_projection_schema = row_block.schema();
}
size_t nrows = row_block.nrows();
@@ -2543,7 +2543,7 @@ Status TabletServiceImpl::HandleContinueScanRequest(const ScanRequestPB* req,
// If people had really large indirect objects, we would currently overshoot
// their requested batch size by a lot.
Arena arena(32 * 1024);
- RowBlock block(scanner->iter()->schema(),
+ RowBlock block(&scanner->iter()->schema(),
FLAGS_scanner_batch_size_rows, &arena);
// TODO(todd): in the future, use the client timeout to set a budget. For now,