You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2026/01/23 23:40:10 UTC

[kudu-CR] WIP [codegen] add CodegenTest.CodegenRandomSchemas scenario

Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23893


Change subject: WIP [codegen] add CodegenTest.CodegenRandomSchemas scenario
......................................................................

WIP [codegen] add CodegenTest.CodegenRandomSchemas scenario

The newly added test scenario reproduces SIGSEGV crashes on RHEL9 with
libgcc of the following version:
  libgcc-11.5.0-11.el9.x86_64

WIP: add proper description

Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
---
M src/kudu/codegen/codegen-test.cc
1 file changed, 261 insertions(+), 6 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/23893/4
-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.

Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23893 )

Change subject: KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario
......................................................................


Patch Set 7:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23893/7/src/kudu/codegen/codegen-test.cc
File src/kudu/codegen/codegen-test.cc:

http://gerrit.cloudera.org:8080/#/c/23893/7/src/kudu/codegen/codegen-test.cc@688
PS7, Line 688:         const size_t num_columns = 1 + gen() % cs_library.size();
             :         VLOG(1) << StringPrintf("thread %2zd: %2zd-column schema",
             :                                   thread_idx, num_columns);
             :         vector<size_t> idx_seq;
             :         idx_seq.reserve(num_columns);
             :         for (size_t i = 0; i < num_columns; ++i) {
             :           idx_seq.push_back(gen() % cs_library.size());
             :         }
             : 
             :         // Create a schema with the given number of columns, picking columns
             :         // from the 'column schema library' in random order.
             :         unordered_set<size_t> seen_idx;
             :         seen_idx.emplace(0);  // the 'key' column is always present
             :         SchemaBuilder sb;
             :         CHECK_OK(sb.AddKeyColumn(cs_library.front()));
             :         for (auto idx : idx_seq) {
             :           if (!seen_idx.emplace(idx).second) {
             :             // A randomly chosen index is duplicate, ignore and continue.
             :             continue;
             :           }
             :           CHECK_OK(sb.AddColumn(cs_library[idx]));
             :         }
> std::sample does exactly this (no repetition of the same element).
That's a good point, done.


http://gerrit.cloudera.org:8080/#/c/23893/7/src/kudu/codegen/codegen-test.cc@716
PS7, Line 716:            vector<size_t> col_indices(gen() % seen_idx.size());
             :             std::iota(col_indices.begin(), col_indices.end(), 0);
             :             std::shuffle(col_indices.begin(), col_indices.end(), gen);
             : 
             :             VLOG(3) << StringPrintf("thread %2zd: %2zd-column projection",
             :                                     thread_idx, col_indices.size());
             : 
             :             // Convert the indices of columns in the 'column schema library'
             :             // into internal column identifiers of this current schema.
             :             vector<ColumnId> col_ids;
             :             col_ids.reserve(col_indices.size());
             :             for (size_t col_idx : col_indices) {
             :               col_ids.push_back(schema.column_id(col_idx));
             :             }
             :             // Create a projection with columns at the specified indices.
             :             CHECK_OK(schema.CreateProjectionByIdsIgnoreMissing(col_ids, &projection));
             :           }
             : 
             :           // Request code generation.
> Could be much simpler with std::sample.
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 7
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Feb 2026 07:21:17 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.

Alexey Serbin has removed a vote on this change.

Change subject: KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 9
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.

Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23893 )

Change subject: KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario
......................................................................


Patch Set 6: Verified+1

TSAN: unrelated test failures in org.apache.kudu.spark.kudu.DefaultSourceTest and delete_tablet-itest


-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 02:29:41 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.

Marton Greber has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23893 )

Change subject: KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario
......................................................................

KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

This new test scenario reproduces SIGSEGV crash on RedHat9 with libgcc
versions 11.5.0-10 and newer.  At least, without KUDU-3736 fix in place,
running the scenario with --codegen_test_random_schemas_runtime_sec=-1
crashes every time on x86_64 RHEL9 when the following libgcc package
is installed:

  libgcc-11.5.0-11.el9.x86_64

Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Reviewed-on: http://gerrit.cloudera.org:8080/23893
Tested-by: Alexey Serbin <al...@apache.org>
Reviewed-by: Zoltan Martonka <zm...@cloudera.com>
Reviewed-by: Marton Greber <gr...@gmail.com>
---
M src/kudu/codegen/codegen-test.cc
1 file changed, 270 insertions(+), 6 deletions(-)

Approvals:
  Alexey Serbin: Verified
  Zoltan Martonka: Looks good to me, but someone else must approve
  Marton Greber: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.

Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23893 )

Change subject: KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario
......................................................................


Patch Set 6: Verified-1

Build Failed 

http://jenkins.kudu.apache.org/job/pre_commit/2244/ : FAILURE


-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 01:10:55 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.

Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23893 )

Change subject: KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario
......................................................................


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23893/6/src/kudu/codegen/codegen-test.cc
File src/kudu/codegen/codegen-test.cc:

http://gerrit.cloudera.org:8080/#/c/23893/6/src/kudu/codegen/codegen-test.cc@74
PS6, Line 74: 60
> > Is it possible to see crash while this runs for 60 seconds?
I updated the commit description in PS7 to clarify that the test crashes only if KUDU-3736 fix isn't in place.  Hopefully, it helps to avoid confusion, if any.



-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Feb 2026 19:51:24 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.

Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23893 )

Change subject: KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario
......................................................................


Patch Set 5:

Build Started http://jenkins.kudu.apache.org/job/pre_commit/2240/


-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Feb 2026 04:53:24 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.

Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23893 )

Change subject: KUDU-3736 add CodegenTest.CodegenRandomSchemas scenario
......................................................................


Patch Set 9: Verified-1

Build Failed 

http://jenkins.kudu.apache.org/job/pre_commit/2253/ : FAILURE


-- 
To view, visit http://gerrit.cloudera.org:8080/23893
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic51e8fec02f74ecc11fa740f05ffeb9a7f41d8d9
Gerrit-Change-Number: 23893
Gerrit-PatchSet: 9
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Fri, 06 Feb 2026 08:22:58 +0000
Gerrit-HasComments: No