You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2025/10/13 22:07:07 UTC

[Impala-ASF-CR] More python3 migration

Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23468


Change subject: More python3 migration
......................................................................

More python3 migration

Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
---
M be/src/codegen/gen_ir_descriptions.py
M bin/bootstrap_system.sh
M bin/dump_breakpad_symbols.py
M bin/gen_build_version.py
M bin/generate_minidump_collection_testdata.py
M bin/get_code_size.py
M bin/inline_pom.py
M bin/load-data.py
M bin/parse-thrift-profile.py
M bin/run-workload.py
M bin/single_node_perf_run.py
M docker/setup_build_context.py
M fe/src/test/java/org/apache/impala/customcluster/LdapImpylaHttpTest.java
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/impala_py_lib/profiles.py
M testdata/bin/check-hbase-nodes.py
M testdata/bin/create-load-data.sh
M testdata/bin/create-tpc-jdbc-tables.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load-tpc-kudu.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/bin/wait-for-hiveserver2.py
M testdata/bin/wait-for-metastore.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapper.py
M tests/comparison/data_generator_mapred_common.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_no_delete_stats_in_truncate.py
M tests/custom_cluster/test_shared_catalogd.py
M tests/metadata/test_column_unicode.py
M tests/shell/test_beeline.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_commandline_jwt_auth.py
M tests/shell/test_shell_commandline_kerberos_auth.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/stress/concurrent_select.py
M tests/stress/extract_min_mem.py
M tests/stress/mem_broker.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/query_runner.py
M tests/stress/runtime_info.py
M tests/stress/util.py
M tests/util/acid_txn.py
M tests/util/auto_scaler.py
M tests/util/concurrent_workload.py
M tests/util/run_impyla_http_query.py
62 files changed, 49 insertions(+), 313 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/23468/1
-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 6:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/20746/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 6
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 16:00:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 5: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 21 Oct 2025 00:15:32 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................

IMPALA-14501: Migrate most scripts from impala-python to impala-python3

To remove the dependency on Python 2, existing scripts need to use
python3 rather than python. These commands find those
locations (for impala-python and regular python):
git grep impala-python | grep -v impala-python3 | grep -v impala-python-common | grep -v init-impala-python
git grep bin/python | grep -v python3

This removes or switches most of these locations by various means:
1. If a python file has a #!/bin/env impala-python (or python) but
   doesn't have a main function, it removes the hash-bang and makes
   sure that the file is not executable.
2. Most scripts can simply switch from impala-python to impala-python3
   (or python to python3) with minimal changes.
3. The cm-api pypi package (which doesn't support Python 3) has been
   replaced by the cm-client pypi package and interfaces have changed.
   Rather than migrating the code (which hasn't been used in years), this
   deletes the old code and stops installing cm-api into the virtualenv.
   The code can be restored and revamped if there is any interest in
   interacting with CM clusters.
4. This switches tests/comparison over to impala-python3, but this code has
   bit-rotted. Some pieces can be run manually, but it can't be fully
   verified with Python 3. It shouldn't hold back the migration on its own.
5. This also replaces locations of impala-python in comments / documentation /
   READMEs.
6. kazoo (used for interacting with HBase) needed to be upgraded to a
   version that supports Python 3. The newest version of kazoo requires
   upgrades of other component versions, so this uses kazoo 2.8.0 to avoid
   needing other upgrades.

The two remaining uses of impala-python are:
 - bin/cmake_aux/create_virtualenv.sh
 - bin/impala-env-versioned-python
These will be removed separately when we drop Python 2 support
completely. In particular, these are useful for testing impala-shell
with Python 2 until we stop supporting Python 2 for impala-shell.

The docker-based tests still use /usr/bin/python, but this can
be switched over independently (and doesn't impact impala-python)

Testing:
 - Ran core job
 - Ran build + dataload on Centos 7, Redhat 8
 - Manual testing of individual scripts (except some bitrotted areas like the
   random query generator)

Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Reviewed-on: http://gerrit.cloudera.org:8080/23468
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Michael Smith <mi...@cloudera.com>
---
M be/src/codegen/gen_ir_descriptions.py
M bin/bootstrap_system.sh
M bin/bootstrap_toolchain.py
M bin/check-rat-report.py
M bin/collect_minidumps.py
M bin/compare_branches.py
M bin/diagnostics/collect_diagnostics.py
M bin/dump_breakpad_symbols.py
M bin/gen-backend-test-script.py
M bin/gen_build_version.py
M bin/generate_minidump_collection_testdata.py
M bin/generate_xml_config.py
M bin/get_code_size.py
M bin/inline_pom.py
M bin/jenkins/critique-gerrit-review.py
M bin/jenkins/dockerized-impala-preserve-vars.py
M bin/jenkins/populate_m2_directory.py
M bin/jenkins/thrift_parser.py
M bin/load-data.py
M bin/parse-thrift-profile.py
M bin/run-workload.py
M bin/set-pythonpath.sh
M bin/single_node_perf_run.py
M bin/validate-unified-backend-test-filters.py
M common/function-registry/gen_builtins_catalog.py
M common/function-registry/gen_geospatial_udf_wrappers.py
M common/thrift/generate_error_codes.py
M common/thrift/generate_metrics.py
M docker/setup_build_context.py
M fe/src/test/java/org/apache/impala/customcluster/LdapImpylaHttpTest.java
M fe/src/test/resources/hive-site.xml.py
M infra/python/README
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/impala_py_lib/jenkins/junitxml_prune_notrun.py
M lib/python/impala_py_lib/profiles.py
M lib/python/setup.py
M shell/CMakeLists.txt
M shell/impala_shell/compatibility.py
M shell/impala_shell/exec_summary.py
M shell/impala_shell/impala_client.py
M shell/impala_shell/impala_shell.py
M shell/impala_shell/impala_shell_config_defaults.py
M shell/impala_shell/option_parser.py
M shell/impala_shell/shell_output.py
M shell/packaging/make_python_package.sh
M shell/setup.py
M testdata/bin/check-hbase-nodes.py
M testdata/bin/create-load-data.sh
M testdata/bin/create-tpc-jdbc-tables.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load-tpc-kudu.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/bin/wait-for-hiveserver2.py
M testdata/bin/wait-for-metastore.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/ozone-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/comparison/ORACLE.txt
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapper.py
M tests/comparison/data_generator_mapred_common.py
M tests/comparison/discrepancy_searcher.py
M tests/comparison/leopard/controller.py
M tests/comparison/leopard/front_end.py
M tests/comparison/tests/README
M tests/comparison/tests/conftest.py
M tests/comparison/util/verify-oracle-connection.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_no_delete_stats_in_truncate.py
M tests/custom_cluster/test_shared_catalogd.py
M tests/metadata/test_column_unicode.py
M tests/shell/test_beeline.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_commandline_jwt_auth.py
M tests/shell/test_shell_commandline_kerberos_auth.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/stress/concurrent_select.py
M tests/stress/extract_min_mem.py
M tests/stress/mem_broker.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/query_runner.py
M tests/stress/runtime_info.py
M tests/stress/util.py
M tests/util/acid_txn.py
M tests/util/auto_scaler.py
M tests/util/concurrent_workload.py
M tests/util/run_impyla_http_query.py
103 files changed, 97 insertions(+), 379 deletions(-)

Approvals:
  Michael Smith: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 10
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 5:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/20733/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Oct 2025 19:25:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23468

to look at the new patch set (#3).

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................

IMPALA-14501: Migrate most scripts from impala-python to impala-python3

To remove the dependency on Python 2, existing scripts need to use
impala-python3 rather than impala-python. This command finds those
locations:
git grep impala-python | grep -v impala-python3 | grep -v impala-python-common | grep -v init-impala-python

This removes or switches most of these locations by various means:
1. If a python file has a #!/bin/env impala-python but doesn't have a main
   function, it removes the hash-bang and makes sure that the file is
   not executable.
2. Most scripts can simply switch from impala-python to impala-python3
   with minimal changes.
3. The cm-api pypi package (which doesn't support Python 3) has been
   replaced by the cm-client pypi package and interfaces have changed.
   Rather than migrating the code (which hasn't been used in years), this
   deletes the old code and stops installing cm-api into the virtualenv.
   The code can be restored and revamped if there is any interest in
   interacting with CM clusters.
4. This switches tests/comparison over to impala-python3, but this code has
   bit-rotted. Some pieces can be run manually, but it can't be fully
   verified with Python 3. It shouldn't hold back the migration on its own.
5. This also replaces locations of impala-python in comments / documentation /
   READMEs.
6. kazoo (used for interacting with HBase) needed to be upgraded to a
   version that supports Python 3. The newest version of kazoo requires
   upgrades of other component versions, so this uses kazoo 2.8.0 to avoid
   needing other upgrades.

The two remaining uses of impala-python are:
 - impala-shell packaging
 - bin/impala-env-versioned-python
These will be removed separately when we drop Python 2 support for
impala-shell.

Testing:
 - Ran core job
 - Manual testing of individual scripts (except some bitrotted areas like the
   random query generator)

Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
---
M be/src/codegen/gen_ir_descriptions.py
M bin/bootstrap_system.sh
M bin/dump_breakpad_symbols.py
M bin/gen_build_version.py
M bin/generate_minidump_collection_testdata.py
M bin/get_code_size.py
M bin/inline_pom.py
M bin/load-data.py
M bin/parse-thrift-profile.py
M bin/run-workload.py
M bin/set-pythonpath.sh
M bin/single_node_perf_run.py
M docker/setup_build_context.py
M fe/src/test/java/org/apache/impala/customcluster/LdapImpylaHttpTest.java
M infra/python/README
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/impala_py_lib/profiles.py
M testdata/bin/check-hbase-nodes.py
M testdata/bin/create-load-data.sh
M testdata/bin/create-tpc-jdbc-tables.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load-tpc-kudu.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/bin/wait-for-hiveserver2.py
M testdata/bin/wait-for-metastore.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/comparison/ORACLE.txt
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapper.py
M tests/comparison/data_generator_mapred_common.py
M tests/comparison/discrepancy_searcher.py
M tests/comparison/leopard/controller.py
M tests/comparison/leopard/front_end.py
M tests/comparison/tests/README
M tests/comparison/tests/conftest.py
M tests/comparison/util/verify-oracle-connection.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_no_delete_stats_in_truncate.py
M tests/custom_cluster/test_shared_catalogd.py
M tests/metadata/test_column_unicode.py
M tests/shell/test_beeline.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_commandline_jwt_auth.py
M tests/shell/test_shell_commandline_kerberos_auth.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/stress/concurrent_select.py
M tests/stress/extract_min_mem.py
M tests/stress/mem_broker.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/query_runner.py
M tests/stress/runtime_info.py
M tests/stress/util.py
M tests/util/acid_txn.py
M tests/util/auto_scaler.py
M tests/util/concurrent_workload.py
M tests/util/run_impyla_http_query.py
71 files changed, 64 insertions(+), 341 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/23468/3
-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 9: Verified+1 Code-Review+2

I think the gerrit-code-review-checks is sufficient for the last fix.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 9
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 16:30:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 3:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/20682/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Oct 2025 20:05:14 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] More python3 migration

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: More python3 migration
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/12560/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Oct 2025 23:10:40 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] More python3 migration

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: More python3 migration
......................................................................


Patch Set 2:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/20671/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Oct 2025 23:25:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23468

to look at the new patch set (#4).

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................

IMPALA-14501: Migrate most scripts from impala-python to impala-python3

To remove the dependency on Python 2, existing scripts need to use
impala-python3 rather than impala-python. This command finds those
locations:
git grep impala-python | grep -v impala-python3 | grep -v impala-python-common | grep -v init-impala-python

This removes or switches most of these locations by various means:
1. If a python file has a #!/bin/env impala-python but doesn't have a main
   function, it removes the hash-bang and makes sure that the file is
   not executable.
2. Most scripts can simply switch from impala-python to impala-python3
   with minimal changes.
3. The cm-api pypi package (which doesn't support Python 3) has been
   replaced by the cm-client pypi package and interfaces have changed.
   Rather than migrating the code (which hasn't been used in years), this
   deletes the old code and stops installing cm-api into the virtualenv.
   The code can be restored and revamped if there is any interest in
   interacting with CM clusters.
4. This switches tests/comparison over to impala-python3, but this code has
   bit-rotted. Some pieces can be run manually, but it can't be fully
   verified with Python 3. It shouldn't hold back the migration on its own.
5. This also replaces locations of impala-python in comments / documentation /
   READMEs.
6. kazoo (used for interacting with HBase) needed to be upgraded to a
   version that supports Python 3. The newest version of kazoo requires
   upgrades of other component versions, so this uses kazoo 2.8.0 to avoid
   needing other upgrades.

The two remaining uses of impala-python are:
 - impala-shell packaging
 - bin/impala-env-versioned-python
These will be removed separately when we drop Python 2 support for
impala-shell.

Testing:
 - Ran core job
 - Manual testing of individual scripts (except some bitrotted areas like the
   random query generator)

Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
---
M be/src/codegen/gen_ir_descriptions.py
M bin/bootstrap_system.sh
M bin/dump_breakpad_symbols.py
M bin/gen_build_version.py
M bin/generate_minidump_collection_testdata.py
M bin/get_code_size.py
M bin/inline_pom.py
M bin/load-data.py
M bin/parse-thrift-profile.py
M bin/run-workload.py
M bin/set-pythonpath.sh
M bin/single_node_perf_run.py
M docker/setup_build_context.py
M fe/src/test/java/org/apache/impala/customcluster/LdapImpylaHttpTest.java
M infra/python/README
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/impala_py_lib/profiles.py
M testdata/bin/check-hbase-nodes.py
M testdata/bin/create-load-data.sh
M testdata/bin/create-tpc-jdbc-tables.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load-tpc-kudu.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/bin/wait-for-hiveserver2.py
M testdata/bin/wait-for-metastore.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/comparison/ORACLE.txt
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapper.py
M tests/comparison/data_generator_mapred_common.py
M tests/comparison/discrepancy_searcher.py
M tests/comparison/leopard/controller.py
M tests/comparison/leopard/front_end.py
M tests/comparison/tests/README
M tests/comparison/tests/conftest.py
M tests/comparison/util/verify-oracle-connection.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_no_delete_stats_in_truncate.py
M tests/custom_cluster/test_shared_catalogd.py
M tests/metadata/test_column_unicode.py
M tests/shell/test_beeline.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_commandline_jwt_auth.py
M tests/shell/test_shell_commandline_kerberos_auth.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/stress/concurrent_select.py
M tests/stress/extract_min_mem.py
M tests/stress/mem_broker.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/query_runner.py
M tests/stress/runtime_info.py
M tests/stress/util.py
M tests/util/acid_txn.py
M tests/util/auto_scaler.py
M tests/util/concurrent_workload.py
M tests/util/run_impyla_http_query.py
71 files changed, 65 insertions(+), 342 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/23468/4
-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23468/3/bin/single_node_perf_run.py
File bin/single_node_perf_run.py:

http://gerrit.cloudera.org:8080/#/c/23468/3/bin/single_node_perf_run.py@1
PS3, Line 1: #!/usr/bin/env impala-python3
> This hits issues with the perf-AB-test job, so it needs more changes: https
Fixed this



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 15 Oct 2025 18:02:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 5:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23468/5/bin/check-rat-report.py
File bin/check-rat-report.py:

http://gerrit.cloudera.org:8080/#/c/23468/5/bin/check-rat-report.py@1
PS5, Line 1: #!/usr/bin/python3
> nit: should we normalize using env?
Yeah, maybe this is a good time to clean that up. Let me take a look.


http://gerrit.cloudera.org:8080/#/c/23468/5/tests/metadata/test_column_unicode.py
File tests/metadata/test_column_unicode.py:

http://gerrit.cloudera.org:8080/#/c/23468/5/tests/metadata/test_column_unicode.py@a2
PS5, Line 2: 
> Why remove "coding: utf-8" here but not later files?
I just saw that this file is 100% ASCII, but we don't really need to remove this.



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Oct 2025 22:26:38 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 5: Code-Review+1

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23468/3/be/src/codegen/gen_ir_descriptions.py
File be/src/codegen/gen_ir_descriptions.py:

http://gerrit.cloudera.org:8080/#/c/23468/3/be/src/codegen/gen_ir_descriptions.py@1
PS3, Line 1: python
> Did another pass to move /usr/bin/env python usages to python3.
Done


http://gerrit.cloudera.org:8080/#/c/23468/3/bin/gen_build_version.py
File bin/gen_build_version.py:

http://gerrit.cloudera.org:8080/#/c/23468/3/bin/gen_build_version.py@1
PS3, Line 1: python
> Done
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Oct 2025 19:01:37 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23468

to look at the new patch set (#5).

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................

IMPALA-14501: Migrate most scripts from impala-python to impala-python3

To remove the dependency on Python 2, existing scripts need to use
python3 rather than python. These commands find those
locations (for impala-python and regular python):
git grep impala-python | grep -v impala-python3 | grep -v impala-python-common | grep -v init-impala-python
git grep bin/python | grep -v python3

This removes or switches most of these locations by various means:
1. If a python file has a #!/bin/env impala-python (or python) but
   doesn't have a main function, it removes the hash-bang and makes
   sure that the file is not executable.
2. Most scripts can simply switch from impala-python to impala-python3
   (or python to python3) with minimal changes.
3. The cm-api pypi package (which doesn't support Python 3) has been
   replaced by the cm-client pypi package and interfaces have changed.
   Rather than migrating the code (which hasn't been used in years), this
   deletes the old code and stops installing cm-api into the virtualenv.
   The code can be restored and revamped if there is any interest in
   interacting with CM clusters.
4. This switches tests/comparison over to impala-python3, but this code has
   bit-rotted. Some pieces can be run manually, but it can't be fully
   verified with Python 3. It shouldn't hold back the migration on its own.
5. This also replaces locations of impala-python in comments / documentation /
   READMEs.
6. kazoo (used for interacting with HBase) needed to be upgraded to a
   version that supports Python 3. The newest version of kazoo requires
   upgrades of other component versions, so this uses kazoo 2.8.0 to avoid
   needing other upgrades.

The two remaining uses of impala-python are:
 - bin/cmake_aux/create_virtualenv.sh
 - bin/impala-env-versioned-python
These will be removed separately when we drop Python 2 support
completely. In particular, these are useful for testing impala-shell
with Python 2 until we stop supporting Python 2 for impala-shell.

The docker-based tests still use /usr/bin/python, but this can
be switched over independently (and doesn't impact impala-python)

Testing:
 - Ran core job
 - Ran build + dataload on Centos 7, Redhat 8
 - Manual testing of individual scripts (except some bitrotted areas like the
   random query generator)

Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
---
M be/src/codegen/gen_ir_descriptions.py
M bin/bootstrap_system.sh
M bin/bootstrap_toolchain.py
M bin/check-rat-report.py
M bin/collect_minidumps.py
M bin/compare_branches.py
M bin/diagnostics/collect_diagnostics.py
M bin/dump_breakpad_symbols.py
M bin/gen-backend-test-script.py
M bin/gen_build_version.py
M bin/generate_minidump_collection_testdata.py
M bin/generate_xml_config.py
M bin/get_code_size.py
M bin/inline_pom.py
M bin/jenkins/dockerized-impala-preserve-vars.py
M bin/jenkins/populate_m2_directory.py
M bin/load-data.py
M bin/parse-thrift-profile.py
M bin/run-workload.py
M bin/set-pythonpath.sh
M bin/single_node_perf_run.py
M bin/validate-unified-backend-test-filters.py
M common/function-registry/gen_builtins_catalog.py
M common/function-registry/gen_geospatial_udf_wrappers.py
M common/thrift/generate_error_codes.py
M common/thrift/generate_metrics.py
M docker/setup_build_context.py
M fe/src/test/java/org/apache/impala/customcluster/LdapImpylaHttpTest.java
M fe/src/test/resources/hive-site.xml.py
M infra/python/README
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/impala_py_lib/jenkins/junitxml_prune_notrun.py
M lib/python/impala_py_lib/profiles.py
M lib/python/setup.py
M shell/CMakeLists.txt
M shell/impala_shell/compatibility.py
M shell/impala_shell/exec_summary.py
M shell/impala_shell/impala_client.py
M shell/impala_shell/impala_shell.py
M shell/impala_shell/impala_shell_config_defaults.py
M shell/impala_shell/option_parser.py
M shell/impala_shell/shell_output.py
M shell/packaging/make_python_package.sh
M shell/setup.py
M testdata/bin/check-hbase-nodes.py
M testdata/bin/create-load-data.sh
M testdata/bin/create-tpc-jdbc-tables.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load-tpc-kudu.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/bin/wait-for-hiveserver2.py
M testdata/bin/wait-for-metastore.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/ozone-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/comparison/ORACLE.txt
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapper.py
M tests/comparison/data_generator_mapred_common.py
M tests/comparison/discrepancy_searcher.py
M tests/comparison/leopard/controller.py
M tests/comparison/leopard/front_end.py
M tests/comparison/tests/README
M tests/comparison/tests/conftest.py
M tests/comparison/util/verify-oracle-connection.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_no_delete_stats_in_truncate.py
M tests/custom_cluster/test_shared_catalogd.py
M tests/metadata/test_column_unicode.py
M tests/shell/test_beeline.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_commandline_jwt_auth.py
M tests/shell/test_shell_commandline_kerberos_auth.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/stress/concurrent_select.py
M tests/stress/extract_min_mem.py
M tests/stress/mem_broker.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/query_runner.py
M tests/stress/runtime_info.py
M tests/stress/util.py
M tests/util/acid_txn.py
M tests/util/auto_scaler.py
M tests/util/concurrent_workload.py
M tests/util/run_impyla_http_query.py
101 files changed, 92 insertions(+), 376 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/23468/5
-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 9: Code-Review+1

(3 comments)

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/jenkins/critique-gerrit-review.py
File bin/jenkins/critique-gerrit-review.py:

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/jenkins/critique-gerrit-review.py@168
PS8, Line 168:     raise Exception("Did not expect flake8-diff to write to stderr:\n{0}".format(stderr))
> Running with verbose output shows
Done


http://gerrit.cloudera.org:8080/#/c/23468/8/bin/load-data.py
File bin/load-data.py:

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/load-data.py@254
PS8, Line 254:     match = re.search(r'dataset:\s*([\w\-\.]+)', input_file.read())
> Needs to be a regex literal
Done


http://gerrit.cloudera.org:8080/#/c/23468/8/shell/setup.py
File shell/setup.py:

http://gerrit.cloudera.org:8080/#/c/23468/8/shell/setup.py@96
PS8, Line 96:       version_match = re.search(r'\d+\.\d+\.\d+', impala_build_version.get_version())
> Need to update this and line 104 to regex literals.
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 9
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 15:58:06 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/jenkins/critique-gerrit-review.py
File bin/jenkins/critique-gerrit-review.py:

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/jenkins/critique-gerrit-review.py@168
PS8, Line 168:     raise Exception("Did not expect flake8-diff to write to stderr:\n{0}".format(stderr))
> Done
I had filed IMPALA-14498 for this. The critique script is checking lines that were not modified. It predates this change. Interestingly, it does this on Ubuntu 22 but not Ubuntu 20.



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 8
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 16:17:30 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 5:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23468/5/bin/check-rat-report.py
File bin/check-rat-report.py:

http://gerrit.cloudera.org:8080/#/c/23468/5/bin/check-rat-report.py@1
PS5, Line 1: #!/usr/bin/python3
> Yeah, maybe this is a good time to clean that up. Let me take a look.
Done


http://gerrit.cloudera.org:8080/#/c/23468/5/tests/metadata/test_column_unicode.py
File tests/metadata/test_column_unicode.py:

http://gerrit.cloudera.org:8080/#/c/23468/5/tests/metadata/test_column_unicode.py@a2
PS5, Line 2: 
> I just saw that this file is 100% ASCII, but we don't really need to remove
Switched back to leaving it in, as it is more consistent with what I did elsewhere.



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 00:24:25 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 8: Verified+1 Code-Review+1

Retriggering auto-critic since it failed, then didn't run for the last patchset. Otherwise looks good.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 8
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 15:30:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] More python3 migration

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23468

to look at the new patch set (#2).

Change subject: More python3 migration
......................................................................

More python3 migration

Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
---
M be/src/codegen/gen_ir_descriptions.py
M bin/bootstrap_system.sh
M bin/dump_breakpad_symbols.py
M bin/gen_build_version.py
M bin/generate_minidump_collection_testdata.py
M bin/get_code_size.py
M bin/inline_pom.py
M bin/load-data.py
M bin/parse-thrift-profile.py
M bin/run-workload.py
M bin/single_node_perf_run.py
M docker/setup_build_context.py
M fe/src/test/java/org/apache/impala/customcluster/LdapImpylaHttpTest.java
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/impala_py_lib/profiles.py
M testdata/bin/check-hbase-nodes.py
M testdata/bin/create-load-data.sh
M testdata/bin/create-tpc-jdbc-tables.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load-tpc-kudu.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/bin/wait-for-hiveserver2.py
M testdata/bin/wait-for-metastore.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapper.py
M tests/comparison/data_generator_mapred_common.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_no_delete_stats_in_truncate.py
M tests/custom_cluster/test_shared_catalogd.py
M tests/metadata/test_column_unicode.py
M tests/shell/test_beeline.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_commandline_jwt_auth.py
M tests/shell/test_shell_commandline_kerberos_auth.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/stress/concurrent_select.py
M tests/stress/extract_min_mem.py
M tests/stress/mem_broker.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/query_runner.py
M tests/stress/runtime_info.py
M tests/stress/util.py
M tests/util/acid_txn.py
M tests/util/auto_scaler.py
M tests/util/concurrent_workload.py
M tests/util/run_impyla_http_query.py
62 files changed, 52 insertions(+), 321 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/23468/2
-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23468/3/bin/single_node_perf_run.py
File bin/single_node_perf_run.py:

http://gerrit.cloudera.org:8080/#/c/23468/3/bin/single_node_perf_run.py@1
PS3, Line 1: #!/usr/bin/env impala-python3
This hits issues with the perf-AB-test job, so it needs more changes: https://jenkins.impala.io/job/perf-AB-test-ub2004/327/

It is something to do with unicode handling for the JSON files.



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Oct 2025 20:13:32 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] More python3 migration

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: More python3 migration
......................................................................


Patch Set 1: -Verified

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/20670/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Oct 2025 22:33:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 3: Code-Review+1

Thank you for cleaning this up!


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Oct 2025 19:57:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/12592/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 7
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 00:24:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 6:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/20742/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 6
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 00:49:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 7: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23468/5/tests/metadata/test_column_unicode.py
File tests/metadata/test_column_unicode.py:

http://gerrit.cloudera.org:8080/#/c/23468/5/tests/metadata/test_column_unicode.py@a2
PS5, Line 2: 
> Switched back to leaving it in, as it is more consistent with what I did el
Ack



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 7
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 15:29:49 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 4:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/20692/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 15 Oct 2025 18:28:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 4:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23468/3/be/src/codegen/gen_ir_descriptions.py
File be/src/codegen/gen_ir_descriptions.py:

http://gerrit.cloudera.org:8080/#/c/23468/3/be/src/codegen/gen_ir_descriptions.py@1
PS3, Line 1: python
Is it possible to use system python3 here?


http://gerrit.cloudera.org:8080/#/c/23468/3/bin/gen_build_version.py
File bin/gen_build_version.py:

http://gerrit.cloudera.org:8080/#/c/23468/3/bin/gen_build_version.py@1
PS3, Line 1: python
Is it possible to use system python3 here?



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 15 Oct 2025 18:09:37 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] More python3 migration

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: More python3 migration
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Oct 2025 03:49:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 5: Code-Review+1

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23468/5/bin/check-rat-report.py
File bin/check-rat-report.py:

http://gerrit.cloudera.org:8080/#/c/23468/5/bin/check-rat-report.py@1
PS5, Line 1: #!/usr/bin/python3
nit: should we normalize using env?

    /usr/bin/env python3


http://gerrit.cloudera.org:8080/#/c/23468/5/tests/metadata/test_column_unicode.py
File tests/metadata/test_column_unicode.py:

http://gerrit.cloudera.org:8080/#/c/23468/5/tests/metadata/test_column_unicode.py@a2
PS5, Line 2: 
Why remove "coding: utf-8" here but not later files?



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Oct 2025 22:24:33 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/12589/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 5
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Oct 2025 19:30:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 3:

(2 comments)

I did another pass to move locations that use OS python over to using python3.

http://gerrit.cloudera.org:8080/#/c/23468/3/be/src/codegen/gen_ir_descriptions.py
File be/src/codegen/gen_ir_descriptions.py:

http://gerrit.cloudera.org:8080/#/c/23468/3/be/src/codegen/gen_ir_descriptions.py@1
PS3, Line 1: python
> Is it possible to use system python3 here?
Did another pass to move /usr/bin/env python usages to python3.


http://gerrit.cloudera.org:8080/#/c/23468/3/bin/gen_build_version.py
File bin/gen_build_version.py:

http://gerrit.cloudera.org:8080/#/c/23468/3/bin/gen_build_version.py@1
PS3, Line 1: python
> Is it possible to use system python3 here?
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Oct 2025 18:58:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 7: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 7
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 05:09:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23468

to look at the new patch set (#6).

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................

IMPALA-14501: Migrate most scripts from impala-python to impala-python3

To remove the dependency on Python 2, existing scripts need to use
python3 rather than python. These commands find those
locations (for impala-python and regular python):
git grep impala-python | grep -v impala-python3 | grep -v impala-python-common | grep -v init-impala-python
git grep bin/python | grep -v python3

This removes or switches most of these locations by various means:
1. If a python file has a #!/bin/env impala-python (or python) but
   doesn't have a main function, it removes the hash-bang and makes
   sure that the file is not executable.
2. Most scripts can simply switch from impala-python to impala-python3
   (or python to python3) with minimal changes.
3. The cm-api pypi package (which doesn't support Python 3) has been
   replaced by the cm-client pypi package and interfaces have changed.
   Rather than migrating the code (which hasn't been used in years), this
   deletes the old code and stops installing cm-api into the virtualenv.
   The code can be restored and revamped if there is any interest in
   interacting with CM clusters.
4. This switches tests/comparison over to impala-python3, but this code has
   bit-rotted. Some pieces can be run manually, but it can't be fully
   verified with Python 3. It shouldn't hold back the migration on its own.
5. This also replaces locations of impala-python in comments / documentation /
   READMEs.
6. kazoo (used for interacting with HBase) needed to be upgraded to a
   version that supports Python 3. The newest version of kazoo requires
   upgrades of other component versions, so this uses kazoo 2.8.0 to avoid
   needing other upgrades.

The two remaining uses of impala-python are:
 - bin/cmake_aux/create_virtualenv.sh
 - bin/impala-env-versioned-python
These will be removed separately when we drop Python 2 support
completely. In particular, these are useful for testing impala-shell
with Python 2 until we stop supporting Python 2 for impala-shell.

The docker-based tests still use /usr/bin/python, but this can
be switched over independently (and doesn't impact impala-python)

Testing:
 - Ran core job
 - Ran build + dataload on Centos 7, Redhat 8
 - Manual testing of individual scripts (except some bitrotted areas like the
   random query generator)

Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
---
M be/src/codegen/gen_ir_descriptions.py
M bin/bootstrap_system.sh
M bin/bootstrap_toolchain.py
M bin/check-rat-report.py
M bin/collect_minidumps.py
M bin/compare_branches.py
M bin/diagnostics/collect_diagnostics.py
M bin/dump_breakpad_symbols.py
M bin/gen-backend-test-script.py
M bin/gen_build_version.py
M bin/generate_minidump_collection_testdata.py
M bin/generate_xml_config.py
M bin/get_code_size.py
M bin/inline_pom.py
M bin/jenkins/critique-gerrit-review.py
M bin/jenkins/dockerized-impala-preserve-vars.py
M bin/jenkins/populate_m2_directory.py
M bin/jenkins/thrift_parser.py
M bin/load-data.py
M bin/parse-thrift-profile.py
M bin/run-workload.py
M bin/set-pythonpath.sh
M bin/single_node_perf_run.py
M bin/validate-unified-backend-test-filters.py
M common/function-registry/gen_builtins_catalog.py
M common/function-registry/gen_geospatial_udf_wrappers.py
M common/thrift/generate_error_codes.py
M common/thrift/generate_metrics.py
M docker/setup_build_context.py
M fe/src/test/java/org/apache/impala/customcluster/LdapImpylaHttpTest.java
M fe/src/test/resources/hive-site.xml.py
M infra/python/README
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/impala_py_lib/jenkins/junitxml_prune_notrun.py
M lib/python/impala_py_lib/profiles.py
M lib/python/setup.py
M shell/CMakeLists.txt
M shell/impala_shell/compatibility.py
M shell/impala_shell/exec_summary.py
M shell/impala_shell/impala_client.py
M shell/impala_shell/impala_shell.py
M shell/impala_shell/impala_shell_config_defaults.py
M shell/impala_shell/option_parser.py
M shell/impala_shell/shell_output.py
M shell/packaging/make_python_package.sh
M shell/setup.py
M testdata/bin/check-hbase-nodes.py
M testdata/bin/create-load-data.sh
M testdata/bin/create-tpc-jdbc-tables.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load-tpc-kudu.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/bin/wait-for-hiveserver2.py
M testdata/bin/wait-for-metastore.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/ozone-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/comparison/ORACLE.txt
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapper.py
M tests/comparison/data_generator_mapred_common.py
M tests/comparison/discrepancy_searcher.py
M tests/comparison/leopard/controller.py
M tests/comparison/leopard/front_end.py
M tests/comparison/tests/README
M tests/comparison/tests/conftest.py
M tests/comparison/util/verify-oracle-connection.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_no_delete_stats_in_truncate.py
M tests/custom_cluster/test_shared_catalogd.py
M tests/metadata/test_column_unicode.py
M tests/shell/test_beeline.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_commandline_jwt_auth.py
M tests/shell/test_shell_commandline_kerberos_auth.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/stress/concurrent_select.py
M tests/stress/extract_min_mem.py
M tests/stress/mem_broker.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/query_runner.py
M tests/stress/runtime_info.py
M tests/stress/util.py
M tests/util/acid_txn.py
M tests/util/auto_scaler.py
M tests/util/concurrent_workload.py
M tests/util/run_impyla_http_query.py
103 files changed, 94 insertions(+), 376 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/23468/6
-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 6
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/jenkins/critique-gerrit-review.py
File bin/jenkins/critique-gerrit-review.py:

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/jenkins/critique-gerrit-review.py@168
PS8, Line 168:     raise Exception("Did not expect flake8-diff to write to stderr:\n{0}".format(stderr))
> We're now consistently getting a failure here.
Running with verbose output shows

    $ PATH=`pwd`/gerrit_critic_venv/bin:$PATH flake8-diff --standard-flake8-output HEAD~1 HEAD -v
    ...
    2025-10-22 08:48:35,647 flake8diff.flake8 INFO checking bin/load-data.py lines 1, 223, 253
    2025-10-22 08:48:35,742 flake8diff.utils ERROR b"<unknown>:254: SyntaxWarning: invalid escape sequence '\\s'\n"
    ...
    2025-10-22 08:48:37,628 flake8diff.flake8 INFO checking shell/setup.py lines 
    2025-10-22 08:48:37,700 flake8diff.utils ERROR b"<unknown>:96: SyntaxWarning: invalid escape sequence '\\d'\n<unknown>:104: SyntaxWarning: invalid escape sequence '\\d'\n"



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 8
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 15:49:44 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/jenkins/critique-gerrit-review.py
File bin/jenkins/critique-gerrit-review.py:

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/jenkins/critique-gerrit-review.py@168
PS8, Line 168:     raise Exception("Did not expect flake8-diff to write to stderr:\n{0}".format(stderr))
We're now consistently getting a failure here.



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 8
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 15:44:01 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has uploaded a new patch set (#9) to the change originally created by Joe McDonnell. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................

IMPALA-14501: Migrate most scripts from impala-python to impala-python3

To remove the dependency on Python 2, existing scripts need to use
python3 rather than python. These commands find those
locations (for impala-python and regular python):
git grep impala-python | grep -v impala-python3 | grep -v impala-python-common | grep -v init-impala-python
git grep bin/python | grep -v python3

This removes or switches most of these locations by various means:
1. If a python file has a #!/bin/env impala-python (or python) but
   doesn't have a main function, it removes the hash-bang and makes
   sure that the file is not executable.
2. Most scripts can simply switch from impala-python to impala-python3
   (or python to python3) with minimal changes.
3. The cm-api pypi package (which doesn't support Python 3) has been
   replaced by the cm-client pypi package and interfaces have changed.
   Rather than migrating the code (which hasn't been used in years), this
   deletes the old code and stops installing cm-api into the virtualenv.
   The code can be restored and revamped if there is any interest in
   interacting with CM clusters.
4. This switches tests/comparison over to impala-python3, but this code has
   bit-rotted. Some pieces can be run manually, but it can't be fully
   verified with Python 3. It shouldn't hold back the migration on its own.
5. This also replaces locations of impala-python in comments / documentation /
   READMEs.
6. kazoo (used for interacting with HBase) needed to be upgraded to a
   version that supports Python 3. The newest version of kazoo requires
   upgrades of other component versions, so this uses kazoo 2.8.0 to avoid
   needing other upgrades.

The two remaining uses of impala-python are:
 - bin/cmake_aux/create_virtualenv.sh
 - bin/impala-env-versioned-python
These will be removed separately when we drop Python 2 support
completely. In particular, these are useful for testing impala-shell
with Python 2 until we stop supporting Python 2 for impala-shell.

The docker-based tests still use /usr/bin/python, but this can
be switched over independently (and doesn't impact impala-python)

Testing:
 - Ran core job
 - Ran build + dataload on Centos 7, Redhat 8
 - Manual testing of individual scripts (except some bitrotted areas like the
   random query generator)

Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
---
M be/src/codegen/gen_ir_descriptions.py
M bin/bootstrap_system.sh
M bin/bootstrap_toolchain.py
M bin/check-rat-report.py
M bin/collect_minidumps.py
M bin/compare_branches.py
M bin/diagnostics/collect_diagnostics.py
M bin/dump_breakpad_symbols.py
M bin/gen-backend-test-script.py
M bin/gen_build_version.py
M bin/generate_minidump_collection_testdata.py
M bin/generate_xml_config.py
M bin/get_code_size.py
M bin/inline_pom.py
M bin/jenkins/critique-gerrit-review.py
M bin/jenkins/dockerized-impala-preserve-vars.py
M bin/jenkins/populate_m2_directory.py
M bin/jenkins/thrift_parser.py
M bin/load-data.py
M bin/parse-thrift-profile.py
M bin/run-workload.py
M bin/set-pythonpath.sh
M bin/single_node_perf_run.py
M bin/validate-unified-backend-test-filters.py
M common/function-registry/gen_builtins_catalog.py
M common/function-registry/gen_geospatial_udf_wrappers.py
M common/thrift/generate_error_codes.py
M common/thrift/generate_metrics.py
M docker/setup_build_context.py
M fe/src/test/java/org/apache/impala/customcluster/LdapImpylaHttpTest.java
M fe/src/test/resources/hive-site.xml.py
M infra/python/README
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M lib/python/impala_py_lib/jenkins/generate_junitxml.py
M lib/python/impala_py_lib/jenkins/junitxml_prune_notrun.py
M lib/python/impala_py_lib/profiles.py
M lib/python/setup.py
M shell/CMakeLists.txt
M shell/impala_shell/compatibility.py
M shell/impala_shell/exec_summary.py
M shell/impala_shell/impala_client.py
M shell/impala_shell/impala_shell.py
M shell/impala_shell/impala_shell_config_defaults.py
M shell/impala_shell/option_parser.py
M shell/impala_shell/shell_output.py
M shell/packaging/make_python_package.sh
M shell/setup.py
M testdata/bin/check-hbase-nodes.py
M testdata/bin/create-load-data.sh
M testdata/bin/create-tpc-jdbc-tables.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load-tpc-kudu.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/bin/wait-for-hiveserver2.py
M testdata/bin/wait-for-metastore.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/ozone-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/comparison/ORACLE.txt
M tests/comparison/cli_options.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapper.py
M tests/comparison/data_generator_mapred_common.py
M tests/comparison/discrepancy_searcher.py
M tests/comparison/leopard/controller.py
M tests/comparison/leopard/front_end.py
M tests/comparison/tests/README
M tests/comparison/tests/conftest.py
M tests/comparison/util/verify-oracle-connection.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_no_delete_stats_in_truncate.py
M tests/custom_cluster/test_shared_catalogd.py
M tests/metadata/test_column_unicode.py
M tests/shell/test_beeline.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_commandline_jwt_auth.py
M tests/shell/test_shell_commandline_kerberos_auth.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/stress/concurrent_select.py
M tests/stress/extract_min_mem.py
M tests/stress/mem_broker.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/query_runner.py
M tests/stress/runtime_info.py
M tests/stress/util.py
M tests/util/acid_txn.py
M tests/util/auto_scaler.py
M tests/util/concurrent_workload.py
M tests/util/run_impyla_http_query.py
103 files changed, 97 insertions(+), 379 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/23468/9
-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 9
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 8:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/load-data.py
File bin/load-data.py:

http://gerrit.cloudera.org:8080/#/c/23468/8/bin/load-data.py@254
PS8, Line 254:     match = re.search('dataset:\s*([\w\-\.]+)', input_file.read())
Needs to be a regex literal

    r'dataset:...'


http://gerrit.cloudera.org:8080/#/c/23468/8/shell/setup.py
File shell/setup.py:

http://gerrit.cloudera.org:8080/#/c/23468/8/shell/setup.py@96
PS8, Line 96:       version_match = re.search('\d+\.\d+\.\d+', impala_build_version.get_version())
Need to update this and line 104 to regex literals.



-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 8
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 15:53:31 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14501: Migrate most scripts from impala-python to impala-python3

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23468 )

Change subject: IMPALA-14501: Migrate most scripts from impala-python to impala-python3
......................................................................


Patch Set 9:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/20747/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/23468
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If209b761290bc7e7c716c312ea757da3e3bca6dc
Gerrit-Change-Number: 23468
Gerrit-PatchSet: 9
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Oct 2025 16:22:31 +0000
Gerrit-HasComments: No