You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/03/13 13:02:00 UTC
[jira] [Commented] (IMPALA-10562)
TestGracefulShutdown::test_shutdown_idle failes in GCE instance
[ https://issues.apache.org/jira/browse/IMPALA-10562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300853#comment-17300853 ]
ASF subversion and git services commented on IMPALA-10562:
----------------------------------------------------------
Commit 2dfc68d85277f05bf20c09e31dd10c9474ada62c in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2dfc68d ]
IMPALA-7712: Support Google Cloud Storage
This patch adds support for GCS(Google Cloud Storage). Using the
gcs-connector, the implementation is similar to other remote
FileSystems.
New flags for GCS:
- num_gcs_io_threads: Number of GCS I/O threads. Defaults to be 16.
Follow-up:
- Support for spilling to GCS will be addressed in IMPALA-10561.
- Support for caching GCS file handles will be addressed in
IMPALA-10568.
- test_concurrent_inserts and test_failing_inserts in
test_acid_stress.py are skipped due to slow file listing on
GCS (IMPALA-10562).
- Some tests are skipped due to issues introduced by /etc/hosts setting
on GCE instances (IMPALA-10563).
Tests:
- Compile and create hdfs test data on a GCE instance. Upload test data
to a GCS bucket. Modify all locations in HMS DB to point to the GCS
bucket. Remove some hdfs caching params. Run CORE tests.
- Compile and load snapshot data to a GCS bucket. Run CORE tests.
Change-Id: Ia91ec956de3b620cccf6a1244b56b7da7a45b32b
Reviewed-on: http://gerrit.cloudera.org:8080/17121
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> TestGracefulShutdown::test_shutdown_idle failes in GCE instance
> ---------------------------------------------------------------
>
> Key: IMPALA-10562
> URL: https://issues.apache.org/jira/browse/IMPALA-10562
> Project: IMPALA
> Issue Type: Test
> Reporter: Quanlong Huang
> Priority: Major
>
> Google Cloud will add a line in /etc/hosts for resolving the hostname of the instance. E.g.
> {code:java}
> [systest@quanlong-compile impala]$ cat /etc/hosts
> 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
> ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
> x.x.x.x quanlong-compile.gce.cloudera.com quanlong-compile # Added by Google
> x.x.x.x metadata.google.internal # Added by Google
> 127.0.0.1 quanlong-compile quanlong-compile.gce.cloudera.com
> {code}
> (The IPs are manually masked)
> This causes connection refused failure in TestGracefulShutdown::test_shutdown_idle
> {code:java}
> tests/custom_cluster/test_restart_services.py:295: in test_shutdown_idle
> query_options={})
> tests/common/impala_test_suite.py:814: in wrapper
> return function(*args, **kwargs)
> tests/common/impala_test_suite.py:822: in execute_query_expect_success
> result = cls.__execute_query(impalad_client, query, query_options, user)
> tests/common/impala_test_suite.py:923: in __execute_query
> return impalad_client.execute(query, user=user)
> tests/common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> tests/beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> tests/beeswax/impala_beeswax.py:363: in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> tests/beeswax/impala_beeswax.py:357: in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> tests/beeswax/impala_beeswax.py:520: in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
> E MESSAGE: Rpc to 127.0.0.1:27001 failed with error 'RemoteShutdown() RPC failed: Network error: Client connection negotiation failed: client connection to 127.0.0.1:27001: connect: Connection refused (error 111)'
> {code}
> The failed command is "{{:shutdown('localhost:27001')}}".
> Checked with the impalad, it's listening on "quanlong-compile.gce.cloudera.com:27001":
> {code:java}
> [systest@quanlong-compile impala]$ ps aux | grep 28771
> systest 28771 11.0 3.5 22579424 2165200 pts/1 Sl 23:28 0:15 /home/systest/impala/be/build/latest/service/impalad -disconnected_session_timeout 21600 -kudu_client_rpc_timeout_ms 0 -kudu_master_hosts localhost -mem_limit=12884
> 901888 -logbufsecs=5 -v=1 -max_log_files=10 -log_filename=impalad_node1 -log_dir=/tmp/ -beeswax_port=21001 -hs2_port=21051 -hs2_http_port=28001 -krpc_port=27001 -state_store_subscriber_port=23001 -webserver_port=25001 --shutdown_
> grace_period_s=1 --hostname=quanlong-compile.gce.cloudera.com --default_query_options=
> systest 30727 0.0 0.0 112828 1032 pts/1 S+ 23:31 0:00 grep --color=auto 28771
> [systest@quanlong-compile impala]$ lsof -p 28771 | grep 27001
> impalad 28771 systest 341u IPv4 577338 0t0 TCP quanlong-compile.gce.cloudera.com:27001 (LISTEN)
> {code}
> The shutdown command works with hostname:
> {code:java}
> [localhost.localdomain:21050] default> :shutdown('localhost:27001');
> Query: :shutdown('localhost:27001')
> Query submitted at: 2021-03-07 23:39:25 (Coordinator: http://quanlong-compile.gce.cloudera.com:25000)
> ERROR: Rpc to 127.0.0.1:27001 failed with error 'RemoteShutdown() RPC failed: Network error: Client connection negotiation failed: client connection to 127.0.0.1:27001: connect: Connection refused (error 111)'
> [localhost.localdomain:21050] default> :shutdown('quanlong-compile.gce.cloudera.com:27001');
> Query: :shutdown('quanlong-compile.gce.cloudera.com:27001')
> Query submitted at: 2021-03-07 23:41:35 (Coordinator: http://quanlong-compile.gce.cloudera.com:25000)
> Query progress can be monitored at: http://quanlong-compile.gce.cloudera.com:25000/query_plan?query_id=81475574dca0db03:6e2f767d00000000
> +-------------------------------------------------------------------------------------------------------------------------------------------+
> | summary |
> +-------------------------------------------------------------------------------------------------------------------------------------------+
> | shutdown grace period left: 1s000ms, deadline left: 1h, queries registered on coordinator: 0, queries executing: 0, fragment instances: 0 |
> +-------------------------------------------------------------------------------------------------------------------------------------------+
> Fetched 1 row(s) in 0.01s
> {code}
> Removing the line "x.x.x.x quanlong-compile.gce.cloudera.com quanlong-compile # Added by Google" can pass the test. Maybe we should add the logics in bin/bootstrap_system.sh when running in GCE instance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org