You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2020/11/03 18:58:00 UTC
[jira] [Updated] (SPARK-33328) Fix Flaky HiveThriftHttpServerSuite
[ https://issues.apache.org/jira/browse/SPARK-33328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-33328:
----------------------------------
Attachment: failure_rate.png
> Fix Flaky HiveThriftHttpServerSuite
> -----------------------------------
>
> Key: SPARK-33328
> URL: https://issues.apache.org/jira/browse/SPARK-33328
> Project: Spark
> Issue Type: Bug
> Components: SQL, Tests
> Affects Versions: 3.1.0
> Reporter: Dongjoon Hyun
> Priority: Major
> Attachments: failure_rate.png
>
>
> After launching successfully `HiveThriftServer2 started successfully`, the test fails due to 500 error.
> - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7-hive-2.3/1516/testReport/
> {code}
> 09:58:03.853 pool-1-thread-1 INFO HiveThriftHttpServerSuite: Trying to start HiveThriftServer2: port=14541, mode=http, attempt=0
> 09:58:06.492 pool-1-thread-1 INFO HiveThriftHttpServerSuite: COMMAND: WrappedArray(../../sbin/start-thriftserver.sh, --master, local, --hiveconf, javax.jdo.option.ConnectionURL=jdbc:derby:;databaseName=/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.7-hive-2.3/target/tmp/spark-6f4abc35-f09c-46e6-b6eb-8a310d557e28;create=true, --hiveconf, hive.metastore.warehouse.dir=/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.7-hive-2.3/target/tmp/spark-2329343a-3ad4-4bfd-943f-6b46984848b8, --hiveconf, hive.server2.thrift.bind.host=localhost, --hiveconf, hive.server2.transport.mode=http, --hiveconf, hive.server2.logging.operation.log.location=/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.7-hive-2.3/target/tmp/spark-77f3b359-1553-40e3-9d75-35c46d2d4d46, --hiveconf, hive.exec.local.scratchdir=/home/jenkins/workspace/spark-master-test-sbt-hadoop-2.7-hive-2.3/target/tmp/spark-8923e61f-36da-4930-b035-6eb3712d41ab, --hiveconf, hive.server2.thrift.http.port=14541, --driver-class-path, /home/jenkins/workspace/spark-master-test-sbt-hadoop-2.7-hive-2.3/target/tmp/spark-d54a7073-2f02-4331-a84d-bbb3b50a47ac, --driver-java-options, -Dlog4j.debug, --conf, spark.ui.enabled=false)
> 09:58:06.492 pool-1-thread-1 INFO HiveThriftHttpServerSuite: OUTPUT: starting org.apache.spark.sql.hive.thriftserver.HiveThriftServer2, logging to /home/jenkins/workspace/spark-master-test-sbt-hadoop-2.7-hive-2.3/logs/spark-jenkins-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-amp-jenkins-worker-06.out
> 09:58:38.688 pool-1-thread-1 INFO HiveThriftHttpServerSuite: HiveThriftServer2 started successfully
> 09:58:38.689 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite INFO HiveThriftHttpServerSuite:
> ===== TEST OUTPUT FOR o.a.s.sql.hive.thriftserver.HiveThriftHttpServerSuite: 'JDBC query execution' =====
> 09:58:38.691 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite INFO Utils: Supplied authorities: localhost:14541
> 09:58:38.691 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite WARN Utils: ***** JDBC param deprecation *****
> 09:58:38.691 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite WARN Utils: The use of hive.server2.transport.mode is deprecated.
> 09:58:38.691 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite WARN Utils: Please use transportMode like so: jdbc:hive2://<host>:<port>/dbName;transportMode=<transport_mode_value>
> 09:58:38.691 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite WARN Utils: ***** JDBC param deprecation *****
> 09:58:38.691 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite WARN Utils: The use of hive.server2.thrift.http.path is deprecated.
> 09:58:38.691 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite WARN Utils: Please use httpPath like so: jdbc:hive2://<host>:<port>/dbName;httpPath=<http_path_value>
> 09:58:38.692 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite INFO Utils: Resolved authority: localhost:14541
> 09:58:38.818 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG RequestAddCookies: CookieSpec selected: default
> 09:58:38.830 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG RequestAuthCache: Auth cache not set in the context
> 09:58:38.832 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG PoolingHttpClientConnectionManager: Connection request: [route: {}->http://localhost:14541][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
> 09:58:38.846 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG PoolingHttpClientConnectionManager: Connection leased: [id: 0][route: {}->http://localhost:14541][total available: 0; route allocated: 1 of 2; total allocated: 1 of 20]
> 09:58:38.847 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG MainClientExec: Opening connection {}->http://localhost:14541
> 09:58:38.850 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG DefaultHttpClientConnectionOperator: Connecting to localhost/127.0.0.1:14541
> 09:58:38.851 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG DefaultHttpClientConnectionOperator: Connection established 127.0.0.1:39668<->127.0.0.1:14541
> 09:58:38.851 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG MainClientExec: Executing request POST /cliservice HTTP/1.1
> 09:58:38.852 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG MainClientExec: Proxy auth state: UNCHALLENGED
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> POST /cliservice HTTP/1.1
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> Content-Type: application/x-thrift
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> Accept: application/x-thrift
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> User-Agent: Java/THttpClient/HC
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> Authorization: Basic amVua2luczphbm9ueW1vdXM=
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> Content-Length: 254
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> Host: localhost:14541
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> Connection: Keep-Alive
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> Accept-Encoding: gzip,deflate
> 09:58:38.854 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 >> X-XSRF-HEADER: true
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "POST /cliservice HTTP/1.1[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "Content-Type: application/x-thrift[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "Accept: application/x-thrift[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "User-Agent: Java/THttpClient/HC[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "Authorization: Basic amVua2luczphbm9ueW1vdXM=[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "Content-Length: 254[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "Host: localhost:14541[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "X-XSRF-HEADER: true[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "[\r][\n]"
> 09:58:38.855 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 >> "[0x80][0x1][0x0][0x1][0x0][0x0][0x0][0xb]OpenSession[0x0][0x0][0x0][0x1][0xc][0x0][0x1][0x8][0x0][0x1][0x0][0x0][0x0][0x9][\r][0x0][0x4][0xb][0xb][0x0][0x0][0x0][0x6][0x0][0x0][0x0][0xe]set:hiveconf:a[0x0][0x0][0x0][0x6]avalue[0x0][0x0][0x0]=set:hiveconf:hive.server2.thrift.resultset.default.fetch.size[0x0][0x0][0x0][0x4]1000[0x0][0x0][0x0][\r]set:hivevar:c[0x0][0x0][0x0][0x6]cvalue[0x0][0x0][0x0][0xe]set:hiveconf:b[0x0][0x0][0x0][0x6]bvalue[0x0][0x0][0x0][\r]set:hivevar:d[0x0][0x0][0x0][0x6]dvalue[0x0][0x0][0x0][0xc]use:database[0x0][0x0][0x0][0x7]default[0x0][0x0]"
> 09:58:39.020 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 << "HTTP/1.1 500 Server Error[\r][\n]"
> 09:58:39.020 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]"
> 09:58:39.020 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 << "Connection: close[\r][\n]"
> 09:58:39.020 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 << "Server: Jetty(9.4.28.v20200408)[\r][\n]"
> 09:58:39.020 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG wire: http-outgoing-0 << "[\r][\n]"
> 09:58:39.023 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 << HTTP/1.1 500 Server Error
> 09:58:39.023 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 << Cache-Control: must-revalidate,no-cache,no-store
> 09:58:39.023 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 << Connection: close
> 09:58:39.023 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG headers: http-outgoing-0 << Server: Jetty(9.4.28.v20200408)
> 09:58:39.028 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG DefaultManagedHttpClientConnection: http-outgoing-0: Close connection
> 09:58:39.028 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG MainClientExec: Connection discarded
> 09:58:39.028 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG PoolingHttpClientConnectionManager: Connection released: [id: 0][route: {}->http://localhost:14541][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
> 09:58:39.028 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite DEBUG MainClientExec: Cancelling request execution
> 09:58:39.029 pool-1-thread-1-ScalaTest-running-HiveThriftHttpServerSuite ERROR HiveConnection: Error opening session
> org.apache.thrift.transport.TTransportException: HTTP Response code: 500
> at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:262)
> at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:316)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
> at org.apache.hive.service.rpc.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:162)
> at org.apache.hive.service.rpc.thrift.TCLIService$Client.OpenSession(TCLIService.java:154)
> at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:680)
> at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:200)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> at java.sql.DriverManager.getConnection(DriverManager.java:664)
> at java.sql.DriverManager.getConnection(DriverManager.java:247)
> at org.apache.spark.sql.hive.thriftserver.HiveThriftJdbcTest.$anonfun$withMultipleConnectionJdbcStatement$1(HiveThriftServer2Suites.scala:1141)
> at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
> at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
> at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
> at scala.collection.TraversableLike.map(TraversableLike.scala:238)
> at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
> at scala.collection.AbstractTraversable.map(Traversable.scala:108)
> at org.apache.spark.sql.hive.thriftserver.HiveThriftJdbcTest.withMultipleConnectionJdbcStatement(HiveThriftServer2Suites.scala:1141)
> at org.apache.spark.sql.hive.thriftserver.HiveThriftJdbcTest.withJdbcStatement(HiveThriftServer2Suites.scala:1177)
> at org.apache.spark.sql.hive.thriftserver.HiveThriftHttpServerSuite.$anonfun$new$119(HiveThriftServer2Suites.scala:1086)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org