[jira] [Commented] (HIVE-14778) document threading model of Streaming API

["Hive QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15497739#comment-15497739 ] 

Hive QA commented on HIVE-14778:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12828885/HIVE-14778.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10527 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_auto_mult_tables]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_bitmap_auto_partitioned]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.testMetaDataCounts
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1215/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1215/console
Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1215/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12828885 - PreCommit-HIVE-MASTER-Build

> document threading model of Streaming API
> -----------------------------------------
>
>                 Key: HIVE-14778
>                 URL: https://issues.apache.org/jira/browse/HIVE-14778
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog, Transactions
>    Affects Versions: 0.14.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-14778.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> The model is not obvious and needs to be documented properly.
> A StreamingConnection internally maintains 2 MetaStoreClient objects (each has 1 Thrift client for actual RPC). Let's call them "primary" and "heartbeat". Each TransactionBatch created from a given StreamingConnection, gets a reference to both of these MetaStoreClients. 
> So the model is that there is at most 1 outstanding (not closed) TransactionBatch for any given StreamingConnection and for any given TransactionBatch there can be at most 2 threads accessing it concurrently. 1 thread calling TransactionBatch.heartbeat() (and nothing else) and the other calling all other methods.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)