You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (JIRA)" <ji...@apache.org> on 2018/06/22 06:50:00 UTC
[jira] [Commented] (HIVE-19950) Hive ACID NOT LOCK LockComponent
Correctly
[ https://issues.apache.org/jira/browse/HIVE-19950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16520052#comment-16520052 ]
Hive QA commented on HIVE-19950:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 3s{color} | {color:blue} standalone-metastore in master has 227 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 20s{color} | {color:red} standalone-metastore: The patch generated 4 new + 531 unchanged - 11 fixed = 535 total (was 542) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 16s{color} | {color:red} standalone-metastore generated 1 new + 227 unchanged - 0 fixed = 228 total (was 227) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 36s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore |
| | Private method org.apache.hadoop.hive.metastore.txn.TxnHandler.shouldNeverHappen(long, long, long) is never called At TxnHandler.java:is never called At TxnHandler.java:[lines 2497-2498] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-11989/dev-support/hive-personality.sh |
| git revision | master / 24e16cc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-11989/yetus/diff-checkstyle-standalone-metastore.txt |
| findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-11989/yetus/new-findbugs-standalone-metastore.html |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-11989/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> Hive ACID NOT LOCK LockComponent Correctly
> ------------------------------------------
>
> Key: HIVE-19950
> URL: https://issues.apache.org/jira/browse/HIVE-19950
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 2.3.2
> Reporter: nickSoul
> Priority: Blocker
> Attachments: patch.patch
>
>
> Hi,
> When using Streaming Mutation recently, I found LockComponents were not locked correctly by current transaction. Below is my test case:
> Step1: Begin a transaction with transactionId 126, and the transaction locks a table. Then hangs the transaction. The lock information were correctly restored in mariaDB
> {code:java}
> MariaDB [hive]> select HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID from HIVE_LOCKS;
> +----------------+----------------+----------+-----------+-----------+----
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +----------------+----------------+----------+-----------+-----------+----
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL | NULL |
> +----------------+----------------+----------+-----------+-----------+---- {code}
>
> Step2: Begin the other transaction with a transactionId 127 before previous transaction 126 finished. Transaction 127 tries to lock the same table too, but failed at first attempt. The lock information were correctly restored in mariaDB, Lock 385 was blocked by Lock 384.
> {code:java}
> MariaDB [hive]> select HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID from HIVE_LOCKS;
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> | 384 | 1 | 126 | test_acid | acid_test | NULL | a | w | 1529512857000 | NULL | NULL |
> | 385 | 1 | 127 | test_acid | acid_test | NULL | w | w | NULL | 384 | 1 |
> +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+{code}
>
> Step3: Then transaction 127 tries to lock the table for a second retry after 30s with another lockId: 386, this time it successfully locked the table, whereas transaction 126 is still holding the lock. Lock informations in MetaStore DB:
> {code:java}
> MariaDB [hive]> select HL_LOCK_EXT_ID,HL_LOCK_INT_ID,HL_TXNID,HL_DB,HL_TABLE,HL_PARTITION,HL_LOCK_STATE,HL_LOCK_TYPE,HL_ACQUIRED_AT,HL_BLOCKEDBY_EXT_ID,HL_BLOCKEDBY_INT_ID from HIVE_LOCKS; +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+ | HL_LOCK_EXT_ID | HL_LOCK_INT_ID | HL_TXNID | HL_DB | HL_TABLE | HL_PARTITION | HL_LOCK_STATE | HL_LOCK_TYPE | HL_ACQUIRED_AT | HL_BLOCKEDBY_EXT_ID | HL_BLOCKEDBY_INT_ID | +----------------+----------------+----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> |384| 1 |126| test_acid | acid_test | NULL | a | w | 1529512857000 | NULL | NULL |
> |385| 1 |127| test_acid |acid_test| NULL| w | w| NULL|384 | 1 |
> |386| 1 |127| test_acid | acid_test | NULL | a | w | 1529513069000 | NULL | NULL |
> +----------+-----------+-----------+--------------+---------------+--------------+----------------+---------------------+---------------------+
> {code}
> {code:java}
> {code}
> After going through the code. I found it dosen't care about other transaction's lock on the LockComponents in second retry.
> I wonder if i use it in a wrong way, or misunderstand sth about ACID in hive.
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)