You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sergey Shelukhin <se...@hortonworks.com> on 2018/06/23 00:29:16 UTC

Review Request 67712: HIVE-19820 add ACID stats support to background stats updater

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67712/
-----------------------------------------------------------

Review request for hive, Eugene Koifman and Seong (Steve) Yeom.


Repository: hive-git


Description
-------

see jira


Diffs
-----

  itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java 580bae9c3f1307325842a08275e085a8e31f9351 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java ddca70497a3f51c3ec9ea532fac2a42aa36149b3 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java dd0929f2b9748d83d55ccc271cec6aa07933bde1 
  ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUpdaterThread.java 14f86eabbcf4bfc38c92294cd5d71d4905eb5c30 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4296084381df1e109248820b96739a4eb5ee0490 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 51e081b22fa27b013715bb6eddf7fbbcf6bbd061 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java 9266879ad0134dbf87598af6f9305b73cc8c40ba 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java 8cc9d2c586a411712d01d599ff2986f6ad5e0cfd 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java e4894fa12bfee78f51f3796e0ccaaf51c7ac4136 
  standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 001c3edcff5a4d0ea67b73e83075b1f867342654 
  standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java d6a882e8e98f92eefbdb7900bdf43e3274a21c5d 
  standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/VerifyingObjectStore.java c9a6a471cb7fc28845efb6d774601dba0cef2a85 


Diff: https://reviews.apache.org/r/67712/diff/1/


Testing
-------


Thanks,

Sergey Shelukhin


Re: Review Request 67712: HIVE-19820 add ACID stats support to background stats updater

Posted by Sergey Shelukhin <se...@hortonworks.com>.

> On June 27, 2018, 4:23 p.m., Eugene Koifman wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java
> > Lines 291 (patched)
> > <https://reviews.apache.org/r/67712/diff/1/?file=2044531#file2044531line291>
> >
> >     there are several read ops in this txn - what semantics is the txn trying to achive here?

Added a comment


- Sergey


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67712/#review205443
-----------------------------------------------------------


On June 23, 2018, 12:29 a.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67712/
> -----------------------------------------------------------
> 
> (Updated June 23, 2018, 12:29 a.m.)
> 
> 
> Review request for hive, Eugene Koifman and Seong (Steve) Yeom.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java 580bae9c3f1307325842a08275e085a8e31f9351 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java ddca70497a3f51c3ec9ea532fac2a42aa36149b3 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java dd0929f2b9748d83d55ccc271cec6aa07933bde1 
>   ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUpdaterThread.java 14f86eabbcf4bfc38c92294cd5d71d4905eb5c30 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4296084381df1e109248820b96739a4eb5ee0490 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 51e081b22fa27b013715bb6eddf7fbbcf6bbd061 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java 9266879ad0134dbf87598af6f9305b73cc8c40ba 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java 8cc9d2c586a411712d01d599ff2986f6ad5e0cfd 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java e4894fa12bfee78f51f3796e0ccaaf51c7ac4136 
>   standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 001c3edcff5a4d0ea67b73e83075b1f867342654 
>   standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java d6a882e8e98f92eefbdb7900bdf43e3274a21c5d 
>   standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/VerifyingObjectStore.java c9a6a471cb7fc28845efb6d774601dba0cef2a85 
> 
> 
> Diff: https://reviews.apache.org/r/67712/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Re: Review Request 67712: HIVE-19820 add ACID stats support to background stats updater

Posted by Eugene Koifman <ek...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67712/#review205443
-----------------------------------------------------------




itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java
Line 424 (original), 424 (patched)
<https://reviews.apache.org/r/67712/#comment288363>

    arg4? arg5? is this decompiled code?



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java
Lines 291 (patched)
<https://reviews.apache.org/r/67712/#comment288365>

    there are several read ops in this txn - what semantics is the txn trying to achive here?



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java
Line 296 (original), 324 (patched)
<https://reviews.apache.org/r/67712/#comment288366>

    0 is not a valid transaction id



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java
Line 412 (original), 440 (patched)
<https://reviews.apache.org/r/67712/#comment288367>

    0 is not a valid txn id


- Eugene Koifman


On June 22, 2018, 5:29 p.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67712/
> -----------------------------------------------------------
> 
> (Updated June 22, 2018, 5:29 p.m.)
> 
> 
> Review request for hive, Eugene Koifman and Seong (Steve) Yeom.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java 580bae9c3f1307325842a08275e085a8e31f9351 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java ddca70497a3f51c3ec9ea532fac2a42aa36149b3 
>   ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java dd0929f2b9748d83d55ccc271cec6aa07933bde1 
>   ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUpdaterThread.java 14f86eabbcf4bfc38c92294cd5d71d4905eb5c30 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 4296084381df1e109248820b96739a4eb5ee0490 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 51e081b22fa27b013715bb6eddf7fbbcf6bbd061 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java 9266879ad0134dbf87598af6f9305b73cc8c40ba 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java 8cc9d2c586a411712d01d599ff2986f6ad5e0cfd 
>   standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java e4894fa12bfee78f51f3796e0ccaaf51c7ac4136 
>   standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 001c3edcff5a4d0ea67b73e83075b1f867342654 
>   standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java d6a882e8e98f92eefbdb7900bdf43e3274a21c5d 
>   standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/VerifyingObjectStore.java c9a6a471cb7fc28845efb6d774601dba0cef2a85 
> 
> 
> Diff: https://reviews.apache.org/r/67712/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Re: Review Request 67712: HIVE-19820 add ACID stats support to background stats updater

Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67712/
-----------------------------------------------------------

(Updated June 27, 2018, 7:55 p.m.)


Review request for hive, Eugene Koifman and Seong (Steve) Yeom.


Repository: hive-git


Description
-------

see jira


Diffs (updated)
-----

  itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java 580bae9c3f 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUpdaterThread.java ddca70497a 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java dd0929f2b9 
  ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUpdaterThread.java 14f86eabbc 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 706d831435 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 51e081b22f 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java 2c3554edc4 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/RawStore.java 8cc9d2c586 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java e4894fa12b 
  standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 001c3edcff 
  standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java d6a882e8e9 
  standalone-metastore/src/test/java/org/apache/hadoop/hive/metastore/VerifyingObjectStore.java c9a6a471cb 


Diff: https://reviews.apache.org/r/67712/diff/2/

Changes: https://reviews.apache.org/r/67712/diff/1-2/


Testing
-------


Thanks,

Sergey Shelukhin