You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Laszlo Pinter via Review Board <no...@reviews.apache.org> on 2019/10/24 15:57:14 UTC
Review Request 71671: HIVE-22401: Refactor CompactorMR
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71671/
-----------------------------------------------------------
Review request for hive, Marta Kuczora, Peter Vary, and Adam Szita.
Repository: hive-git
Description
-------
HIVE-22401: Refactor CompactorMR
At the moment CompactorMR contains all the methods which handles compaction. Since compaction can also be triggered as hive queries not only as MR jobs, it makes sense to move out these methods to a separate class, to avoid confusion.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 0f1579aa542f83b68f2efc92e08e6c0a32bd113d
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/InsertOnlyMajorQueryCompactor.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java PRE-CREATION
Diff: https://reviews.apache.org/r/71671/diff/1/
Testing
-------
Thanks,
Laszlo Pinter
Re: Review Request 71671: HIVE-22401: Refactor CompactorMR
Posted by Laszlo Pinter via Review Board <no...@reviews.apache.org>.
> On Oct. 28, 2019, 5:44 p.m., Peter Vary wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java
> > Lines 85-87 (patched)
> > <https://reviews.apache.org/r/71671/diff/3/?file=2170351#file2170351line85>
> >
> > Maybe this should be checked outside? This is something general? Or am I mistaken?
In my next change, related to query based minor compaction, I will refactor this part as well.
- Laszlo
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71671/#review218420
-----------------------------------------------------------
On Oct. 24, 2019, 3:57 p.m., Laszlo Pinter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71671/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2019, 3:57 p.m.)
>
>
> Review request for hive, Marta Kuczora, Peter Vary, and Adam Szita.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-22401: Refactor CompactorMR
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 0f1579aa542f83b68f2efc92e08e6c0a32bd113d
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/71671/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Laszlo Pinter
>
>
Re: Review Request 71671: HIVE-22401: Refactor CompactorMR
Posted by Peter Vary via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71671/#review218420
-----------------------------------------------------------
Thanks for the patch.
A nit and a question.
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java
Line 239 (original), 220 (patched)
<https://reviews.apache.org/r/71671/#comment306127>
Could we remove the extra spaces if we are here, please?
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java
Lines 85-87 (patched)
<https://reviews.apache.org/r/71671/#comment306128>
Maybe this should be checked outside? This is something general? Or am I mistaken?
- Peter Vary
On okt. 24, 2019, 3:57 du, Laszlo Pinter wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71671/
> -----------------------------------------------------------
>
> (Updated okt. 24, 2019, 3:57 du)
>
>
> Review request for hive, Marta Kuczora, Peter Vary, and Adam Szita.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-22401: Refactor CompactorMR
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 0f1579aa542f83b68f2efc92e08e6c0a32bd113d
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MajorQueryCompactor.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MmMajorQueryCompactor.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactor.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/QueryCompactorFactory.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/71671/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Laszlo Pinter
>
>