You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@falcon.apache.org by pavan kumar kolamuri <pa...@gmail.com> on 2015/10/08 07:01:48 UTC

Review Request 39115: Retention support for Falcon Unit

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39115/
-----------------------------------------------------------

Review request for Falcon.


Bugs: FALCON-1516
    https://issues.apache.org/jira/browse/FALCON-1516


Repository: falcon-git


Description
-------

As of now Falcon Unit only supports Process execution , Adding support for Retention support for Falcon Unit useful to simplify integration tests in future.


Diffs
-----

  client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 265e08c 
  common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
  prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6 
  unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b 
  unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java PRE-CREATION 
  unit/src/main/java/org/apache/falcon/unit/LocalSchedulableEntityManager.java PRE-CREATION 
  unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 995af2b 
  unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java fa9c664 
  unit/src/test/resources/infeed.xml 509d868 

Diff: https://reviews.apache.org/r/39115/diff/


Testing
-------

Added Unit Tests


Thanks,

pavan kumar kolamuri

Re: Review Request 39115: Retention support for Falcon Unit

Posted by pavan kumar kolamuri <pa...@gmail.com>.


> On Oct. 8, 2015, 6:04 a.m., sandeep samudrala wrote:
> > unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java, line 354
> > <https://reviews.apache.org/r/39115/diff/1/?file=1092735#file1092735line354>
> >
> >     This logic of hardcoding is also in eviction code. Can this be extracted to a method?

In eviction it needs a string for for coord frequency and here it needs time. I don't think we can generalize it . Pls suggest if there is something we can do


- pavan kumar


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39115/#review101872
-----------------------------------------------------------


On Oct. 8, 2015, 9:55 a.m., pavan kumar kolamuri wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39115/
> -----------------------------------------------------------
> 
> (Updated Oct. 8, 2015, 9:55 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1516
>     https://issues.apache.org/jira/browse/FALCON-1516
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> As of now Falcon Unit only supports Process execution , Adding support for Retention support for Falcon Unit useful to simplify integration tests in future.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 265e08c 
>   client/src/main/java/org/apache/falcon/client/FalconClient.java 20f6447 
>   common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
>   prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b 
>   unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/LocalSchedulableEntityManager.java PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 995af2b 
>   unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java fa9c664 
>   unit/src/test/resources/infeed.xml 509d868 
> 
> Diff: https://reviews.apache.org/r/39115/diff/
> 
> 
> Testing
> -------
> 
> Added Unit Tests
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 39115: Retention support for Falcon Unit

Posted by sandeep samudrala <sa...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39115/#review101872
-----------------------------------------------------------



client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java (line 79)
<https://reviews.apache.org/r/39115/#comment159371>

    formatting



unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java (line 354)
<https://reviews.apache.org/r/39115/#comment159374>

    This logic of hardcoding is also in eviction code. Can this be extracted to a method?


- sandeep samudrala


On Oct. 8, 2015, 5:01 a.m., pavan kumar kolamuri wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39115/
> -----------------------------------------------------------
> 
> (Updated Oct. 8, 2015, 5:01 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1516
>     https://issues.apache.org/jira/browse/FALCON-1516
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> As of now Falcon Unit only supports Process execution , Adding support for Retention support for Falcon Unit useful to simplify integration tests in future.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 265e08c 
>   common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
>   prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b 
>   unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/LocalSchedulableEntityManager.java PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 995af2b 
>   unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java fa9c664 
>   unit/src/test/resources/infeed.xml 509d868 
> 
> Diff: https://reviews.apache.org/r/39115/diff/
> 
> 
> Testing
> -------
> 
> Added Unit Tests
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 39115: Retention support for Falcon Unit

Posted by pavan kumar kolamuri <pa...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39115/#review101880
-----------------------------------------------------------



client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java (line 79)
<https://reviews.apache.org/r/39115/#comment159380>

    will fix


- pavan kumar kolamuri


On Oct. 8, 2015, 5:01 a.m., pavan kumar kolamuri wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39115/
> -----------------------------------------------------------
> 
> (Updated Oct. 8, 2015, 5:01 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1516
>     https://issues.apache.org/jira/browse/FALCON-1516
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> As of now Falcon Unit only supports Process execution , Adding support for Retention support for Falcon Unit useful to simplify integration tests in future.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 265e08c 
>   common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
>   prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b 
>   unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/LocalSchedulableEntityManager.java PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 995af2b 
>   unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java fa9c664 
>   unit/src/test/resources/infeed.xml 509d868 
> 
> Diff: https://reviews.apache.org/r/39115/diff/
> 
> 
> Testing
> -------
> 
> Added Unit Tests
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 39115: Retention support for Falcon Unit

Posted by pavan kumar kolamuri <pa...@gmail.com>.


> On Oct. 8, 2015, 5:52 a.m., Pallavi Rao wrote:
> > unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java, line 133
> > <https://reviews.apache.org/r/39115/diff/1/?file=1092732#file1092732line133>
> >
> >     With this new method, shouldn't we merge the older getInstanceStatus method with this and use that through out, rather than having 2 similar methods?

Yes i want to refactor existing api's in another when i add other entity api's in another jira


- pavan kumar


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39115/#review101868
-----------------------------------------------------------


On Oct. 8, 2015, 5:01 a.m., pavan kumar kolamuri wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39115/
> -----------------------------------------------------------
> 
> (Updated Oct. 8, 2015, 5:01 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1516
>     https://issues.apache.org/jira/browse/FALCON-1516
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> As of now Falcon Unit only supports Process execution , Adding support for Retention support for Falcon Unit useful to simplify integration tests in future.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 265e08c 
>   common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
>   prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b 
>   unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/LocalSchedulableEntityManager.java PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 995af2b 
>   unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java fa9c664 
>   unit/src/test/resources/infeed.xml 509d868 
> 
> Diff: https://reviews.apache.org/r/39115/diff/
> 
> 
> Testing
> -------
> 
> Added Unit Tests
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 39115: Retention support for Falcon Unit

Posted by Pallavi Rao <pa...@inmobi.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39115/#review101868
-----------------------------------------------------------



unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java (line 133)
<https://reviews.apache.org/r/39115/#comment159370>

    With this new method, shouldn't we merge the older getInstanceStatus method with this and use that through out, rather than having 2 similar methods?


- Pallavi Rao


On Oct. 8, 2015, 5:01 a.m., pavan kumar kolamuri wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39115/
> -----------------------------------------------------------
> 
> (Updated Oct. 8, 2015, 5:01 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1516
>     https://issues.apache.org/jira/browse/FALCON-1516
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> As of now Falcon Unit only supports Process execution , Adding support for Retention support for Falcon Unit useful to simplify integration tests in future.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 265e08c 
>   common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
>   prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b 
>   unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/LocalSchedulableEntityManager.java PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 995af2b 
>   unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java fa9c664 
>   unit/src/test/resources/infeed.xml 509d868 
> 
> Diff: https://reviews.apache.org/r/39115/diff/
> 
> 
> Testing
> -------
> 
> Added Unit Tests
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 39115: Retention support for Falcon Unit

Posted by Pallavi Rao <pa...@inmobi.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39115/#review101894
-----------------------------------------------------------

Ship it!


Assuming that the review comment of code refactor will be fixed in FALCON-1519.

- Pallavi Rao


On Oct. 8, 2015, 9:55 a.m., pavan kumar kolamuri wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39115/
> -----------------------------------------------------------
> 
> (Updated Oct. 8, 2015, 9:55 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1516
>     https://issues.apache.org/jira/browse/FALCON-1516
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> As of now Falcon Unit only supports Process execution , Adding support for Retention support for Falcon Unit useful to simplify integration tests in future.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 265e08c 
>   client/src/main/java/org/apache/falcon/client/FalconClient.java 20f6447 
>   common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
>   prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6 
>   unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b 
>   unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java PRE-CREATION 
>   unit/src/main/java/org/apache/falcon/unit/LocalSchedulableEntityManager.java PRE-CREATION 
>   unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 995af2b 
>   unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java fa9c664 
>   unit/src/test/resources/infeed.xml 509d868 
> 
> Diff: https://reviews.apache.org/r/39115/diff/
> 
> 
> Testing
> -------
> 
> Added Unit Tests
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 39115: Retention support for Falcon Unit

Posted by pavan kumar kolamuri <pa...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39115/
-----------------------------------------------------------

(Updated Oct. 8, 2015, 9:55 a.m.)


Review request for Falcon.


Changes
-------

Formatting changes done


Bugs: FALCON-1516
    https://issues.apache.org/jira/browse/FALCON-1516


Repository: falcon-git


Description
-------

As of now Falcon Unit only supports Process execution , Adding support for Retention support for Falcon Unit useful to simplify integration tests in future.


Diffs (updated)
-----

  client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 265e08c 
  client/src/main/java/org/apache/falcon/client/FalconClient.java 20f6447 
  common/src/main/java/org/apache/falcon/util/DateUtil.java f89ef64 
  prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6 
  unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b 
  unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java PRE-CREATION 
  unit/src/main/java/org/apache/falcon/unit/LocalSchedulableEntityManager.java PRE-CREATION 
  unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 995af2b 
  unit/src/test/java/org/apache/falcon/unit/TestFalconUnit.java fa9c664 
  unit/src/test/resources/infeed.xml 509d868 

Diff: https://reviews.apache.org/r/39115/diff/


Testing
-------

Added Unit Tests


Thanks,

pavan kumar kolamuri