You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Dmitriy Ryaboy <dv...@gmail.com> on 2011/07/04 06:45:47 UTC
Re: Review Request: PIG-2142: Allow registering multiple jars from DFS via
single statement
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/976/
-----------------------------------------------------------
(Updated 2011-07-04 04:45:47.639319)
Review request for pig and Daniel Dai.
Changes
-------
Here's the actual patch.
Summary
-------
Posting on behalf of Raghu.
Daniel, can you take a look since this collides directly with your changes in PIG-1566?
Argh, reviewboard apparently does not get git diffs. We'll try to make something it gets and add later.. please see the patch on the Jira in the meantime.
This addresses bug PIG-2142.
https://issues.apache.org/jira/browse/PIG-2142
Diffs (updated)
-----
trunk/src/org/apache/pig/PigServer.java 1142512
trunk/src/org/apache/pig/impl/io/FileLocalizer.java 1142512
trunk/test/org/apache/pig/test/TestPigServer.java 1142512
Diff: https://reviews.apache.org/r/976/diff
Testing
-------
Thanks,
Dmitriy
Re: Review Request: PIG-2142: Allow registering multiple jars from DFS via
single statement
Posted by Raghu Angadi <ra...@apache.org>.
> On 2011-07-07 01:16:22, Daniel Dai wrote:
> > trunk/src/org/apache/pig/impl/io/FileLocalizer.java, line 796
> > <https://reviews.apache.org/r/976/diff/1/?file=21967#file21967line796>
> >
> > Is it possible to avoid a listStatus call if we are not use globbing?
globStatus() does the check. The check itself is pretty involved. If a path does not have a glob, it eventually results in only one call to getFileStatus().
- Raghu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/976/#review983
-----------------------------------------------------------
On 2011-07-04 04:45:47, Dmitriy Ryaboy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/976/
> -----------------------------------------------------------
>
> (Updated 2011-07-04 04:45:47)
>
>
> Review request for pig and Daniel Dai.
>
>
> Summary
> -------
>
> Posting on behalf of Raghu.
> Daniel, can you take a look since this collides directly with your changes in PIG-1566?
>
> Argh, reviewboard apparently does not get git diffs. We'll try to make something it gets and add later.. please see the patch on the Jira in the meantime.
>
>
> This addresses bug PIG-2142.
> https://issues.apache.org/jira/browse/PIG-2142
>
>
> Diffs
> -----
>
> trunk/src/org/apache/pig/PigServer.java 1142512
> trunk/src/org/apache/pig/impl/io/FileLocalizer.java 1142512
> trunk/test/org/apache/pig/test/TestPigServer.java 1142512
>
> Diff: https://reviews.apache.org/r/976/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dmitriy
>
>
Re: Review Request: PIG-2142: Allow registering multiple jars from DFS via
single statement
Posted by Daniel Dai <da...@gmail.com>.
> On 2011-07-07 01:16:22, Daniel Dai wrote:
> > trunk/src/org/apache/pig/impl/io/FileLocalizer.java, line 796
> > <https://reviews.apache.org/r/976/diff/1/?file=21967#file21967line796>
> >
> > Is it possible to avoid a listStatus call if we are not use globbing?
>
> Raghu Angadi wrote:
> globStatus() does the check. The check itself is pretty involved. If a path does not have a glob, it eventually results in only one call to getFileStatus().
Does globStatus check it on local side?
- Daniel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/976/#review983
-----------------------------------------------------------
On 2011-07-04 04:45:47, Dmitriy Ryaboy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/976/
> -----------------------------------------------------------
>
> (Updated 2011-07-04 04:45:47)
>
>
> Review request for pig and Daniel Dai.
>
>
> Summary
> -------
>
> Posting on behalf of Raghu.
> Daniel, can you take a look since this collides directly with your changes in PIG-1566?
>
> Argh, reviewboard apparently does not get git diffs. We'll try to make something it gets and add later.. please see the patch on the Jira in the meantime.
>
>
> This addresses bug PIG-2142.
> https://issues.apache.org/jira/browse/PIG-2142
>
>
> Diffs
> -----
>
> trunk/src/org/apache/pig/PigServer.java 1142512
> trunk/src/org/apache/pig/impl/io/FileLocalizer.java 1142512
> trunk/test/org/apache/pig/test/TestPigServer.java 1142512
>
> Diff: https://reviews.apache.org/r/976/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dmitriy
>
>
Re: Review Request: PIG-2142: Allow registering multiple jars from DFS via
single statement
Posted by Raghu Angadi <ra...@apache.org>.
> On 2011-07-07 01:16:22, Daniel Dai wrote:
> > trunk/src/org/apache/pig/impl/io/FileLocalizer.java, line 796
> > <https://reviews.apache.org/r/976/diff/1/?file=21967#file21967line796>
> >
> > Is it possible to avoid a listStatus call if we are not use globbing?
>
> Raghu Angadi wrote:
> globStatus() does the check. The check itself is pretty involved. If a path does not have a glob, it eventually results in only one call to getFileStatus().
>
> Daniel Dai wrote:
> Does globStatus check it on local side?
what do mean by 'local side'? globStatus() is client side wrapper over listStatus and getFileStatus.
- Raghu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/976/#review983
-----------------------------------------------------------
On 2011-07-04 04:45:47, Dmitriy Ryaboy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/976/
> -----------------------------------------------------------
>
> (Updated 2011-07-04 04:45:47)
>
>
> Review request for pig and Daniel Dai.
>
>
> Summary
> -------
>
> Posting on behalf of Raghu.
> Daniel, can you take a look since this collides directly with your changes in PIG-1566?
>
> Argh, reviewboard apparently does not get git diffs. We'll try to make something it gets and add later.. please see the patch on the Jira in the meantime.
>
>
> This addresses bug PIG-2142.
> https://issues.apache.org/jira/browse/PIG-2142
>
>
> Diffs
> -----
>
> trunk/src/org/apache/pig/PigServer.java 1142512
> trunk/src/org/apache/pig/impl/io/FileLocalizer.java 1142512
> trunk/test/org/apache/pig/test/TestPigServer.java 1142512
>
> Diff: https://reviews.apache.org/r/976/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dmitriy
>
>
Re: Review Request: PIG-2142: Allow registering multiple jars from DFS via
single statement
Posted by Daniel Dai <da...@gmail.com>.
> On 2011-07-07 01:16:22, Daniel Dai wrote:
> > trunk/src/org/apache/pig/impl/io/FileLocalizer.java, line 796
> > <https://reviews.apache.org/r/976/diff/1/?file=21967#file21967line796>
> >
> > Is it possible to avoid a listStatus call if we are not use globbing?
>
> Raghu Angadi wrote:
> globStatus() does the check. The check itself is pretty involved. If a path does not have a glob, it eventually results in only one call to getFileStatus().
>
> Daniel Dai wrote:
> Does globStatus check it on local side?
>
> Raghu Angadi wrote:
> what do mean by 'local side'? globStatus() is client side wrapper over listStatus and getFileStatus.
Ok, I get you. What I concern is to put unnecessary burden to hdfs. But since you are hdfs committer, I can trust you to make the right decision for hdfs :)
+1 on the patch.
- Daniel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/976/#review983
-----------------------------------------------------------
On 2011-07-04 04:45:47, Dmitriy Ryaboy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/976/
> -----------------------------------------------------------
>
> (Updated 2011-07-04 04:45:47)
>
>
> Review request for pig and Daniel Dai.
>
>
> Summary
> -------
>
> Posting on behalf of Raghu.
> Daniel, can you take a look since this collides directly with your changes in PIG-1566?
>
> Argh, reviewboard apparently does not get git diffs. We'll try to make something it gets and add later.. please see the patch on the Jira in the meantime.
>
>
> This addresses bug PIG-2142.
> https://issues.apache.org/jira/browse/PIG-2142
>
>
> Diffs
> -----
>
> trunk/src/org/apache/pig/PigServer.java 1142512
> trunk/src/org/apache/pig/impl/io/FileLocalizer.java 1142512
> trunk/test/org/apache/pig/test/TestPigServer.java 1142512
>
> Diff: https://reviews.apache.org/r/976/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dmitriy
>
>
Re: Review Request: PIG-2142: Allow registering multiple jars from DFS via
single statement
Posted by Daniel Dai <da...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/976/#review983
-----------------------------------------------------------
trunk/src/org/apache/pig/impl/io/FileLocalizer.java
<https://reviews.apache.org/r/976/#comment2048>
Is it possible to avoid a listStatus call if we are not use globbing?
- Daniel
On 2011-07-04 04:45:47, Dmitriy Ryaboy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/976/
> -----------------------------------------------------------
>
> (Updated 2011-07-04 04:45:47)
>
>
> Review request for pig and Daniel Dai.
>
>
> Summary
> -------
>
> Posting on behalf of Raghu.
> Daniel, can you take a look since this collides directly with your changes in PIG-1566?
>
> Argh, reviewboard apparently does not get git diffs. We'll try to make something it gets and add later.. please see the patch on the Jira in the meantime.
>
>
> This addresses bug PIG-2142.
> https://issues.apache.org/jira/browse/PIG-2142
>
>
> Diffs
> -----
>
> trunk/src/org/apache/pig/PigServer.java 1142512
> trunk/src/org/apache/pig/impl/io/FileLocalizer.java 1142512
> trunk/test/org/apache/pig/test/TestPigServer.java 1142512
>
> Diff: https://reviews.apache.org/r/976/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dmitriy
>
>