You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by Suhas Vasu <su...@gmail.com> on 2014/09/08 08:57:19 UTC
Review Request 25425: FALCON-677. Feed data and stats path defaults
to /tmp/
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/
-----------------------------------------------------------
Review request for Falcon and shwethags.
Repository: falcon-git
Description
-------
If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
Diffs
-----
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
Diff: https://reviews.apache.org/r/25425/diff/
Testing
-------
Thanks,
Suhas Vasu
Re: Review Request 25425: FALCON-677. Feed data and stats path
defaults to /tmp/
Posted by sh...@inmobi.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/#review52700
-----------------------------------------------------------
lookks good otherwise. Can you just verify the end to end flow once?
common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java
<https://reviews.apache.org/r/25425/#comment91683>
don't need validFeed bool?
- shwethags
On Sept. 9, 2014, 10:08 a.m., Suhas Vasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25425/
> -----------------------------------------------------------
>
> (Updated Sept. 9, 2014, 10:08 a.m.)
>
>
> Review request for Falcon and shwethags.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
>
> Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/falcon/entity/FeedHelper.java 323188d
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
> common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 8fd56e1
> common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
>
> Diff: https://reviews.apache.org/r/25425/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Suhas Vasu
>
>
Re: Review Request 25425: FALCON-677. Feed data and stats path
defaults to /tmp/
Posted by sh...@inmobi.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/#review52826
-----------------------------------------------------------
Ship it!
Ship It!
- shwethags
On Sept. 9, 2014, 12:01 p.m., Suhas Vasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25425/
> -----------------------------------------------------------
>
> (Updated Sept. 9, 2014, 12:01 p.m.)
>
>
> Review request for Falcon and shwethags.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
>
> Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/falcon/entity/FeedHelper.java 323188d
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
> common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 8fd56e1
> common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
>
> Diff: https://reviews.apache.org/r/25425/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Suhas Vasu
>
>
Re: Review Request 25425: FALCON-677. Feed data and stats path
defaults to /tmp/
Posted by Suhas Vasu <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/
-----------------------------------------------------------
(Updated Sept. 9, 2014, 12:01 p.m.)
Review request for Falcon and shwethags.
Repository: falcon-git
Description
-------
If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
Diffs (updated)
-----
common/src/main/java/org/apache/falcon/entity/FeedHelper.java 323188d
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 8fd56e1
common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
Diff: https://reviews.apache.org/r/25425/diff/
Testing
-------
Thanks,
Suhas Vasu
Re: Review Request 25425: FALCON-677. Feed data and stats path
defaults to /tmp/
Posted by Suhas Vasu <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/#review52701
-----------------------------------------------------------
common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java
<https://reviews.apache.org/r/25425/#comment91684>
yes its redundant. will remove
- Suhas Vasu
On Sept. 9, 2014, 10:08 a.m., Suhas Vasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25425/
> -----------------------------------------------------------
>
> (Updated Sept. 9, 2014, 10:08 a.m.)
>
>
> Review request for Falcon and shwethags.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
>
> Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/falcon/entity/FeedHelper.java 323188d
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
> common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 8fd56e1
> common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
>
> Diff: https://reviews.apache.org/r/25425/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Suhas Vasu
>
>
Re: Review Request 25425: FALCON-677. Feed data and stats path
defaults to /tmp/
Posted by Suhas Vasu <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/
-----------------------------------------------------------
(Updated Sept. 9, 2014, 10:08 a.m.)
Review request for Falcon and shwethags.
Changes
-------
Updating the diff based on code review
Repository: falcon-git
Description
-------
If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
Diffs (updated)
-----
common/src/main/java/org/apache/falcon/entity/FeedHelper.java 323188d
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 8fd56e1
common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
Diff: https://reviews.apache.org/r/25425/diff/
Testing
-------
Thanks,
Suhas Vasu
Re: Review Request 25425: FALCON-677. Feed data and stats path
defaults to /tmp/
Posted by Suhas Vasu <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/
-----------------------------------------------------------
(Updated Sept. 8, 2014, 7:37 p.m.)
Review request for Falcon and shwethags.
Changes
-------
Changes based on comments
Repository: falcon-git
Description
-------
If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
Diffs (updated)
-----
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 8fd56e1
common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
Diff: https://reviews.apache.org/r/25425/diff/
Testing
-------
Thanks,
Suhas Vasu
Re: Review Request 25425: FALCON-677. Feed data and stats path
defaults to /tmp/
Posted by sh...@inmobi.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/#review52583
-----------------------------------------------------------
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java
<https://reviews.apache.org/r/25425/#comment91438>
Use StringUtils.equals which handles null
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java
<https://reviews.apache.org/r/25425/#comment91437>
return null for location
- shwethags
On Sept. 8, 2014, 7:02 a.m., Suhas Vasu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25425/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2014, 7:02 a.m.)
>
>
> Review request for Falcon and shwethags.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
>
> Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
> common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
>
> Diff: https://reviews.apache.org/r/25425/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Suhas Vasu
>
>
Re: Review Request 25425: FALCON-677. Feed data and stats path
defaults to /tmp/
Posted by Suhas Vasu <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25425/
-----------------------------------------------------------
(Updated Sept. 8, 2014, 7:02 a.m.)
Review request for Falcon and shwethags.
Repository: falcon-git
Description
-------
If a feed does not have stats path specified, it assumes it as /tmp/ on the hdfs. Ideally we should be throwing an error that the specified feed does not have a stats path.
Users may have specified to delete the stats path if it exists, in the prepare tag in workflow. In such a scenario the whole of /tmp/ is cleared, which is totally undesirable.
Diffs
-----
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java 4eb3d60
common/src/test/java/org/apache/falcon/entity/FileSystemStorageTest.java a78c678
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java 1fa6758
oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 23c01a9
Diff: https://reviews.apache.org/r/25425/diff/
Testing
-------
Thanks,
Suhas Vasu