You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by tushargosavi <gi...@git.apache.org> on 2016/06/15 11:44:16 UTC
[GitHub] apex-malhar pull request #318: APEXMALHAR-2119 add setters for partition cou...
GitHub user tushargosavi opened a pull request:
https://github.com/apache/apex-malhar/pull/318
APEXMALHAR-2119 add setters for partition count and index.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tushargosavi/incubator-apex-malhar APEXMALHAR-2119
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/318.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #318
----
commit 82cfbbbcca80c4f6eb4d76cb35a6fb5402db0d58
Author: Tushar R. Gosavi <tu...@apache.org>
Date: 2016-06-15T11:42:16Z
APEXMALHAR-2119 add setters for partition count and index.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #318: APEXMALHAR-2119 add setters for partition cou...
Posted by amberarrow <gi...@git.apache.org>.
Github user amberarrow commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/318#discussion_r67351327
--- Diff: library/src/main/java/com/datatorrent/lib/io/fs/AbstractFileInputOperator.java ---
@@ -1110,6 +1116,16 @@ public String toString()
return "DirectoryScanner [filePatternRegexp=" + filePatternRegexp + " partitionIndex=" +
partitionIndex + " partitionCount=" + partitionCount + "]";
}
+
+ protected void setPartitionIndex(int partitionIndex)
+ {
+ this.partitionIndex = partitionIndex;
+ }
+
+ protected void setPartitionCount(int partitionCount)
+ {
+ this.partitionCount = partitionCount;
+ }
--- End diff --
Don't see a test for the newly added methods setPartitionIndex and setPartitionCount (since the rationale in the ticket is "Inherited DirectoryScanner can not set them", we should have a test that shows that this situation has
been remedied).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #318: APEXMALHAR-2119 add setters for partition cou...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/apex-malhar/pull/318
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #318: APEXMALHAR-2119 add setters for partition cou...
Posted by tushargosavi <gi...@git.apache.org>.
Github user tushargosavi commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/318#discussion_r67808791
--- Diff: library/src/main/java/com/datatorrent/lib/io/fs/AbstractFileInputOperator.java ---
@@ -1110,6 +1116,16 @@ public String toString()
return "DirectoryScanner [filePatternRegexp=" + filePatternRegexp + " partitionIndex=" +
partitionIndex + " partitionCount=" + partitionCount + "]";
}
+
+ protected void setPartitionIndex(int partitionIndex)
+ {
+ this.partitionIndex = partitionIndex;
+ }
+
+ protected void setPartitionCount(int partitionCount)
+ {
+ this.partitionCount = partitionCount;
+ }
--- End diff --
With the kryo.clone change in createPartition, it is getting set on the derived object from there. Initially createPartition was creating object of type DirectoryScanner, so derived class had no option to override it and create object of own type, and then set the count and index value, but it could not as those were private without setters in base class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #318: APEXMALHAR-2119 add setters for partition cou...
Posted by tushargosavi <gi...@git.apache.org>.
Github user tushargosavi commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/318#discussion_r67288912
--- Diff: library/src/main/java/com/datatorrent/lib/io/fs/AbstractFileInputOperator.java ---
@@ -1056,10 +1056,15 @@ protected Pattern getRegex()
return pathSet;
}
+ protected int getParition(String filePathStr)
--- End diff --
done ..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #318: APEXMALHAR-2119 add setters for partition cou...
Posted by amberarrow <gi...@git.apache.org>.
Github user amberarrow commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/318#discussion_r67174126
--- Diff: library/src/main/java/com/datatorrent/lib/io/fs/AbstractFileInputOperator.java ---
@@ -1056,10 +1056,15 @@ protected Pattern getRegex()
return pathSet;
}
+ protected int getParition(String filePathStr)
--- End diff --
getParition => getPartition
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #318: APEXMALHAR-2119 add setters for partition cou...
Posted by tushargosavi <gi...@git.apache.org>.
Github user tushargosavi commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/318#discussion_r67813977
--- Diff: library/src/main/java/com/datatorrent/lib/io/fs/AbstractFileInputOperator.java ---
@@ -1110,6 +1116,16 @@ public String toString()
return "DirectoryScanner [filePatternRegexp=" + filePatternRegexp + " partitionIndex=" +
partitionIndex + " partitionCount=" + partitionCount + "]";
}
+
+ protected void setPartitionIndex(int partitionIndex)
+ {
+ this.partitionIndex = partitionIndex;
+ }
+
+ protected void setPartitionCount(int partitionCount)
+ {
+ this.partitionCount = partitionCount;
+ }
--- End diff --
Added test for setPartition and verified that the base function usage the value set.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---