You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by chaithu14 <gi...@git.apache.org> on 2016/07/07 05:45:30 UTC
[GitHub] apex-malhar pull request #334: APEXMALHAR-2136 1) Fixed the null pointer exc...
GitHub user chaithu14 opened a pull request:
https://github.com/apache/apex-malhar/pull/334
APEXMALHAR-2136 1) Fixed the null pointer exception issue. 2) Added getter and setter for IncrementalCheckpointManager
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chaithu14/incubator-apex-malhar APEXMALHAR-2136-NPE-RecoverWindows
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/334.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #334
----
commit 15b4858b72955ff2ce83e1817ac3f64c15347106
Author: Chaitanya <ch...@datatorrent.com>
Date: 2016-07-07T05:42:45Z
APEXMALHAR-2136 1) Fixed the null pointer exception issue. 2) Added getter and setter for IncrementalCheckpointManager
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #334: APEXMALHAR-2136 1) Fixed the null pointer exc...
Posted by chandnisingh <gi...@git.apache.org>.
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/334#discussion_r69942500
--- Diff: library/src/main/java/org/apache/apex/malhar/lib/state/managed/AbstractManagedStateImpl.java ---
@@ -536,6 +538,16 @@ public void setDurationPreventingFreeingSpace(Duration durationPreventingFreeing
this.durationPreventingFreeingSpace = durationPreventingFreeingSpace;
}
+ public IncrementalCheckpointManager getCheckpointManager()
+ {
+ return checkpointManager;
+ }
+
+ public void setCheckpointManager(IncrementalCheckpointManager checkpointManager)
--- End diff --
```@NotNull``` to checkpointManager and please use Preconditions.checkNotNull(...)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #334: APEXMALHAR-2136 1) Fixed the null pointer exc...
Posted by chaithu14 <gi...@git.apache.org>.
Github user chaithu14 commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/334#discussion_r69860038
--- Diff: library/src/main/java/org/apache/apex/malhar/lib/state/managed/AbstractManagedStateImpl.java ---
@@ -203,7 +203,12 @@ public void setup(OperatorContext context)
//delete all the wal files with windows > activationWindow.
//All the wal files with windows <= activationWindow are loaded and kept separately as recovered data.
try {
- for (long recoveredWindow : checkpointManager.getWindowIds(operatorContext.getId())) {
+ long[] recoveredWindows = checkpointManager.getWindowIds(operatorContext.getId());
+ if (recoveredWindows == null) {
+ readerService = Executors.newFixedThreadPool(numReaders, new NameableThreadFactory("managedStateReaders"));
+ return;
+ }
+ for (long recoveredWindow : recoveredWindows) {
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #334: APEXMALHAR-2136 1) Fixed the null pointer exc...
Posted by chaithu14 <gi...@git.apache.org>.
Github user chaithu14 commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/334#discussion_r69947853
--- Diff: library/src/main/java/org/apache/apex/malhar/lib/state/managed/AbstractManagedStateImpl.java ---
@@ -536,6 +538,16 @@ public void setDurationPreventingFreeingSpace(Duration durationPreventingFreeing
this.durationPreventingFreeingSpace = durationPreventingFreeingSpace;
}
+ public IncrementalCheckpointManager getCheckpointManager()
+ {
+ return checkpointManager;
+ }
+
+ public void setCheckpointManager(IncrementalCheckpointManager checkpointManager)
--- End diff --
Updated as per the comment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #334: APEXMALHAR-2136 1) Fixed the null pointer exc...
Posted by chandnisingh <gi...@git.apache.org>.
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/334#discussion_r69857595
--- Diff: library/src/main/java/org/apache/apex/malhar/lib/state/managed/AbstractManagedStateImpl.java ---
@@ -203,7 +203,12 @@ public void setup(OperatorContext context)
//delete all the wal files with windows > activationWindow.
//All the wal files with windows <= activationWindow are loaded and kept separately as recovered data.
try {
- for (long recoveredWindow : checkpointManager.getWindowIds(operatorContext.getId())) {
+ long[] recoveredWindows = checkpointManager.getWindowIds(operatorContext.getId());
+ if (recoveredWindows == null) {
+ readerService = Executors.newFixedThreadPool(numReaders, new NameableThreadFactory("managedStateReaders"));
--- End diff --
This is already being done at the end. why do you need to duplicate the stmt here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #334: APEXMALHAR-2136 1) Fixed the null pointer exc...
Posted by chandnisingh <gi...@git.apache.org>.
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/apex-malhar/pull/334#discussion_r69857656
--- Diff: library/src/main/java/org/apache/apex/malhar/lib/state/managed/AbstractManagedStateImpl.java ---
@@ -203,7 +203,12 @@ public void setup(OperatorContext context)
//delete all the wal files with windows > activationWindow.
//All the wal files with windows <= activationWindow are loaded and kept separately as recovered data.
try {
- for (long recoveredWindow : checkpointManager.getWindowIds(operatorContext.getId())) {
+ long[] recoveredWindows = checkpointManager.getWindowIds(operatorContext.getId());
+ if (recoveredWindows == null) {
+ readerService = Executors.newFixedThreadPool(numReaders, new NameableThreadFactory("managedStateReaders"));
+ return;
+ }
+ for (long recoveredWindow : recoveredWindows) {
--- End diff --
You can put this block within ```if (recoveredWindows == null)```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] apex-malhar pull request #334: APEXMALHAR-2136 1) Fixed the null pointer exc...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/apex-malhar/pull/334
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---