You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Puneet Gupta <pu...@gmail.com> on 2016/05/04 10:17:59 UTC
Review Request 46968: LENS-1029: lens-server-snapshotter died in one
of the scenarios
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/
-----------------------------------------------------------
Review request for lens.
Bugs: lens-1029
https://issues.apache.org/jira/browse/lens-1029
Repository: lens
Description
-------
- Service level persistence isolation. If persisting one service fails, other services should still be persisted.
- Persistence thread to run only in case SERVER_RESTART_ENABLED = true
- Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
- Catching Exception instead of IOException to prevent the persistence task from dying.
- using writeObject() instead of writeInt() for automatic null handling.
Diffs
-----
lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
Diff: https://reviews.apache.org/r/46968/diff/
Testing
-------
Relying on exiting test case to check persistence TestServerRestart.
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**Apart from this**
- Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
- Did some local testing to check graceful shutdown of ScheduledExecutorService
Don't think we need a test cases for above two cince its supported out of box by java
Thanks,
Puneet Gupta
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
> On May 4, 2016, 11:50 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, lines 313-314
> > <https://reviews.apache.org/r/46968/diff/1/?file=1370653#file1370653line313>
> >
> > Why is this removed?
This is moved to init.
> On May 4, 2016, 11:50 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 353
> > <https://reviews.apache.org/r/46968/diff/1/?file=1370653#file1370653line353>
> >
> > Should we catch Throwable?
Exception should be ok. Didn't want to catch Errors like OutOfMemoryError in which case the server state itself is not stable. This should be handled else where.
> On May 4, 2016, 11:50 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 420
> > <https://reviews.apache.org/r/46968/diff/1/?file=1370653#file1370653line420>
> >
> > typo in comment
Will update this
- Puneet
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131654
-----------------------------------------------------------
On May 4, 2016, 10:17 a.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 4, 2016, 10:17 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
> On May 4, 2016, 11:50 a.m., Rajat Khandelwal wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, lines 313-314
> > <https://reviews.apache.org/r/46968/diff/1/?file=1370653#file1370653line313>
> >
> > Why is this removed?
>
> Puneet Gupta wrote:
> This is moved to init.
Removed the feature as discuused in other comments
- Puneet
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131654
-----------------------------------------------------------
On May 7, 2016, 10:58 a.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 7, 2016, 10:58 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
> lens-server/src/main/resources/lensserver-default.xml 1a15658
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
> src/site/apt/admin/config.apt 5466e7a
> src/site/apt/admin/deployment.apt b4f4d0a
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Rajat Khandelwal <ra...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131654
-----------------------------------------------------------
lens-server/src/main/java/org/apache/lens/server/LensServices.java
<https://reviews.apache.org/r/46968/#comment195686>
Why is this removed?
lens-server/src/main/java/org/apache/lens/server/LensServices.java (line 341)
<https://reviews.apache.org/r/46968/#comment195688>
Should we catch Throwable?
lens-server/src/main/java/org/apache/lens/server/LensServices.java (line 399)
<https://reviews.apache.org/r/46968/#comment195689>
typo in comment
- Rajat Khandelwal
On May 4, 2016, 3:47 p.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 4, 2016, 3:47 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
> On May 5, 2016, 6:43 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 422
> > <https://reviews.apache.org/r/46968/diff/2/?file=1370733#file1370733line422>
> >
> > Why are we putting such a wait counter? Shouldnt we simple join the thread and wait for it to completely write the persist state?
>
> Puneet Gupta wrote:
> serverSnapshotScheduler.shutdown() initiates a graceful shutdown but one needs to wait for the shutdown using awaitTermination() which takes wait time as input.
> The options to wait are
> 1. Infinite wait.
> 2. Timed Wait. Incase there is some issue like OOM or Filesystem realted issues which delays the writing, wanted to try max for 5 minutes.
>
> But since after we come out of awaitTermination we are calling persistLensServiceState() again, the first wait can be made infinite too (as the secon d call to persistLensServiceState has no wait limit anyway)
>
> Will have an infinite WAIT.
Made wait infinite
- Puneet
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131792
-----------------------------------------------------------
On May 7, 2016, 10:58 a.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 7, 2016, 10:58 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
> lens-server/src/main/resources/lensserver-default.xml 1a15658
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
> src/site/apt/admin/config.apt 5466e7a
> src/site/apt/admin/deployment.apt b4f4d0a
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
> On May 5, 2016, 6:43 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 422
> > <https://reviews.apache.org/r/46968/diff/2/?file=1370733#file1370733line422>
> >
> > Why are we putting such a wait counter? Shouldnt we simple join the thread and wait for it to completely write the persist state?
serverSnapshotScheduler.shutdown() initiates a graceful shutdown but one needs to wait for the shutdown using awaitTermination() which takes wait time as input.
The options to wait are
1. Infinite wait.
2. Timed Wait. Incase there is some issue like OOM or Filesystem realted issues which delays the writing, wanted to try max for 5 minutes.
But since after we come out of awaitTermination we are calling persistLensServiceState() again, the first wait can be made infinite too (as the secon d call to persistLensServiceState has no wait limit anyway)
Will have an infinite WAIT.
- Puneet
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131792
-----------------------------------------------------------
On May 4, 2016, 12:03 p.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 4, 2016, 12:03 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131792
-----------------------------------------------------------
lens-server/src/main/java/org/apache/lens/server/LensServices.java (line 401)
<https://reviews.apache.org/r/46968/#comment195823>
Why are we putting such a wait counter? Shouldnt we simple join the thread and wait for it to completely write the persist state?
- Amareshwari Sriramadasu
On May 4, 2016, 12:03 p.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 4, 2016, 12:03 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review132214
-----------------------------------------------------------
Ship it!
Ship It!
- Amareshwari Sriramadasu
On May 9, 2016, 6:36 a.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 9, 2016, 6:36 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
> lens-server/src/main/resources/lensserver-default.xml 1a15658
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
> src/site/apt/admin/config.apt 5466e7a
> src/site/apt/admin/deployment.apt b4f4d0a
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/
-----------------------------------------------------------
(Updated May 9, 2016, 6:36 a.m.)
Review request for lens.
Changes
-------
**Reverted last patch** (Fiexd review comment - added back err counter on failing to close file)
**Ran Build**
[INFO] Lens Checkstyle Rules ............................. SUCCESS [1.554s]
[INFO] Lens .............................................. SUCCESS [4.660s]
[INFO] Lens API .......................................... SUCCESS [22.497s]
[INFO] Lens API for server and extensions ................ SUCCESS [14.501s]
[INFO] Lens Cube ......................................... SUCCESS [7:03.831s]
[INFO] Lens DB storage ................................... SUCCESS [15.612s]
[INFO] Lens Query Library ................................ SUCCESS [11.233s]
[INFO] Lens Hive Driver .................................. SUCCESS [2:26.034s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [29.249s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [14.076s]
[INFO] Lens Server ....................................... SUCCESS [13:24.649s]
[INFO] Lens client ....................................... SUCCESS [1:43.343s]
[INFO] Lens CLI .......................................... SUCCESS [2:10.200s]
[INFO] Lens Examples ..................................... SUCCESS [7.444s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.634s]
[INFO] Lens Distribution ................................. SUCCESS [8.993s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:07.881s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.595s]
[INFO] Lens Regression ................................... SUCCESS [11.479s]
[INFO] Lens UI ........................................... SUCCESS [49.357s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31:09.528s
[INFO] Finished at: Sat May 07 12:23:59 UTC 2016
Bugs: lens-1029
https://issues.apache.org/jira/browse/lens-1029
Repository: lens
Description
-------
- Service level persistence isolation. If persisting one service fails, other services should still be persisted.
- Persistence thread to run only in case SERVER_RESTART_ENABLED = true
- Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
- Catching Exception instead of IOException to prevent the persistence task from dying.
- using writeObject() instead of writeInt() for automatic null handling.
Diffs (updated)
-----
lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
lens-server/src/main/resources/lensserver-default.xml 1a15658
lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
src/site/apt/admin/config.apt 5466e7a
src/site/apt/admin/deployment.apt b4f4d0a
Diff: https://reviews.apache.org/r/46968/diff/
Testing
-------
Relying on exiting test case to check persistence TestServerRestart.
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**Apart from this**
- Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
- Did some local testing to check graceful shutdown of ScheduledExecutorService
Don't think we need a test cases for above two cince its supported out of box by java
Thanks,
Puneet Gupta
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/
-----------------------------------------------------------
(Updated May 9, 2016, 6:26 a.m.)
Review request for lens.
Changes
-------
Fiexd review comment - added back err counter on failing to close file
Bugs: lens-1029
https://issues.apache.org/jira/browse/lens-1029
Repository: lens
Description
-------
- Service level persistence isolation. If persisting one service fails, other services should still be persisted.
- Persistence thread to run only in case SERVER_RESTART_ENABLED = true
- Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
- Catching Exception instead of IOException to prevent the persistence task from dying.
- using writeObject() instead of writeInt() for automatic null handling.
Diffs (updated)
-----
lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
lens-server/src/main/resources/lensserver-default.xml 1a15658
lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
src/site/apt/admin/config.apt 5466e7a
src/site/apt/admin/deployment.apt b4f4d0a
Diff: https://reviews.apache.org/r/46968/diff/
Testing
-------
Relying on exiting test case to check persistence TestServerRestart.
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**Apart from this**
- Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
- Did some local testing to check graceful shutdown of ScheduledExecutorService
Don't think we need a test cases for above two cince its supported out of box by java
Thanks,
Puneet Gupta
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
> On May 9, 2016, 5:27 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 446
> > <https://reviews.apache.org/r/46968/diff/3/?file=1375652#file1375652line446>
> >
> > Can we put back the counter?
This may not be req as this is file system close (and not file close).
- Puneet
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review132206
-----------------------------------------------------------
On May 9, 2016, 6:26 a.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 9, 2016, 6:26 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
> lens-server/src/main/resources/lensserver-default.xml 1a15658
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
> src/site/apt/admin/config.apt 5466e7a
> src/site/apt/admin/deployment.apt b4f4d0a
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Amareshwari Sriramadasu <am...@apache.org>.
> On May 9, 2016, 5:27 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 446
> > <https://reviews.apache.org/r/46968/diff/3/?file=1375652#file1375652line446>
> >
> > Can we put back the counter?
>
> Puneet Gupta wrote:
> This may not be req as this is file system close (and not file close).
Makes sense. Ship verison #3
- Amareshwari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review132206
-----------------------------------------------------------
On May 9, 2016, 6:36 a.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 9, 2016, 6:36 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
> lens-server/src/main/resources/lensserver-default.xml 1a15658
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
> src/site/apt/admin/config.apt 5466e7a
> src/site/apt/admin/deployment.apt b4f4d0a
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review132206
-----------------------------------------------------------
lens-server/src/main/java/org/apache/lens/server/LensServices.java (line 420)
<https://reviews.apache.org/r/46968/#comment196368>
Can we put back the counter?
- Amareshwari Sriramadasu
On May 7, 2016, 10:58 a.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 7, 2016, 10:58 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
> lens-server/src/main/resources/lensserver-default.xml 1a15658
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
> src/site/apt/admin/config.apt 5466e7a
> src/site/apt/admin/deployment.apt b4f4d0a
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/
-----------------------------------------------------------
(Updated May 7, 2016, 10:58 a.m.)
Review request for lens.
Changes
-------
- Removed timed wait on stop
- Merged lens.server.restart.enabled and lens.server.recover.onrestart into one property lens.server.state.persistence.interval.millis
Bugs: lens-1029
https://issues.apache.org/jira/browse/lens-1029
Repository: lens
Description
-------
- Service level persistence isolation. If persisting one service fails, other services should still be persisted.
- Persistence thread to run only in case SERVER_RESTART_ENABLED = true
- Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
- Catching Exception instead of IOException to prevent the persistence task from dying.
- using writeObject() instead of writeInt() for automatic null handling.
Diffs (updated)
-----
lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
lens-server/src/main/resources/lensserver-default.xml 1a15658
lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
src/site/apt/admin/config.apt 5466e7a
src/site/apt/admin/deployment.apt b4f4d0a
Diff: https://reviews.apache.org/r/46968/diff/
Testing
-------
Relying on exiting test case to check persistence TestServerRestart.
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**Apart from this**
- Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
- Did some local testing to check graceful shutdown of ScheduledExecutorService
Don't think we need a test cases for above two cince its supported out of box by java
Thanks,
Puneet Gupta
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
> On May 5, 2016, 6:18 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 242
> > <https://reviews.apache.org/r/46968/diff/2/?file=1370733#file1370733line242>
> >
> > Right now, We have two different configurations - which are independent:
> >
> > <property>
> > <name>lens.server.restart.enabled</name>
> > <value>true</value>
> > <description>If flag is enabled, all the services will be persisted to persistent
> > location passed.
> > </description>
> > </property>
> >
> > <property>
> > <name>lens.server.recover.onrestart</name>
> > <value>true</value>
> > <description>If the flag is enabled, all the services will be started from last
> > saved state, if disabled all the services will start afresh
> > </description>
> > </property>
> >
> > With this change done, we are coupling them. If lens.server.recover.onrestart is true, then lens.server.restart.enabled should also be true.
> >
> > I'm fine with the change. But just calling out.
Rajat had similar query. Actually this happned by mistake while refatoring code (i was not expecting mutiple configs for this feature and overlooked the second one).
But now if I think of it, one configuration should suffice as poointed out by Amareshwari
We can merge the two options into one and make it simpler. If we do this then to achieve the following, we would need some manual intervention
Use Case : lens.server.restart.enabled = true and lens.server.recover.onrestart = true. Now we shutdown the server and change lens.server.restart.enabled to false. In this case the server will recover using old state but will not persist any new states. Now if we restart the server again, it will use the old state once again to recover.
To achieve the same after merging the features (there will be only one property lens.server.state.persistence.enabled = true) , user will have to copy persisted files after the shutdown (say copy A). Then restart the server which will use the last persisted state (same as copy A) to recover and will start persisting the new states. Now if we want to restart once again from old state (A), user will have to manually overwrite the persisted files with copied persisted files (copy A) before starting the server. This is a rare use case though ( mostly for testing)
pros of merging : simplicity and single flow
cons : manual intervention in some rare cases
Thoughts ?
- Puneet
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131786
-----------------------------------------------------------
On May 4, 2016, 12:03 p.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 4, 2016, 12:03 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
> On May 5, 2016, 6:18 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/LensServices.java, line 242
> > <https://reviews.apache.org/r/46968/diff/2/?file=1370733#file1370733line242>
> >
> > Right now, We have two different configurations - which are independent:
> >
> > <property>
> > <name>lens.server.restart.enabled</name>
> > <value>true</value>
> > <description>If flag is enabled, all the services will be persisted to persistent
> > location passed.
> > </description>
> > </property>
> >
> > <property>
> > <name>lens.server.recover.onrestart</name>
> > <value>true</value>
> > <description>If the flag is enabled, all the services will be started from last
> > saved state, if disabled all the services will start afresh
> > </description>
> > </property>
> >
> > With this change done, we are coupling them. If lens.server.recover.onrestart is true, then lens.server.restart.enabled should also be true.
> >
> > I'm fine with the change. But just calling out.
>
> Puneet Gupta wrote:
> Rajat had similar query. Actually this happned by mistake while refatoring code (i was not expecting mutiple configs for this feature and overlooked the second one).
> But now if I think of it, one configuration should suffice as poointed out by Amareshwari
>
> We can merge the two options into one and make it simpler. If we do this then to achieve the following, we would need some manual intervention
>
> Use Case : lens.server.restart.enabled = true and lens.server.recover.onrestart = true. Now we shutdown the server and change lens.server.restart.enabled to false. In this case the server will recover using old state but will not persist any new states. Now if we restart the server again, it will use the old state once again to recover.
>
> To achieve the same after merging the features (there will be only one property lens.server.state.persistence.enabled = true) , user will have to copy persisted files after the shutdown (say copy A). Then restart the server which will use the last persisted state (same as copy A) to recover and will start persisting the new states. Now if we want to restart once again from old state (A), user will have to manually overwrite the persisted files with copied persisted files (copy A) before starting the server. This is a rare use case though ( mostly for testing)
>
> pros of merging : simplicity and single flow
> cons : manual intervention in some rare cases
>
> Thoughts ?
Merged into one property
- Puneet
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131786
-----------------------------------------------------------
On May 7, 2016, 10:58 a.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 7, 2016, 10:58 a.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 23537cb
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
> lens-server/src/main/resources/lensserver-default.xml 1a15658
> lens-server/src/test/java/org/apache/lens/server/TestServerRestart.java 1fa61ef
> src/site/apt/admin/config.apt 5466e7a
> src/site/apt/admin/deployment.apt b4f4d0a
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Amareshwari Sriramadasu <am...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/#review131786
-----------------------------------------------------------
lens-server/src/main/java/org/apache/lens/server/LensServices.java (line 242)
<https://reviews.apache.org/r/46968/#comment195820>
Right now, We have two different configurations - which are independent:
<property>
<name>lens.server.restart.enabled</name>
<value>true</value>
<description>If flag is enabled, all the services will be persisted to persistent
location passed.
</description>
</property>
<property>
<name>lens.server.recover.onrestart</name>
<value>true</value>
<description>If the flag is enabled, all the services will be started from last
saved state, if disabled all the services will start afresh
</description>
</property>
With this change done, we are coupling them. If lens.server.recover.onrestart is true, then lens.server.restart.enabled should also be true.
I'm fine with the change. But just calling out.
- Amareshwari Sriramadasu
On May 4, 2016, 12:03 p.m., Puneet Gupta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46968/
> -----------------------------------------------------------
>
> (Updated May 4, 2016, 12:03 p.m.)
>
>
> Review request for lens.
>
>
> Bugs: lens-1029
> https://issues.apache.org/jira/browse/lens-1029
>
>
> Repository: lens
>
>
> Description
> -------
>
> - Service level persistence isolation. If persisting one service fails, other services should still be persisted.
> - Persistence thread to run only in case SERVER_RESTART_ENABLED = true
> - Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
> - Catching Exception instead of IOException to prevent the persistence task from dying.
> - using writeObject() instead of writeInt() for automatic null handling.
>
>
> Diffs
> -----
>
> lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
> lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
>
> Diff: https://reviews.apache.org/r/46968/diff/
>
>
> Testing
> -------
>
> Relying on exiting test case to check persistence TestServerRestart.
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
> Running org.apache.lens.server.TestServerRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
>
> Results :
>
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
>
>
> **Apart from this**
> - Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
> - Did some local testing to check graceful shutdown of ScheduledExecutorService
> Don't think we need a test cases for above two cince its supported out of box by java
>
>
> Thanks,
>
> Puneet Gupta
>
>
Re: Review Request 46968: LENS-1029: lens-server-snapshotter died in
one of the scenarios
Posted by Puneet Gupta <pu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46968/
-----------------------------------------------------------
(Updated May 4, 2016, 12:03 p.m.)
Review request for lens.
Changes
-------
Fixed Typo
Bugs: lens-1029
https://issues.apache.org/jira/browse/lens-1029
Repository: lens
Description
-------
- Service level persistence isolation. If persisting one service fails, other services should still be persisted.
- Persistence thread to run only in case SERVER_RESTART_ENABLED = true
- Moved form Timer to ScheduledExecutorService. Graceful shutdown of ScheduledExecutorService enabled to allow a running persistence task, if any, to finish
- Catching Exception instead of IOException to prevent the persistence task from dying.
- using writeObject() instead of writeInt() for automatic null handling.
Diffs (updated)
-----
lens-query-lib/src/main/java/org/apache/lens/lib/query/AbstractFileFormatter.java 8c06621
lens-server/src/main/java/org/apache/lens/server/LensServices.java 48b3e00
Diff: https://reviews.apache.org/r/46968/diff/
Testing
-------
Relying on exiting test case to check persistence TestServerRestart.
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testQueryService**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 206.408 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**mvn test -Dtest=org.apache.lens.server.TestServerRestart#testSessionRestart**
Running org.apache.lens.server.TestServerRestart
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.711 sec - in org.apache.lens.server.TestServerRestart
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
**Apart from this**
- Did some local testing for making sure null Integers are persisted using writeObject and can be read back as well.
- Did some local testing to check graceful shutdown of ScheduledExecutorService
Don't think we need a test cases for above two cince its supported out of box by java
Thanks,
Puneet Gupta