You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by kalyan kumar kalvagadda via Review Board <no...@reviews.apache.org> on 2020/09/17 15:40:23 UTC
Review Request 72882: SENTRY-2558: Issue in creating full snapshot
when the storage descriptor for a table is null.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72882/
-----------------------------------------------------------
Review request for sentry and Na Li.
Bugs: SENTRY-2558
https://issues.apache.org/jira/browse/SENTRY-2558
Repository: sentry
Description
-------
When the storage descriptor for the table is null, sentry throws NullPointer Exception and fails to create a snapshot.
020-09-17 15:08:43,647 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.lang.NullPointerException at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInitializer.java:369) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-09-17 15:08:43,650 INFO org.apache.sentry.service.thrift.FullUpdateInitializer: For db = t
est_db4 table = test_tb5 total number of partitions = 0 2020-09-17 15:08:43,657 INFO hive.metastore: Closed a connection to metastore, current connections: 8 2020-09-17 15:08:43,657 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@68c2c9a6 rejected from java.util.concurrent.ThreadPoolExecutor@1a289856[Shutting down, pool size = 7, active threads = 7, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInitia
lizer.java:366) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Instead, it should ignore the object and move-on.
Diffs
-----
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/service/thrift/FullUpdateInitializer.java 668a4ca05
Diff: https://reviews.apache.org/r/72882/diff/1/
Testing
-------
I instrumented the failure by manually updating HMS database made sure that logs look similar to one in the description. I verified ths issue is not observed with the fix made.
Thanks,
kalyan kumar kalvagadda
Re: Review Request 72882: SENTRY-2558: Issue in creating full
snapshot when the storage descriptor for a table is null.
Posted by kalyan kumar kalvagadda via Review Board <no...@reviews.apache.org>.
> On Sept. 17, 2020, 3:44 p.m., Na Li wrote:
> > Ship It!
Thanks for the review. Will merge the change after the unit tests are green.
- kalyan kumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72882/#review221882
-----------------------------------------------------------
On Sept. 17, 2020, 3:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72882/
> -----------------------------------------------------------
>
> (Updated Sept. 17, 2020, 3:40 p.m.)
>
>
> Review request for sentry and Na Li.
>
>
> Bugs: SENTRY-2558
> https://issues.apache.org/jira/browse/SENTRY-2558
>
>
> Repository: sentry
>
>
> Description
> -------
>
> When the storage descriptor for the table is null, sentry throws NullPointer Exception and fails to create a snapshot.
>
> 020-09-17 15:08:43,647 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.lang.NullPointerException at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInitializer.java:369) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-09-17 15:08:43,650 INFO org.apache.sentry.service.thrift.FullUpdateInitializer: For db =
test_db4 table = test_tb5 total number of partitions = 0 2020-09-17 15:08:43,657 INFO hive.metastore: Closed a connection to metastore, current connections: 8 2020-09-17 15:08:43,657 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@68c2c9a6 rejected from java.util.concurrent.ThreadPoolExecutor@1a289856[Shutting down, pool size = 7, active threads = 7, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInit
ializer.java:366) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
> Instead, it should ignore the object and move-on.
>
>
> Diffs
> -----
>
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/service/thrift/FullUpdateInitializer.java 668a4ca05
>
>
> Diff: https://reviews.apache.org/r/72882/diff/1/
>
>
> Testing
> -------
>
> I instrumented the failure by manually updating HMS database made sure that logs look similar to one in the description. I verified ths issue is not observed with the fix made.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>
Re: Review Request 72882: SENTRY-2558: Issue in creating full
snapshot when the storage descriptor for a table is null.
Posted by Na Li via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72882/#review221882
-----------------------------------------------------------
Ship it!
Ship It!
- Na Li
On Sept. 17, 2020, 3:40 p.m., kalyan kumar kalvagadda wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72882/
> -----------------------------------------------------------
>
> (Updated Sept. 17, 2020, 3:40 p.m.)
>
>
> Review request for sentry and Na Li.
>
>
> Bugs: SENTRY-2558
> https://issues.apache.org/jira/browse/SENTRY-2558
>
>
> Repository: sentry
>
>
> Description
> -------
>
> When the storage descriptor for the table is null, sentry throws NullPointer Exception and fails to create a snapshot.
>
> 020-09-17 15:08:43,647 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.lang.NullPointerException at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInitializer.java:369) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-09-17 15:08:43,650 INFO org.apache.sentry.service.thrift.FullUpdateInitializer: For db =
test_db4 table = test_tb5 total number of partitions = 0 2020-09-17 15:08:43,657 INFO hive.metastore: Closed a connection to metastore, current connections: 8 2020-09-17 15:08:43,657 ERROR org.apache.sentry.service.thrift.FullUpdateInitializer: Failed to execute task java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@68c2c9a6 rejected from java.util.concurrent.ThreadPoolExecutor@1a289856[Shutting down, pool size = 7, active threads = 7, queued tasks = 0, completed tasks = 9] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) at org.apache.sentry.service.thrift.FullUpdateInitializer$TableTask.doTask(FullUpdateInit
ializer.java:366) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:222) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:258) at org.apache.sentry.service.thrift.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:190) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
> Instead, it should ignore the object and move-on.
>
>
> Diffs
> -----
>
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/service/thrift/FullUpdateInitializer.java 668a4ca05
>
>
> Diff: https://reviews.apache.org/r/72882/diff/1/
>
>
> Testing
> -------
>
> I instrumented the failure by manually updating HMS database made sure that logs look similar to one in the description. I verified ths issue is not observed with the fix made.
>
>
> Thanks,
>
> kalyan kumar kalvagadda
>
>