You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/11/26 01:50:08 UTC
[GitHub] [druid] FrankChen021 opened a new pull request #11723: Improve exception message when loading data from web-console
FrankChen021 opened a new pull request #11723:
URL: https://github.com/apache/druid/pull/11723
Fixes #11651
### Description
This PR improves the exception message when using the web console wizard to load data by:
1. catching exception that is thrown when constructing kafka consumer object and then re-throwing a SamplerException which will be automatically mapped to a JSON object by existing `SamplerExceptionMapper`
2. since `SamplerExceptionMapper` returns the exception message to the client without any stack trace, it's not able to know where the exception comes from, so we also loges the cause of SamplerException to log so that we know where the exception is thrown from server side log
Now the web-console showes the error as followes:
<hr>
<!-- Check the items by putting "x" in the brackets for the done things. Not all of these items apply to every PR. Remove the items which are not done or not relevant to the PR. None of the items from the checklist below are strictly necessary, but it would be very helpful if you at least self-review the PR. -->
This PR has:
- [X] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [X] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [X] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cryptoe commented on a change in pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
cryptoe commented on a change in pull request #11723:
URL: https://github.com/apache/druid/pull/11723#discussion_r730574530
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/overlord/sampler/SamplerExceptionMapper.java
##########
@@ -28,13 +29,21 @@
@Provider
public class SamplerExceptionMapper implements ExceptionMapper<SamplerException>
{
+ private static final Logger LOG = new Logger(SamplerExceptionMapper.class);
+
@Override
public Response toResponse(SamplerException exception)
{
+ String message = exception.getMessage() == null ? "The sampler encountered an issue" : exception.getMessage();
+
+ // stack trace is not returned to the client,
Review comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cryptoe commented on a change in pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
cryptoe commented on a change in pull request #11723:
URL: https://github.com/apache/druid/pull/11723#discussion_r728867499
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamSamplerSpec.java
##########
@@ -92,9 +93,22 @@ public SamplerResponse sample()
);
inputFormat = null;
} else {
+ RecordSupplier<PartitionIdType, SequenceOffsetType, RecordType> recordSupplier;
+
+ try {
+ recordSupplier = createRecordSupplier();
+ }
+ catch (Exception e) {
+ if (e.getCause() != null) {
+ throw new SamplerException(e, "Unable to sample data: %s. Cause: %s", e.getMessage(), e.getCause().getMessage());
Review comment:
Nit: Can we change it to say 'unable to create Record Supplier' as this call starts a kakfa/kinessis consumer which can fail due to N reasons.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cryptoe commented on pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
cryptoe commented on pull request #11723:
URL: https://github.com/apache/druid/pull/11723#issuecomment-945371919
lgtm !!
Thanks @FrankChen021 !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cryptoe commented on a change in pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
cryptoe commented on a change in pull request #11723:
URL: https://github.com/apache/druid/pull/11723#discussion_r730574387
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/overlord/sampler/SamplerExceptionMapper.java
##########
@@ -28,13 +29,21 @@
@Provider
public class SamplerExceptionMapper implements ExceptionMapper<SamplerException>
{
+ private static final Logger LOG = new Logger(SamplerExceptionMapper.class);
+
@Override
public Response toResponse(SamplerException exception)
{
+ String message = exception.getMessage() == null ? "The sampler encountered an issue" : exception.getMessage();
+
+ // stack trace is not returned to the client,
+ // so the exception should be logged so that we know where the exception is thrown
Review comment:
```suggestion
// Logging the stack trace and returning the exception message in the response
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a change in pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on a change in pull request #11723:
URL: https://github.com/apache/druid/pull/11723#discussion_r716017391
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamSamplerSpec.java
##########
@@ -92,9 +93,22 @@ public SamplerResponse sample()
);
inputFormat = null;
} else {
+ RecordSupplier<PartitionIdType, SequenceOffsetType, RecordType> recordSupplier;
+
+ try {
+ recordSupplier = createRecordSupplier();
+ }
+ catch (Exception e) {
Review comment:
There're also other code that throw SamplerException, so I think `SamplerExceptionMapper` is a good place to handle the exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 merged pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
FrankChen021 merged pull request #11723:
URL: https://github.com/apache/druid/pull/11723
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on pull request #11723:
URL: https://github.com/apache/druid/pull/11723#issuecomment-985524999
CI reports lower branch coverage, though there's new UT case added. I think it's OK to merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a change in pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on a change in pull request #11723:
URL: https://github.com/apache/druid/pull/11723#discussion_r730412089
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/overlord/sampler/SamplerExceptionMapper.java
##########
@@ -28,13 +29,21 @@
@Provider
public class SamplerExceptionMapper implements ExceptionMapper<SamplerException>
{
+ private static final Logger LOG = new Logger(SamplerExceptionMapper.class);
+
@Override
public Response toResponse(SamplerException exception)
{
+ String message = exception.getMessage() == null ? "The sampler encountered an issue" : exception.getMessage();
+
+ // stack trace is not returned to the client,
+ // so the cause of the exception should be logged so that we know where the exception is thrown
+ LOG.error(exception.getCause(), message);
Review comment:
fixed.
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamSamplerSpec.java
##########
@@ -92,9 +93,22 @@ public SamplerResponse sample()
);
inputFormat = null;
} else {
+ RecordSupplier<PartitionIdType, SequenceOffsetType, RecordType> recordSupplier;
+
+ try {
+ recordSupplier = createRecordSupplier();
+ }
+ catch (Exception e) {
+ if (e.getCause() != null) {
+ throw new SamplerException(e, "Unable to sample data: %s. Cause: %s", e.getMessage(), e.getCause().getMessage());
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on pull request #11723:
URL: https://github.com/apache/druid/pull/11723#issuecomment-955161993
Hi @abhishekagarwal87 , do you have any other comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] asdf2014 commented on a change in pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on a change in pull request #11723:
URL: https://github.com/apache/druid/pull/11723#discussion_r754791590
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamSamplerSpec.java
##########
@@ -92,9 +93,22 @@ public SamplerResponse sample()
);
inputFormat = null;
} else {
+ RecordSupplier<PartitionIdType, SequenceOffsetType, RecordType> recordSupplier;
+
+ try {
+ recordSupplier = createRecordSupplier();
+ }
+ catch (Exception e) {
+ if (e.getCause() != null) {
+ throw new SamplerException(e, "Unable to create RecordSupplier: %s. Cause: %s", e.getMessage(), e.getCause().getMessage());
Review comment:
How about using this Druid style, as follows:
```suggestion
throw new SamplerException(e, "Unable to create RecordSupplier: %s because: %s", e.getMessage(), e.getCause().getMessage());
```
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamSamplerSpec.java
##########
@@ -92,9 +93,22 @@ public SamplerResponse sample()
);
inputFormat = null;
} else {
+ RecordSupplier<PartitionIdType, SequenceOffsetType, RecordType> recordSupplier;
+
+ try {
+ recordSupplier = createRecordSupplier();
+ }
+ catch (Exception e) {
+ if (e.getCause() != null) {
+ throw new SamplerException(e, "Unable to create RecordSupplier: %s. Cause: %s", e.getMessage(), e.getCause().getMessage());
Review comment:
It's very useful to print the cause, but in this situation, I might want to know the root cause, so maybe we can use `Throwables.getRootCause(xxx)` instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] abhishekagarwal87 commented on a change in pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
abhishekagarwal87 commented on a change in pull request #11723:
URL: https://github.com/apache/druid/pull/11723#discussion_r712189972
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamSamplerSpec.java
##########
@@ -92,9 +93,22 @@ public SamplerResponse sample()
);
inputFormat = null;
} else {
+ RecordSupplier<PartitionIdType, SequenceOffsetType, RecordType> recordSupplier;
+
+ try {
+ recordSupplier = createRecordSupplier();
+ }
+ catch (Exception e) {
Review comment:
I think we should log the stack trace here instead of doing it in `toResponse`. This way, it is being logged closer to where the exception is getting generated from.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] cryptoe commented on a change in pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
cryptoe commented on a change in pull request #11723:
URL: https://github.com/apache/druid/pull/11723#discussion_r728864819
##########
File path: indexing-service/src/main/java/org/apache/druid/indexing/overlord/sampler/SamplerExceptionMapper.java
##########
@@ -28,13 +29,21 @@
@Provider
public class SamplerExceptionMapper implements ExceptionMapper<SamplerException>
{
+ private static final Logger LOG = new Logger(SamplerExceptionMapper.class);
+
@Override
public Response toResponse(SamplerException exception)
{
+ String message = exception.getMessage() == null ? "The sampler encountered an issue" : exception.getMessage();
+
+ // stack trace is not returned to the client,
+ // so the cause of the exception should be logged so that we know where the exception is thrown
+ LOG.error(exception.getCause(), message);
Review comment:
log the entire exception so that we get the stack trace ?
We can return only the message to the UI but IMHO we should log the entire stack trace on the server side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 closed pull request #11723: Improve exception message when loading data from web-console
Posted by GitBox <gi...@apache.org>.
FrankChen021 closed pull request #11723:
URL: https://github.com/apache/druid/pull/11723
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org