You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2019/11/18 16:20:00 UTC

[jira] [Commented] (IMPALA-9163) Log more information about duplicate row errors when inserting into Kudu

    [ https://issues.apache.org/jira/browse/IMPALA-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976675#comment-16976675 ] 

Tim Armstrong commented on IMPALA-9163:
---------------------------------------

Agree it'd be nice to provide at least some of the key values. It's probably easiest to continue using the existing error log mechanism, since that doesn't require client 

{noformat}
[localhost:21000] functional_kudu> insert into functional_kudu.alltypes  select * from functional_kudu.alltypes;
Query: insert into functional_kudu.alltypes  select * from functional_kudu.alltypes
Query submitted at: 2019-11-18 08:12:25 (Coordinator: http://tarmstrong-box:25000)
Query progress can be monitored at: http://tarmstrong-box:25000/query_plan?query_id=d84bce52b99feeae:af775de000000000
WARNINGS: Key already present in Kudu table 'impala::functional_kudu.alltypes'. (1 of 7300 similar)


Modified 0 row(s), 7300 row error(s) in 0.22s
[localhost:21000] functional_kudu> 
{noformat}

I wonder if we could tweak the error merging so that it preserved some of the original error messages instead of immediately merging them all. We'd also need to include the key in the error message. I think we need to be careful with that actually, since we don't want user data to get logged into the INFO logs and in some cases status construction can log the message.

Here's the error message template:
https://github.com/apache/impala/blob/master/common/thrift/generate_error_codes.py#L290

Merging the errors:
https://github.com/apache/impala/blob/c47fca5/be/src/util/error-util.cc#L203

Output of merged error:
https://github.com/apache/impala/blob/c47fca5/be/src/util/error-util.cc#L156

> Log more information about duplicate row errors when inserting into Kudu
> ------------------------------------------------------------------------
>
>                 Key: IMPALA-9163
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9163
>             Project: IMPALA
>          Issue Type: Wish
>            Reporter: Andrew Wong
>            Priority: Major
>              Labels: kudu, usability
>
> When inserting rows into a Kudu table wherein some row keys already exist, depending on the expectations of the dataset, it might be nice to know which write operations were rejected due to a duplicate key error. Today, when inserting such rows through Hue or Impala shell, users are met with a more general error:
> {{Key already present in Kudu table 'default.loadgen_auto_157eac2da1dc4df2824c9a1d51bb3f3f'.}}
> While this nicely avoids excessive error messages (per IMPALA-3704) when there are many duplicate rows, in cases where few duplicate rows are expected, knowing exactly which rows offended the uniqueness constraint might be nice instead. Not sure what exact form factor this might take, but it seems like it'd be a usability win in some cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org