[jira] [Commented] (SQOOP-1295) Sqoop Merge supports only one column merge key. Merge fails if source table as composite key

["Ajay Chitre (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SQOOP-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940850#comment-13940850 ] 

Ajay Chitre commented on SQOOP-1295:
------------------------------------

Hello,

Virendra (Singh) & I ran into this issue while working on a project at Cisco.  We are importing some tables from Teradata that have UPI (unique primary key) which is made up of more than one column.  When we tried to use:

--merge-key ‘column1, column2’

We got an error saying: “Cannot join values on null key. Did you specify a key column that exists?”

To get around this, we added a method (getKeyObject) in MergeMapperBase given below that takes in a comma separated list.  This is being called in ‘processRecord’ method as follows:

String keyObj = getKeyObject(fieldMap, keyColName, "\u0001");

We’ve attached two files (DlMergeMapperBase.java & DlMergeMapperBase_Test) for your review.  If the ‘Dev’ team thinks this is useful we will submit a patch.

Thanks.

- Ajay Chitre
achitre@cisco.com

Virendra Singh
virsingh@cisco.com


    public String getKeyObject(Map<String, Object> fieldMap, String mergeKey, String fieldSeparator)
            throws IOException {
        assert (fieldMap != null);
        assert (mergeKey != null);
        assert (fieldSeparator != null);

        String key = "";
        String columns[] = mergeKey.split(",");
        for (int i = 0; i < columns.length; i++) {
            if (i != 0) {
                key += fieldSeparator;
            }
            Object value = fieldMap.get(columns[i].trim());
            if (value == null) {
                throw new IOException("Cannot join values on null key. "
                        + "Did you specify a key column that exists?");
            }
            key += value.toString();
        }
        return key;
    }


> Sqoop Merge supports only one column merge key. Merge fails if source table as composite key
> --------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-1295
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1295
>             Project: Sqoop
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 1.4.4
>         Environment: Linux
>            Reporter: Virendra Singh
>             Fix For: 1.4.5
>
>         Attachments: DlMergeMapperBase.java, DlMergeMapperBase_Test.java
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)