You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by kevin00chen <gi...@git.apache.org> on 2016/08/22 14:52:01 UTC
[GitHub] sqoop pull request #26: [SQOOP-3002] sqoop merge tool composite merge-key
GitHub user kevin00chen opened a pull request:
https://github.com/apache/sqoop/pull/26
[SQOOP-3002] sqoop merge tool composite merge-key
JIRA Issue:https://issues.apache.org/jira/browse/SQOOP-3002
Sqoop Merge Tool can just specify one column by using --merge-key argument.
When i need to specify two or more column, i need to modify some source code
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kevin00chen/sqoop my_change
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/sqoop/pull/26.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #26
----
commit cd1e840c8dfb6261aa3be81b9c4881e80bc038bd
Author: KaimingChen <ka...@163.com>
Date: 2016-08-22T14:34:45Z
sqoop merge tool composite merge-key
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] sqoop pull request #26: [SQOOP-3002] sqoop merge tool composite merge-key
Posted by kevin00chen <gi...@git.apache.org>.
Github user kevin00chen commented on a diff in the pull request:
https://github.com/apache/sqoop/pull/26#discussion_r75698508
--- Diff: src/java/org/apache/sqoop/mapreduce/MergeMapperBase.java ---
@@ -76,9 +76,10 @@ protected void processRecord(SqoopRecord r, Context c)
}
Object keyObj = null;
if (keyColName.contains(",")) {
+ String connectStr = new String(new byte[]{1});
StringBuilder keyFieldsSb = new StringBuilder();
for (String str : keyColName.split(",")) {
- keyFieldsSb.append("+").append(fieldMap.get(str).toString());
+ keyFieldsSb.append(connectStr).append(fieldMap.get(str).toString());
--- End diff --
for example one table has two column, a and b
Field a | Field b
------------ | -------------
a+ | b
a | +b
when use "+" to connect two field, two record will has same keyObj.
To avoid this i use a String contains one byte.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] sqoop pull request #26: [SQOOP-3002] sqoop merge tool composite merge-key
Posted by kevin00chen <gi...@git.apache.org>.
Github user kevin00chen commented on a diff in the pull request:
https://github.com/apache/sqoop/pull/26#discussion_r75698166
--- Diff: src/java/org/apache/sqoop/mapreduce/MergeMapperBase.java ---
@@ -76,9 +76,10 @@ protected void processRecord(SqoopRecord r, Context c)
}
Object keyObj = null;
if (keyColName.contains(",")) {
+ String connectStr = new String(new byte[]{1});
StringBuilder keyFieldsSb = new StringBuilder();
for (String str : keyColName.split(",")) {
- keyFieldsSb.append("+").append(fieldMap.get(str).toString());
+ keyFieldsSb.append(connectStr).append(fieldMap.get(str).toString());
--- End diff --
for example one table has two column, a and b
Field a | Field b
------------ | -------------
a+ | b
a | +b
when use "+" to connect two field, two record will has same keyObj.
To avoid this i use a String contains one byte.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] sqoop issue #26: [SQOOP-3002] sqoop merge tool composite merge-key
Posted by liz-z17 <gi...@git.apache.org>.
Github user liz-z17 commented on the issue:
https://github.com/apache/sqoop/pull/26
Hi @kevin00chen ,
Sqoop project currently does not accept pull requests. To contribute, you will need to generate patch and upload it to the JIRA (of course if there's no corresponding JIRA issue, you will also need to create one first).
See the instructions here: https://cwiki.apache.org/confluence/display/SQOOP/How+to+Contribute
If you need more directions, I'd be happy to help!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---