You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by qianmoQ <gi...@git.apache.org> on 2016/12/21 08:39:53 UTC
[GitHub] sqoop pull request #32: Branch 1.4.6
GitHub user qianmoQ opened a pull request:
https://github.com/apache/sqoop/pull/32
Branch 1.4.6
Sqoop Version 1.4.6
i use sqoop import parquet file to hive when hadoop ha
`
2016-12-20 20:51:52,249 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1482237596040_0003_000002
2016-12-20 20:51:52,447 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-12-20 20:51:52,476 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2016-12-20 20:51:52,476 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 3 cluster_timestamp: 1482237596040 } attemptId: 2 } keyId: 475466983)
2016-12-20 20:51:52,675 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2016-12-20 20:51:52,676 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2016-12-20 20:51:52,728 WARN [main] org.kitesdk.data.spi.Registration: Not loading URI patterns in org.kitesdk.data.spi.hive.Loader
2016-12-20 20:51:52,939 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.IllegalArgumentException: java.net.UnknownHostException: chorus
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.IllegalArgumentException: java.net.UnknownHostException: chorus
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:519)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:499)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1598)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:499)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:285)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1556)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
Caused by: java.lang.IllegalArgumentException: java.net.UnknownHostException: chorus
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:411)
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311)
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:150)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
at org.kitesdk.data.spi.filesystem.Loader$URIBuilder.getFromOptions(Loader.java:66)
at org.kitesdk.data.spi.filesystem.Loader$URIBuilder.getFromOptions(Loader.java:50)
at org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:106)
at org.kitesdk.data.Datasets.load(Datasets.java:103)
at org.kitesdk.data.Datasets.load(Datasets.java:165)
at org.kitesdk.data.mapreduce.DatasetKeyOutputFormat.load(DatasetKeyOutputFormat.java:510)
at org.kitesdk.data.mapreduce.DatasetKeyOutputFormat.getOutputCommitter(DatasetKeyOutputFormat.java:473)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:517)
... 11 more
Caused by: java.net.UnknownHostException: chorus
... 30 more
`
execute shell script:
`
sqoop import --connect jdbc:mysql://localhost:3306/chorus --username chorus --password 'chorus' --hive-database default --table project_info --target-dir /apps/hive/warehouse/chorus_mbx_project.db/mbxtest2 --columns project_id,project_code,project_name,project_desc --m 1 --append --as-parquetfile
`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/sqoop branch-1.4.6
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/sqoop/pull/32.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #32
----
commit c2317fd064243a6cabcb35534f157d0127d63e23
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-03-17T16:00:20Z
SQOOP-2176: Update change log with 1.4.6 release
(Qian Xu via Gwen Shapira)
commit 08c98bd7cefb1663b135d4c1604c8d3b41931ead
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-03-23T02:15:29Z
SQOOP-2177: Update build file and docs with new version
(Qian Xu via Gwen Shapira)
commit 847f3f13c144051a9ee183265ba8ee1ec84e8e4e
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-03-24T13:43:30Z
SQOOP-2257: Parquet target for imports with Hive overwrite option does not work
(Qian Xu via Jarek Jarcec Cecho)
(cherry picked from commit fa8518998f527653587c8f74ebf9990c22842dbb)
commit 1c826339364638a9db2b87c60359f3e9912a21bb
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-03-24T23:46:34Z
SQOOP-2254: Failed to build release notes
(Qian Xu via Gwen Shapira)
(cherry picked from commit bbfa59a61cf7835034823d153817ecc5ab472e3c)
commit e28452a84efcc2bd24f1eb867098e39f6efecdd5
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-03-24T23:54:26Z
SQOOP-2259: Update change log with 1.4.6 release (2nd)
(Qian Xu via Gwen Shapira)
(cherry picked from commit eaf03baab5ce5431b5cfecc60ff63ffe8a1f3331)
commit 2e75a215c6419bd3b3fabc4429d0b6fa785939aa
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-03-25T16:55:46Z
SQOOP-2263: Sqoop1 has some files without a copyright header
(Qian Xu via Gwen Shapira)
(cherry picked from commit 1a6a83607d7e725c60a72573587827bd615f248a)
commit 9348d6a63801e952117c3014cf71fdd773ec2c11
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-04-03T16:11:34Z
SQOOP-2282: Add validation check for --hive-import and --append
(Abraham Elmahrek via Jarek Jarcec Cecho)
(cherry picked from commit 64c5fc3755bb26b48f515e1c2da6c5c5249e2b76)
commit 75c7b5ab472649bf75bab1aac1db0155e691ff64
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-04-03T16:45:55Z
SQOOP-2281: Set overwrite on kite dataset
(Abraham Elmahrek via Jarek Jarcec Cecho)
(cherry picked from commit 032bc54c86c79f2f032211ce6505ff9c9fbce31e)
commit d2f6c84d34ac9f31cc890f6164deb52e20e7a8a4
Author: Abraham Elmahrek <ab...@apache.org>
Date: 2015-04-06T22:44:03Z
SQOOP-2284: Update license file
(Qian Xu via Abraham Elmahrek)
commit 621be4a6b7f49020f67ac738249bd04779ad00d1
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-04-08T02:15:59Z
SQOOP-2283: Support usage of --exec and --password-alias
(Abraham Elmahrek via Gwen Shapira)
(cherry picked from commit d32137f15c06827dea50b293f4d422481cb374cd)
commit b87b994c571cc509d9bca060356767b5621cf0fe
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-04-08T02:39:02Z
SQOOP-2286: Ensure Sqoop generates valid avro column names
(Abraham Elmahrek via Gwen Shapira)
(cherry picked from commit baf51351281842bd660572fcc05c89d6407913c5)
commit 5270908b3e3c0429855ac316abc7cec52fc3dbf3
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-04-09T00:57:57Z
SQOOP-2252: Add default to Avro Schema
(Andrew Stevenson via Jarek Jarcec Cecho)
(cherry picked from commit 8050107ecf0df6d8dc3869f6c82fe84ee098f08a)
commit 739733853e2daeef9e76aa1240b8b552bc881261
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-04-09T03:14:19Z
SQOOP-2264: Exclude and remove SqoopUserGuide.xml from git repository
(Qian Xu via Jarek Jarcec Cecho)
(cherry picked from commit c51c7daeeda1704375ceacb97fc5d5793a6b9f71)
Conflicts:
src/docs/user/SqoopUserGuide.xml
commit 73081a5614fc330345fbe10c6b772c5a6eab4b76
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-04-09T14:58:02Z
SQOOP-2290: java.lang.ArrayIndexOutOfBoundsException thrown when malformed column mapping is provided
(Venkat Ranganathan via Gwen Shapira)
(cherry picked from commit 292fd08b7e1f64756fd132f790e11605bfea9ea1)
commit 08c61a11e10bbe15a789fa12c8780c182b235de8
Author: Gwen Shapira <cs...@gmail.com>
Date: 2015-04-09T15:12:59Z
SQOOP-2291: Create rc1 release note
(Qian Xu via Gwen Shapira)
(cherry picked from commit 29c1ae8adc4e42b3c55a3fecbb657002392544be)
commit 67e8c9a9e8048dee227682e3fc76aedae1d100de
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-04-21T14:02:37Z
SQOOP-2294: Change to Avro schema name breaks some use cases
(Qian Xu via Jarek Jarcec Cecho)
commit a74089dece04a483ce6510b62949e00e6f120705
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-04-22T02:43:38Z
SQOOP-2317: Update change log with 1.4.6 release (rc2)
(Qian Xu via Jarek Jarcec Cecho)
commit 957d65de7282ac54796b8cd90de4f14b6949bf49
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-04-24T04:09:51Z
SQOOP-2324: Remove extra license handling for consistency
(Qian Xu via Jarek Jarcec Cecho)
commit c0c5a81723759fa575844a0a1eae8f510fa32c25
Author: Jarek Jarcec Cecho <ja...@apache.org>
Date: 2015-04-24T16:15:41Z
SQOOP-2325: Update change log for 1.4.6 release (rc3)
(Qian Xu via Jarek Jarcec Cecho)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] sqoop issue #32: i use sqoop import parquet file to hive when hadoop ha erro...
Posted by liz-z17 <gi...@git.apache.org>.
Github user liz-z17 commented on the issue:
https://github.com/apache/sqoop/pull/32
Hi @qianmoQ ,
Sqoop project currently does not accept pull requests. To contribute, you will need to generate patch and upload it to the JIRA.
See the instructions here: https://cwiki.apache.org/confluence/display/SQOOP/How+to+Contribute
If you need more directions, I'd be happy to help!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---