You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Jarek Cecho <ja...@apache.org> on 2012/10/31 01:26:46 UTC
Review Request: SQOOP-656: End to end submission engine
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7783/
-----------------------------------------------------------
Review request for Sqoop.
Description
-------
This is sort of mega patch that is implementing first (not exactly stable and finished) version of submission engine. I've got like dozen other notes what to improve and implement, but the patch is extreme in size already. So I would like to propose getting it in, to give others chance to see what I'm working on at the moment and also retrieve their feedback. In meantime I'll continue my effort to make it very stable and very "nice" :-) My justification for doing that is that since entire sqoop 2 is alpha, it doesn't hurt committing alpha submission engine :-)
Submission engine is asynchronous pluggable interface for submitting mapreduce jobs. I've provided default mapreduce implementation (implementation that directly talks to JobTracker).
Procedure to get it working on dev box:
* Configure your hadoop cluster as usual
* Put config files to /etc/hadoop/conf or reconfigure sqoop to use different directory
* Put all required hadoop jars to tomcat lib folder (issue with resolving jar dependency is still open, so this is like temporal hack)
* Use client command submission to launch actual mapreduce jobs
This addresses bug SQOOP-656.
https://issues.apache.org/jira/browse/SQOOP-656
Diffs
-----
client/src/main/java/org/apache/sqoop/client/core/Environment.java cb92998d2054c8c7adbbf8fb1764fd5269d474b9
client/src/main/java/org/apache/sqoop/client/core/RequestCache.java 03b1eb001f23aa5b3992a479ca9c2fc65136a0bd
client/src/main/java/org/apache/sqoop/client/request/Request.java b243dfdfe4421236f865c50529a553314ff5b1b3
client/src/main/java/org/apache/sqoop/client/request/SubmissionRequest.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java ea1e5d811da9fd817e20363f26498795b29985f8
client/src/main/java/org/apache/sqoop/client/shell/SqoopShell.java c9ac848812ec2ae1ed4f1d15daff9eb6a0e19425
client/src/main/java/org/apache/sqoop/client/shell/SubmissionCommand.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/shell/SubmissionStatusFunction.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/shell/SubmissionStopFunction.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java.orig d2fb6cecdf8a6baed02705bf17abf9a788b0c4c0
client/src/main/java/org/apache/sqoop/client/utils/SubmissionDisplayer.java PRE-CREATION
client/src/main/java/org/apache/sqoop/client/utils/ThrowableDisplayer.java 8d6e9b4814a53270e0a1920ce870ad0bef355e47
common/src/main/java/org/apache/sqoop/common/ImmutableContext.java PRE-CREATION
common/src/main/java/org/apache/sqoop/common/MapContext.java PRE-CREATION
common/src/main/java/org/apache/sqoop/common/MutableContext.java PRE-CREATION
common/src/main/java/org/apache/sqoop/common/MutableMapContext.java PRE-CREATION
common/src/main/java/org/apache/sqoop/json/JobBean.java 7fd2ce52b7c563ad8570d8ebf88b2143ff5aea29
common/src/main/java/org/apache/sqoop/json/SubmissionBean.java PRE-CREATION
common/src/main/java/org/apache/sqoop/json/ThrowableBean.java 91914e84eee4c3e4ee8dc38a1449e51b54e2d2b2
common/src/main/java/org/apache/sqoop/model/FormUtils.java 424810c1b36c4b8c5ebca143121a151e499d8231
common/src/main/java/org/apache/sqoop/model/MSubmission.java PRE-CREATION
common/src/main/java/org/apache/sqoop/submission/SubmissionStatus.java PRE-CREATION
common/src/main/java/org/apache/sqoop/submission/counter/Counter.java PRE-CREATION
common/src/main/java/org/apache/sqoop/submission/counter/CounterGroup.java PRE-CREATION
common/src/main/java/org/apache/sqoop/submission/counter/Counters.java PRE-CREATION
common/src/main/java/org/apache/sqoop/utils/ClassLoadingUtils.java 0c7d09f189c1a2fa96e9c5b782f6693e30cdd62a
common/src/main/java/org/apache/sqoop/utils/ClassUtils.java PRE-CREATION
common/src/test/java/org/apache/sqoop/json/TestSubmissionBean.java PRE-CREATION
common/src/test/java/org/apache/sqoop/json/TestThrowableBean.java 19a0a27764f957998a7256c98126d2d0d29d1544
common/src/test/java/org/apache/sqoop/utils/TestClassLoadingUtils.java e56e017e40bc21e84844ebb20ae72fc81c52eb92
common/src/test/java/org/apache/sqoop/utils/TestClassUtils.java PRE-CREATION
connector/connector-generic-jdbc/pom.xml be4cedd23210d9a5405d4f5ac5f4ee4fbeececf8
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExecutor.java 702dd7e3ff50a203d7d5107de7178cc5517a29eb
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportDestroyer.java cba8f71f59df7068d6d043509c1cfc8927420aa8
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java a21dc7647e1b405510e050e8006435a293cd3b05
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java 0289e29e041527f189a87c44e8e4737a27f83b95
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportDestroyer.java b0a0f62fc85c6284e346f68e7dfc62a05bb52623
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 4499fda0e31ff4d01b30b8e39fefda9e37c2912f
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportInitializer.java 75f3e56a1077cda06c4b3d711c37a84ea7419c15
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartition.java cba313b43075713fb61c3f6661ceeac2c5a94b9e
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java b741b74186ffdf3047f7a0f1f9ace62f5cb865d6
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ConnectionConfiguration.java 212bdf3f41d3ad2ab5f95f0913b1b089f22829ab
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ExportJobConfiguration.java 91004cf61d954d9e0381573d7543aac0ee10d892
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ImportJobConfiguration.java 31ce7777e2f2226787f8a10e96c7a002fbe0413e
connector/connector-generic-jdbc/src/main/resources/generic-jdbc-connector-resources.properties 9c8e290bf2dc56b35081ca5501d8a49ef70831b9
connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportExtractor.java 70e29e534dffb79dc478876049b26c7ae66beed4
connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportInitializer.java 546559348e3b4fa49915565c1742732230252ff9
connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportPartitioner.java 0e95a43e59dc74f864f190df8f225afa0076a4a5
core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 273b4864457ad237c3b5eca9fbb0d98fdbeea993
core/src/main/java/org/apache/sqoop/connector/ConnectorManager.java f7228d37e6642e30fce2cba29ee1596fd48a44f5
core/src/main/java/org/apache/sqoop/core/Context.java 6eeed137581d3f014b8deeb2d9f9951ec622f0c4
core/src/main/java/org/apache/sqoop/core/SqoopConfiguration.java 47340f97251e9b9cbce429ec9286c788067662bd
core/src/main/java/org/apache/sqoop/framework/FrameworkConstants.java 662a50c77360ae57e9d39a3642dc178585be2cfd
core/src/main/java/org/apache/sqoop/framework/FrameworkError.java e0d91d4cee29cea8c6da5a67967f17ca8ab84345
core/src/main/java/org/apache/sqoop/framework/FrameworkManager.java c243275cc7d9bc5473144f18bd68a807e5b11462
core/src/main/java/org/apache/sqoop/framework/SubmissionEngine.java PRE-CREATION
core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java PRE-CREATION
core/src/main/java/org/apache/sqoop/framework/configuration/ImportJobConfiguration.java 1d1466126eb347f181bef167b8c4ebc51e469a09
core/src/main/java/org/apache/sqoop/job/JobConstants.java 2b0ec18b96429e92b3d84595a1c6fe4223d4bdf5
core/src/main/java/org/apache/sqoop/job/JobEngine.java fa3e484ad24e116113cbe9af24dc312f3a158efd
core/src/main/java/org/apache/sqoop/job/PrefixContext.java PRE-CREATION
core/src/main/java/org/apache/sqoop/job/etl/EtlContext.java 09eca58d170492e7621264b35b0cfeec52eb07ac
core/src/main/java/org/apache/sqoop/job/etl/EtlFramework.java ce7f9884961ddfac77f0973194686a1f7ba35c3c
core/src/main/java/org/apache/sqoop/job/etl/EtlMutableContext.java e1119567d899f8c234b14eb40efcd531336a9a9e
core/src/main/java/org/apache/sqoop/job/etl/EtlOptions.java e45c0ff4a2185f04f2631aeeacfff85a48904278
core/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceImportLoader.java 854d325ed3336f7f4b69eec19576fdfc2e826621
core/src/main/java/org/apache/sqoop/job/etl/HdfsTextImportLoader.java 240265bd8a6a314b826b18b5c930adacd9c4a18c
core/src/main/java/org/apache/sqoop/job/mr/ConfigurationUtils.java PRE-CREATION
core/src/main/java/org/apache/sqoop/job/mr/MrExecution.java bd4c108dbc84be65bdc9960a674631168173d9c4
core/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java f5ae3df1510c2e4f725ccc5d252a71346a0a93ec
core/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 0a9f46d48754dffe2034c63f3736f20dfb3a9c9e
core/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java 23fcb6254c2437ce74813dd84601c090341d9669
core/src/main/java/org/apache/sqoop/job/mr/SqoopSplit.java f936f6e876785b747a7d3ee600e1d641fb448792
core/src/main/java/org/apache/sqoop/repository/JdbcRepository.java 8d7b95cd55ba65b1f597baf7aa3cf96cd830a6cf
core/src/main/java/org/apache/sqoop/repository/JdbcRepositoryContext.java b2274d335fc9bba919b71e89a7d2661a0b5356a9
core/src/main/java/org/apache/sqoop/repository/JdbcRepositoryHandler.java b0c9780bbca33f3fbd3622492576f4074f2ffad7
core/src/main/java/org/apache/sqoop/repository/JdbcRepositoryProvider.java e7b6771a715e2b92728bbd70b72f7f30b6c0c68e
core/src/main/java/org/apache/sqoop/repository/Repository.java ec746d16d5cccd355a917c3b9eb1a0fa45c7cad5
core/src/main/java/org/apache/sqoop/repository/RepositoryError.java ff53b1302dce25526ead7c0d15ece2ff2c3b9b5d
core/src/main/java/org/apache/sqoop/repository/RepositoryManager.java 0fbeeb3d6e5c8b3d5d5e0890f04763425b542970
core/src/main/java/org/apache/sqoop/repository/RepositoryProvider.java 1b5d00dc6b8006b845462a5f2ee6e79983622fcf
core/src/main/resources/framework-resources.properties 4706cf48ce5f0c1cdd1e06d1d2a5a1b6f97a2838
core/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java 64c767cf97e5cd5b321fc7e5c5caa55b25bcc5d8
core/src/test/java/org/apache/sqoop/job/TestJobEngine.java e653c22e6494d3d3e9449716170c057133384adf
core/src/test/java/org/apache/sqoop/job/TestMapReduce.java 7646f5732dc45cb18985c1c60c47c35185aaa83d
dist/src/main/server/conf/sqoop.properties 882191dbbf8ff8ddd68bdf4de61584281150b63c
pom.xml eea0350d5e0d4ed5dd428ebcaee39304c403baef
repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepoError.java 94119b1f50cc611b767cb20468dce447e45efb36
repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java 0ce8832f9fb57997955113f6dba8b551ad989b7b
repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaConstants.java 95461c9349607ef5ae453172498d43fdc698a300
repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaQuery.java cddace7fd9e5c4fa5a67fdcae845a57b5a9d35e2
repository/repository-derby/src/test/java/org/apache/sqoop/repository/derby/DerbyTestCase.java ae59933e690a06a346849527c4d90f9f5d20f4da
repository/repository-derby/src/test/java/org/apache/sqoop/repository/derby/TestSubmissionHandling.java PRE-CREATION
server/pom.xml 78ad8ee56472bdf5e508c6b641c7fdc92b6aec3a
server/src/main/java/org/apache/sqoop/handler/ConnectionRequestHandler.java eba334e57bc56becc33d13fd606a11a4158c87fc
server/src/main/java/org/apache/sqoop/handler/ConnectorRequestHandler.java fda91fd69dc023fede6add6183a311f54dab4d49
server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java 0589e30f9ecd30ee8b0a54fbd5167ac27f9a24d5
server/src/main/java/org/apache/sqoop/handler/SubmissionRequestHandler.java PRE-CREATION
server/src/main/java/org/apache/sqoop/server/RequestContext.java 78950f6901c40e34f521e4d41c2e9b995f5f34d9
server/src/main/java/org/apache/sqoop/server/ServerInitializer.java 993c153820891d826e191257fa562d7d12852380
server/src/main/java/org/apache/sqoop/server/v1/SubmissionServlet.java PRE-CREATION
server/src/main/webapp/WEB-INF/web.xml 69229bfdb89ffcbba699bf77e12333fcc4af586c
spi/src/main/java/org/apache/sqoop/job/etl/CallbackBase.java PRE-CREATION
spi/src/main/java/org/apache/sqoop/job/etl/Context.java fc01c963c01a0e3f786574457e5ccd7775d66d69
spi/src/main/java/org/apache/sqoop/job/etl/Destroyer.java af766f3bc4554f864e4da903dc9ab2e2d4fe1eb3
spi/src/main/java/org/apache/sqoop/job/etl/Exporter.java ef690bfdc776cba8cab1856e79bb0f93c0e4ba68
spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java 20bdeda28dec0dc5a64e5a40469288efa0d6af09
spi/src/main/java/org/apache/sqoop/job/etl/Importer.java f0a8d1afdf1d32d77bef44be310f5f84d0097d9b
spi/src/main/java/org/apache/sqoop/job/etl/Initializer.java 75bd42e75f4050eaf9fb9459fe9b81526d3c604e
spi/src/main/java/org/apache/sqoop/job/etl/Loader.java 54749274c147bcde7a5e728dd09502a744839a9a
spi/src/main/java/org/apache/sqoop/job/etl/MutableContext.java 03678c58b4d18062ea458b4cc712928eace33ea4
spi/src/main/java/org/apache/sqoop/job/etl/Options.java 2dc467180da9f3d849f45b823aabb49e9f80bf6d
spi/src/main/java/org/apache/sqoop/job/etl/Partition.java 8834c80e54d8993b6db67999c31b2bf0dbaadc43
spi/src/main/java/org/apache/sqoop/job/etl/Partitioner.java 21310be43086631aa943945d0d69ab1ed502b6ff
submission/mapreduce/pom.xml PRE-CREATION
submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/Constants.java PRE-CREATION
submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java PRE-CREATION
submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionError.java PRE-CREATION
submission/pom.xml PRE-CREATION
Diff: https://reviews.apache.org/r/7783/diff/
Testing
-------
I've tested it on real cluster and I was able to move first rows! Otherwise my patch is commenting a lot of tests to disable them for the time being.
Thanks,
Jarek Cecho
Re: Review Request: SQOOP-656: End to end submission engine
Posted by Bilung Lee <bl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7783/#review13067
-----------------------------------------------------------
Ship it!
Ship It!
- Bilung Lee
On Oct. 31, 2012, 12:26 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7783/
> -----------------------------------------------------------
>
> (Updated Oct. 31, 2012, 12:26 a.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> This is sort of mega patch that is implementing first (not exactly stable and finished) version of submission engine. I've got like dozen other notes what to improve and implement, but the patch is extreme in size already. So I would like to propose getting it in, to give others chance to see what I'm working on at the moment and also retrieve their feedback. In meantime I'll continue my effort to make it very stable and very "nice" :-) My justification for doing that is that since entire sqoop 2 is alpha, it doesn't hurt committing alpha submission engine :-)
>
> Submission engine is asynchronous pluggable interface for submitting mapreduce jobs. I've provided default mapreduce implementation (implementation that directly talks to JobTracker).
>
> Procedure to get it working on dev box:
> * Configure your hadoop cluster as usual
> * Put config files to /etc/hadoop/conf or reconfigure sqoop to use different directory
> * Put all required hadoop jars to tomcat lib folder (issue with resolving jar dependency is still open, so this is like temporal hack)
> * Use client command submission to launch actual mapreduce jobs
>
>
> This addresses bug SQOOP-656.
> https://issues.apache.org/jira/browse/SQOOP-656
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/sqoop/client/core/Environment.java cb92998d2054c8c7adbbf8fb1764fd5269d474b9
> client/src/main/java/org/apache/sqoop/client/core/RequestCache.java 03b1eb001f23aa5b3992a479ca9c2fc65136a0bd
> client/src/main/java/org/apache/sqoop/client/request/Request.java b243dfdfe4421236f865c50529a553314ff5b1b3
> client/src/main/java/org/apache/sqoop/client/request/SubmissionRequest.java PRE-CREATION
> client/src/main/java/org/apache/sqoop/client/shell/CreateConnectionFunction.java ea1e5d811da9fd817e20363f26498795b29985f8
> client/src/main/java/org/apache/sqoop/client/shell/SqoopShell.java c9ac848812ec2ae1ed4f1d15daff9eb6a0e19425
> client/src/main/java/org/apache/sqoop/client/shell/SubmissionCommand.java PRE-CREATION
> client/src/main/java/org/apache/sqoop/client/shell/SubmissionStartFunction.java PRE-CREATION
> client/src/main/java/org/apache/sqoop/client/shell/SubmissionStatusFunction.java PRE-CREATION
> client/src/main/java/org/apache/sqoop/client/shell/SubmissionStopFunction.java PRE-CREATION
> client/src/main/java/org/apache/sqoop/client/utils/FormFiller.java.orig d2fb6cecdf8a6baed02705bf17abf9a788b0c4c0
> client/src/main/java/org/apache/sqoop/client/utils/SubmissionDisplayer.java PRE-CREATION
> client/src/main/java/org/apache/sqoop/client/utils/ThrowableDisplayer.java 8d6e9b4814a53270e0a1920ce870ad0bef355e47
> common/src/main/java/org/apache/sqoop/common/ImmutableContext.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/common/MapContext.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/common/MutableContext.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/common/MutableMapContext.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/json/JobBean.java 7fd2ce52b7c563ad8570d8ebf88b2143ff5aea29
> common/src/main/java/org/apache/sqoop/json/SubmissionBean.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/json/ThrowableBean.java 91914e84eee4c3e4ee8dc38a1449e51b54e2d2b2
> common/src/main/java/org/apache/sqoop/model/FormUtils.java 424810c1b36c4b8c5ebca143121a151e499d8231
> common/src/main/java/org/apache/sqoop/model/MSubmission.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/submission/SubmissionStatus.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/submission/counter/Counter.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/submission/counter/CounterGroup.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/submission/counter/Counters.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/utils/ClassLoadingUtils.java 0c7d09f189c1a2fa96e9c5b782f6693e30cdd62a
> common/src/main/java/org/apache/sqoop/utils/ClassUtils.java PRE-CREATION
> common/src/test/java/org/apache/sqoop/json/TestSubmissionBean.java PRE-CREATION
> common/src/test/java/org/apache/sqoop/json/TestThrowableBean.java 19a0a27764f957998a7256c98126d2d0d29d1544
> common/src/test/java/org/apache/sqoop/utils/TestClassLoadingUtils.java e56e017e40bc21e84844ebb20ae72fc81c52eb92
> common/src/test/java/org/apache/sqoop/utils/TestClassUtils.java PRE-CREATION
> connector/connector-generic-jdbc/pom.xml be4cedd23210d9a5405d4f5ac5f4ee4fbeececf8
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExecutor.java 702dd7e3ff50a203d7d5107de7178cc5517a29eb
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportDestroyer.java cba8f71f59df7068d6d043509c1cfc8927420aa8
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportInitializer.java a21dc7647e1b405510e050e8006435a293cd3b05
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java 0289e29e041527f189a87c44e8e4737a27f83b95
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportDestroyer.java b0a0f62fc85c6284e346f68e7dfc62a05bb52623
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 4499fda0e31ff4d01b30b8e39fefda9e37c2912f
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportInitializer.java 75f3e56a1077cda06c4b3d711c37a84ea7419c15
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartition.java cba313b43075713fb61c3f6661ceeac2c5a94b9e
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java b741b74186ffdf3047f7a0f1f9ace62f5cb865d6
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ConnectionConfiguration.java 212bdf3f41d3ad2ab5f95f0913b1b089f22829ab
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ExportJobConfiguration.java 91004cf61d954d9e0381573d7543aac0ee10d892
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ImportJobConfiguration.java 31ce7777e2f2226787f8a10e96c7a002fbe0413e
> connector/connector-generic-jdbc/src/main/resources/generic-jdbc-connector-resources.properties 9c8e290bf2dc56b35081ca5501d8a49ef70831b9
> connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportExtractor.java 70e29e534dffb79dc478876049b26c7ae66beed4
> connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportInitializer.java 546559348e3b4fa49915565c1742732230252ff9
> connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportPartitioner.java 0e95a43e59dc74f864f190df8f225afa0076a4a5
> core/src/main/java/org/apache/sqoop/connector/ConnectorHandler.java 273b4864457ad237c3b5eca9fbb0d98fdbeea993
> core/src/main/java/org/apache/sqoop/connector/ConnectorManager.java f7228d37e6642e30fce2cba29ee1596fd48a44f5
> core/src/main/java/org/apache/sqoop/core/Context.java 6eeed137581d3f014b8deeb2d9f9951ec622f0c4
> core/src/main/java/org/apache/sqoop/core/SqoopConfiguration.java 47340f97251e9b9cbce429ec9286c788067662bd
> core/src/main/java/org/apache/sqoop/framework/FrameworkConstants.java 662a50c77360ae57e9d39a3642dc178585be2cfd
> core/src/main/java/org/apache/sqoop/framework/FrameworkError.java e0d91d4cee29cea8c6da5a67967f17ca8ab84345
> core/src/main/java/org/apache/sqoop/framework/FrameworkManager.java c243275cc7d9bc5473144f18bd68a807e5b11462
> core/src/main/java/org/apache/sqoop/framework/SubmissionEngine.java PRE-CREATION
> core/src/main/java/org/apache/sqoop/framework/SubmissionRequest.java PRE-CREATION
> core/src/main/java/org/apache/sqoop/framework/configuration/ImportJobConfiguration.java 1d1466126eb347f181bef167b8c4ebc51e469a09
> core/src/main/java/org/apache/sqoop/job/JobConstants.java 2b0ec18b96429e92b3d84595a1c6fe4223d4bdf5
> core/src/main/java/org/apache/sqoop/job/JobEngine.java fa3e484ad24e116113cbe9af24dc312f3a158efd
> core/src/main/java/org/apache/sqoop/job/PrefixContext.java PRE-CREATION
> core/src/main/java/org/apache/sqoop/job/etl/EtlContext.java 09eca58d170492e7621264b35b0cfeec52eb07ac
> core/src/main/java/org/apache/sqoop/job/etl/EtlFramework.java ce7f9884961ddfac77f0973194686a1f7ba35c3c
> core/src/main/java/org/apache/sqoop/job/etl/EtlMutableContext.java e1119567d899f8c234b14eb40efcd531336a9a9e
> core/src/main/java/org/apache/sqoop/job/etl/EtlOptions.java e45c0ff4a2185f04f2631aeeacfff85a48904278
> core/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceImportLoader.java 854d325ed3336f7f4b69eec19576fdfc2e826621
> core/src/main/java/org/apache/sqoop/job/etl/HdfsTextImportLoader.java 240265bd8a6a314b826b18b5c930adacd9c4a18c
> core/src/main/java/org/apache/sqoop/job/mr/ConfigurationUtils.java PRE-CREATION
> core/src/main/java/org/apache/sqoop/job/mr/MrExecution.java bd4c108dbc84be65bdc9960a674631168173d9c4
> core/src/main/java/org/apache/sqoop/job/mr/SqoopInputFormat.java f5ae3df1510c2e4f725ccc5d252a71346a0a93ec
> core/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 0a9f46d48754dffe2034c63f3736f20dfb3a9c9e
> core/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java 23fcb6254c2437ce74813dd84601c090341d9669
> core/src/main/java/org/apache/sqoop/job/mr/SqoopSplit.java f936f6e876785b747a7d3ee600e1d641fb448792
> core/src/main/java/org/apache/sqoop/repository/JdbcRepository.java 8d7b95cd55ba65b1f597baf7aa3cf96cd830a6cf
> core/src/main/java/org/apache/sqoop/repository/JdbcRepositoryContext.java b2274d335fc9bba919b71e89a7d2661a0b5356a9
> core/src/main/java/org/apache/sqoop/repository/JdbcRepositoryHandler.java b0c9780bbca33f3fbd3622492576f4074f2ffad7
> core/src/main/java/org/apache/sqoop/repository/JdbcRepositoryProvider.java e7b6771a715e2b92728bbd70b72f7f30b6c0c68e
> core/src/main/java/org/apache/sqoop/repository/Repository.java ec746d16d5cccd355a917c3b9eb1a0fa45c7cad5
> core/src/main/java/org/apache/sqoop/repository/RepositoryError.java ff53b1302dce25526ead7c0d15ece2ff2c3b9b5d
> core/src/main/java/org/apache/sqoop/repository/RepositoryManager.java 0fbeeb3d6e5c8b3d5d5e0890f04763425b542970
> core/src/main/java/org/apache/sqoop/repository/RepositoryProvider.java 1b5d00dc6b8006b845462a5f2ee6e79983622fcf
> core/src/main/resources/framework-resources.properties 4706cf48ce5f0c1cdd1e06d1d2a5a1b6f97a2838
> core/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java 64c767cf97e5cd5b321fc7e5c5caa55b25bcc5d8
> core/src/test/java/org/apache/sqoop/job/TestJobEngine.java e653c22e6494d3d3e9449716170c057133384adf
> core/src/test/java/org/apache/sqoop/job/TestMapReduce.java 7646f5732dc45cb18985c1c60c47c35185aaa83d
> dist/src/main/server/conf/sqoop.properties 882191dbbf8ff8ddd68bdf4de61584281150b63c
> pom.xml eea0350d5e0d4ed5dd428ebcaee39304c403baef
> repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepoError.java 94119b1f50cc611b767cb20468dce447e45efb36
> repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java 0ce8832f9fb57997955113f6dba8b551ad989b7b
> repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaConstants.java 95461c9349607ef5ae453172498d43fdc698a300
> repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaQuery.java cddace7fd9e5c4fa5a67fdcae845a57b5a9d35e2
> repository/repository-derby/src/test/java/org/apache/sqoop/repository/derby/DerbyTestCase.java ae59933e690a06a346849527c4d90f9f5d20f4da
> repository/repository-derby/src/test/java/org/apache/sqoop/repository/derby/TestSubmissionHandling.java PRE-CREATION
> server/pom.xml 78ad8ee56472bdf5e508c6b641c7fdc92b6aec3a
> server/src/main/java/org/apache/sqoop/handler/ConnectionRequestHandler.java eba334e57bc56becc33d13fd606a11a4158c87fc
> server/src/main/java/org/apache/sqoop/handler/ConnectorRequestHandler.java fda91fd69dc023fede6add6183a311f54dab4d49
> server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java 0589e30f9ecd30ee8b0a54fbd5167ac27f9a24d5
> server/src/main/java/org/apache/sqoop/handler/SubmissionRequestHandler.java PRE-CREATION
> server/src/main/java/org/apache/sqoop/server/RequestContext.java 78950f6901c40e34f521e4d41c2e9b995f5f34d9
> server/src/main/java/org/apache/sqoop/server/ServerInitializer.java 993c153820891d826e191257fa562d7d12852380
> server/src/main/java/org/apache/sqoop/server/v1/SubmissionServlet.java PRE-CREATION
> server/src/main/webapp/WEB-INF/web.xml 69229bfdb89ffcbba699bf77e12333fcc4af586c
> spi/src/main/java/org/apache/sqoop/job/etl/CallbackBase.java PRE-CREATION
> spi/src/main/java/org/apache/sqoop/job/etl/Context.java fc01c963c01a0e3f786574457e5ccd7775d66d69
> spi/src/main/java/org/apache/sqoop/job/etl/Destroyer.java af766f3bc4554f864e4da903dc9ab2e2d4fe1eb3
> spi/src/main/java/org/apache/sqoop/job/etl/Exporter.java ef690bfdc776cba8cab1856e79bb0f93c0e4ba68
> spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java 20bdeda28dec0dc5a64e5a40469288efa0d6af09
> spi/src/main/java/org/apache/sqoop/job/etl/Importer.java f0a8d1afdf1d32d77bef44be310f5f84d0097d9b
> spi/src/main/java/org/apache/sqoop/job/etl/Initializer.java 75bd42e75f4050eaf9fb9459fe9b81526d3c604e
> spi/src/main/java/org/apache/sqoop/job/etl/Loader.java 54749274c147bcde7a5e728dd09502a744839a9a
> spi/src/main/java/org/apache/sqoop/job/etl/MutableContext.java 03678c58b4d18062ea458b4cc712928eace33ea4
> spi/src/main/java/org/apache/sqoop/job/etl/Options.java 2dc467180da9f3d849f45b823aabb49e9f80bf6d
> spi/src/main/java/org/apache/sqoop/job/etl/Partition.java 8834c80e54d8993b6db67999c31b2bf0dbaadc43
> spi/src/main/java/org/apache/sqoop/job/etl/Partitioner.java 21310be43086631aa943945d0d69ab1ed502b6ff
> submission/mapreduce/pom.xml PRE-CREATION
> submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/Constants.java PRE-CREATION
> submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java PRE-CREATION
> submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionError.java PRE-CREATION
> submission/pom.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/7783/diff/
>
>
> Testing
> -------
>
> I've tested it on real cluster and I was able to move first rows! Otherwise my patch is commenting a lot of tests to disable them for the time being.
>
>
> Thanks,
>
> Jarek Cecho
>
>