You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@oozie.apache.org by Peter Cseh <ge...@cloudera.com> on 2017/01/05 13:16:47 UTC

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated Jan. 5, 2017, 1:16 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
  core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
  core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
  docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 

Diff: https://reviews.apache.org/r/54705/diff/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 1143-1148 (original), 991-1001 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597293#file1597293line1195>
> >
> >     Instead of doing this (which I know is equivalent to what we currently do), would it better to call ````check(...)```` instead?  I think it handles stuff better (e.g. fallback to data file, etc).  Or maybe we can extract out the common code into a new method and have them both call that, if we don't want all of ````check(...)```` here.
> 
> Peter Bacsko wrote:
>     You mean to call check() when the Launcher is already running?

Hmm, I don't remember what I was getting at here; let's ignore this for now.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 1113-1117 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597293#file1597293line1328>
> >
> >     Do we need this?
> 
> Peter Bacsko wrote:
>     We might not need -Dlog4j.debug (although I think it's useful when we have logging problems) but the rest was copy-pasted from MRApps.java or already existed (your early code). The hadoop.root.logger and hadoop.root.logfile props are definitely necessary, perhaps the root loggers could be configurable.

The other stuff sounds fine then, but the ````log4j.debug=true```` turns on debug logging for Log4j itself, right?  I don't think we need that.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 1115 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597293#file1597293line1330>
> >
> >     Does this mean the launcher logs can't be larger than 1GB?
> 
> Peter Bacsko wrote:
>     I don't know - this value was already here :)

It looks like this gets passed to the log4j config for the container.  It essentially sets the max filesize for when to rollover the log file.  Looking at how MRApps sets it (https://github.com/apache/hadoop/blob/f67237cbe7bc48a1b9088e990800b37529f1db2a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java#L710) to MRJobConfig.MR_AM_LOG_KB, which is 0 by default (indicating no rollover).  Given that, I'm thinking we should set this to 0 so we don't rollover - I'm not sure how well Yarn will handle a rolled over log file.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java
> > Lines 156-158 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597300#file1597300line160>
> >
> >     We shouldn't need the MR jars in the Spark Action :)
> 
> Peter Cseh wrote:
>     Oozie's SparkExample needs MR jars on the classpath.

That's surprising, but okay.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
> > Lines 1787-1821 (original), 1640-1674 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597330#file1597330line1790>
> >
> >     What's the plan here?
> 
> Peter Bacsko wrote:
>     Perhaps we don't need this test anymore.
>     
>     I'm not entirely familiar with MapReduce ACLs but I think it's related to the LauncherMapper. But we don't have a mapper anymore, but a standalone AM. Do we still care about MR ACLs?

I'm not either, but it sounds like this doesn't apply anymore so we should delete it.


- Robert


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On Jan. 23, 2017, 11:43 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2017, 11:43 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6e30d7eb667fe74440281fab3ca06c9f4df7bcf7 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/FSUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml ad103864d7631e9d3678a828bf297ca1c80f09ca 
>   core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 8ec38e498f31bb69923125e34cda5abfc8e72aa9 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 15f4195074c935bfaf941d2d0dc502105c4294b1 
>   release-log.txt ca9cd43260ffd3b5468631ff3c6edf2d90457e1c 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 7de2e453e150d0cede961f9be622c4b55396d5d5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/4/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, lines 1328-1332
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597293#file1597293line1328>
> >
> >     Do we need this?

We might not need -Dlog4j.debug (although I think it's useful when we have logging problems) but the rest was copy-pasted from MRApps.java or already existed (your early code). The hadoop.root.logger and hadoop.root.logfile props are definitely necessary, perhaps the root loggers could be configurable.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, line 1330
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597293#file1597293line1330>
> >
> >     Does this mean the launcher logs can't be larger than 1GB?

I don't know - this value was already here :)


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, line 1556
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597293#file1597293line1556>
> >
> >     Do we still need this method?  We have ````createYarnClient```` now.

It's not used directly by JavaActionExecutor, but MapReduceActionExecutor and SqoopActionExecutor need job clients in check() / end ().


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/main/resources/oozie-log4j.properties, lines 36-56
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597323#file1597323line36>
> >
> >     This is the issue we discussed in today's meeting about the logger breaking things, right?
> >     
> >     In any case, I don't think we want to get rid of this.

We will restore this to the original.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java, lines 279-280
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597330#file1597330line279>
> >
> >     ?

Ah, I forgot about this.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java, lines 1790-1824
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597330#file1597330line1790>
> >
> >     What's the plan here?

Perhaps we don't need this test anymore.

I'm not entirely familiar with MapReduce ACLs but I think it's related to the LauncherMapper. But we don't have a mapper anymore, but a standalone AM. Do we still care about MR ACLs?


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java, line 272
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597386#file1597386line272>
> >
> >     I'm not sure we need to print the security manager here

Right, I forgot to remove this.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > docs/src/site/twiki/DG_QuickStart.twiki, line 51
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597359#file1597359line51>
> >
> >     0.12.1 < 0.16.0, right?

This might be due to a merge problem, will check.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, lines 1195-1205
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597293#file1597293line1195>
> >
> >     Instead of doing this (which I know is equivalent to what we currently do), would it better to call ````check(...)```` instead?  I think it handles stuff better (e.g. fallback to data file, etc).  Or maybe we can extract out the common code into a new method and have them both call that, if we don't want all of ````check(...)```` here.

You mean to call check() when the Launcher is already running?


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/WorkflowJobBean.java, line 608
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597281#file1597281line608>
> >
> >     I was looking at http://stackoverflow.com/questions/933447/how-do-you-cast-a-list-of-supertypes-to-a-list-of-subtypes.
> >     
> >     This still generics an unchecked warning, so is there an advantage to this ugly double casting vs just doing ````(List)````?

I think I was just experimenting with something and ended up committing this. Agree, this has no advantage at all.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On jan. 6, 2017, 1:59 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated jan. 6, 2017, 1:59 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> Diff: https://reviews.apache.org/r/54705/diff/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Attila Sasvari <as...@cloudera.com>.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java, line 48
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597305#file1597305line48>
> >
> >     This isn't deprecated, is it?
> 
> Peter Bacsko wrote:
>     This is a result of Attila's commit: https://issues.apache.org/jira/browse/OOZIE-2316 although I'm not sure why this is in the map, because both XOozieClient.RM and "yarn.resourcemanager.address" are the same. I'll ask him.

It is mistake that somehow slipped through the previous review.


- Attila


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On Jan. 6, 2017, 1:59 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 6, 2017, 1:59 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> Diff: https://reviews.apache.org/r/54705/diff/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java
> > Lines 156-158 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597300#file1597300line160>
> >
> >     We shouldn't need the MR jars in the Spark Action :)
> 
> Peter Cseh wrote:
>     Oozie's SparkExample needs MR jars on the classpath.
> 
> Robert Kanter wrote:
>     That's surprising, but okay.

I'm not sure which Spark code I ran, but it missed org.apache.hadoop.mapred.TextInputFormat


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
> > Lines 1787-1821 (original), 1640-1674 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597330#file1597330line1790>
> >
> >     What's the plan here?
> 
> Peter Bacsko wrote:
>     Perhaps we don't need this test anymore.
>     
>     I'm not entirely familiar with MapReduce ACLs but I think it's related to the LauncherMapper. But we don't have a mapper anymore, but a standalone AM. Do we still care about MR ACLs?
> 
> Robert Kanter wrote:
>     I'm not either, but it sounds like this doesn't apply anymore so we should delete it.

Checked with Geza, will be removed


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On márc. 22, 2017, 8:34 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated márc. 22, 2017, 8:34 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1779
>     https://issues.apache.org/jira/browse/OOZIE-1779
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   bin/test-patch-20-tests 22f472c07e80fdb38c632329a5707998ab352172 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/cli/ValidationUtil.java b1da2af070970add01031d6208a395520cd7055c 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 73708080e624e775f7d058cd3377ad69d8399264 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ca168e084998b5dd564536078a7a69efd7291828 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 8ddb1f873f0c69aadf5f4a5161d8b21f7392fa1e 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 39daca7b3050c259a36d54710b4c1279f3acdeac 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 5fd30f7858c5c4c70dcbdb6df86904518d92f7f3 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 2ba1999ba7d5e892bd7e37d5fd8d1f926a84b48b 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java 52e03d9839f01e8fcbc9c0fdee179467b9ac5c98 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 4cc284ae52ac50cce36c251536f045804e351f7a 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/PurgeService.java c2c966b2cd056b77ae52164cb3f40955e98ae69a 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java d263567eab9fc9b8763296c35e0a904499eabc11 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/servlet/JVMInfo.java bf2b01db340a014d8831c5186188741f866e6559 
>   core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java d4a098949534f417938a65c27e395c9ac1f71ca7 
>   core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java fc3a6bc19d6a04e489c3f65c8248478a58540ede 
>   core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java d08521ca28d9ae4c8582c6ce3a8e6ca35a26f8dd 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 83b455ebc2629fddccf44b61b1051f932e1d40a3 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 03f428a47e34f2d07ea8fff755f29d222d6aeb9a 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherHCatURIHandler.java 4288974b036b68b42d260d674aaaa0d345484123 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestScriptLanguageActionExecutor.java 0adc190d56ec163d454d64ae341b6a658ac14781 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 564db2a5150d949e2e9274984c1d67acce6b5447 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java e41df8719fa361adca5406ddf1da919f9e78016d 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 6fd1a1aa5eab59c085551be8e52fa0d01c79c4da 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java 1278efa0775c32c16cb38309ab8e92e08d4eb2ec 
>   core/src/test/java/org/apache/oozie/servlet/TestAdminServlet.java 8e3502c343744b7e70d2033643c9596c0c029778 
>   core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java a3f3b9c28c0f34381930a7457b5b622695948742 
>   core/src/test/java/org/apache/oozie/test/MiniHCatServer.java b2f7282a30bc8ed269d9c4208036421047647ef0 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 58f4beab9e707a3d8542253df596a566e65eff3c 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/TestIOUtils.java 6fcdf5526927a3b8033c7a0c747b7480a5f199e7 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 6a9e4b4bfd50954c399dd094c3fee82c491b6800 
>   docs/src/site/twiki/DG_HCatalogIntegration.twiki 39df67c3f5c5df0bb0521f613a66862c1b0a0ed8 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   docs/src/site/twiki/WebServicesAPI.twiki a63a68cb27de05d03ad173caeb18f5b17e20867b 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 460f6e521cb8fe2e508ec7086081d49890efb037 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 98a2359a3ca476a0516a9746da2c1e91857143fd 
>   release-log.txt 4e7097b17a2ab1036135b27e9bdf3554fa2923ae 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java fcaadefd215368565fb126ca92515792e1f1a09b 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java 7edb40d9d7cba6ef0603de25c9cee55f39b75f3b 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java faeff2a7f5f24f1680df5e529bbb48f49c750e09 
>   sharelib/hcatalog/src/test/java/org/apache/oozie/util/TestHCatURI.java ce690da986f3bbbab7f74e771b1fe9e32f047b27 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bc8ab29c25405806f40232d32254ba2a7f5271e4 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 7ae48e168a310b4d55feda7cbce8b2d29c2ec505 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java ba393eee53e92d144159f83893d4baab00737a44 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 14ab65e7d00c426036ced3e80400c80deea0e282 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java c24d95c88be0dbf6fe80241d887a70100a5e1b4a 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb8f56e9b66de0eb2799e68bc56d0ee36d4ba317 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/5/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java, lines 160-162
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597300#file1597300line160>
> >
> >     We shouldn't need the MR jars in the Spark Action :)

Oozie's SparkExample needs MR jars on the classpath.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On Jan. 23, 2017, 11:43 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2017, 11:43 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6e30d7eb667fe74440281fab3ca06c9f4df7bcf7 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/FSUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml ad103864d7631e9d3678a828bf297ca1c80f09ca 
>   core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 8ec38e498f31bb69923125e34cda5abfc8e72aa9 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 15f4195074c935bfaf941d2d0dc502105c4294b1 
>   release-log.txt ca9cd43260ffd3b5468631ff3c6edf2d90457e1c 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 7de2e453e150d0cede961f9be622c4b55396d5d5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> Diff: https://reviews.apache.org/r/54705/diff/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java, line 30
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597368#file1597368line30>
> >
> >     Why do we need a factory for this?  Creating one seems pretty straightforward.

It helps unit testing. When you write unit test for LauncherAM, you want to avoid creating real LauncherCallbackNotifier instances because when you invoke notifyURL() it would connect to a real URL. If you mock the factory, it can return notifier mocks and you can do Mockito.verify() on them.

I know it seems like overkill but I think it's a good pattern to enhance testability.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > findbugs-filter.xml, line 18
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597361#file1597361line18>
> >
> >     I'm fine with ignoring some findbugs stuff (Hadoop has a ton of these), but I think we should add a comment for each exclusion explaining it.  As it stands, I only know the class and findbugs pattern name.  It would be nice to at least narrow it down to the method and a reason why we can exclude it.

OK, I'll add some comments.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On jan. 6, 2017, 1:59 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated jan. 6, 2017, 1:59 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> Diff: https://reviews.apache.org/r/54705/diff/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java
> > Line 148 (original), 148 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597294#file1597294line150>
> >
> >     I don't think we need this anymore?  Probably not for a long time.

I've tried it out and it's not working anyways. It could be removed as part of OOZIE-2601


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > docs/src/site/twiki/DG_QuickStart.twiki
> > Lines 49-50 (original), 45-46 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597359#file1597359line49>
> >
> >     Are these still separate?  If so, there's no reason to split these from ````hadoop.version```` anymore now that Hadoop 1 is gone.

https://issues.apache.org/jira/browse/OOZIE-2884


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > docs/src/site/twiki/ENG_Building.twiki
> > Lines 220-221 (original), 216-217 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597360#file1597360line220>
> >
> >     Same comment as in DG_QuickStart

https://issues.apache.org/jira/browse/OOZIE-2884


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On March 22, 2017, 8:34 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated March 22, 2017, 8:34 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1779
>     https://issues.apache.org/jira/browse/OOZIE-1779
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   bin/test-patch-20-tests 22f472c07e80fdb38c632329a5707998ab352172 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/cli/ValidationUtil.java b1da2af070970add01031d6208a395520cd7055c 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 73708080e624e775f7d058cd3377ad69d8399264 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ca168e084998b5dd564536078a7a69efd7291828 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 8ddb1f873f0c69aadf5f4a5161d8b21f7392fa1e 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 39daca7b3050c259a36d54710b4c1279f3acdeac 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 5fd30f7858c5c4c70dcbdb6df86904518d92f7f3 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 2ba1999ba7d5e892bd7e37d5fd8d1f926a84b48b 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java 52e03d9839f01e8fcbc9c0fdee179467b9ac5c98 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 4cc284ae52ac50cce36c251536f045804e351f7a 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/PurgeService.java c2c966b2cd056b77ae52164cb3f40955e98ae69a 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java d263567eab9fc9b8763296c35e0a904499eabc11 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/servlet/JVMInfo.java bf2b01db340a014d8831c5186188741f866e6559 
>   core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java d4a098949534f417938a65c27e395c9ac1f71ca7 
>   core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java fc3a6bc19d6a04e489c3f65c8248478a58540ede 
>   core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java d08521ca28d9ae4c8582c6ce3a8e6ca35a26f8dd 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 83b455ebc2629fddccf44b61b1051f932e1d40a3 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 03f428a47e34f2d07ea8fff755f29d222d6aeb9a 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherHCatURIHandler.java 4288974b036b68b42d260d674aaaa0d345484123 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestScriptLanguageActionExecutor.java 0adc190d56ec163d454d64ae341b6a658ac14781 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 564db2a5150d949e2e9274984c1d67acce6b5447 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java e41df8719fa361adca5406ddf1da919f9e78016d 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 6fd1a1aa5eab59c085551be8e52fa0d01c79c4da 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java 1278efa0775c32c16cb38309ab8e92e08d4eb2ec 
>   core/src/test/java/org/apache/oozie/servlet/TestAdminServlet.java 8e3502c343744b7e70d2033643c9596c0c029778 
>   core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java a3f3b9c28c0f34381930a7457b5b622695948742 
>   core/src/test/java/org/apache/oozie/test/MiniHCatServer.java b2f7282a30bc8ed269d9c4208036421047647ef0 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 58f4beab9e707a3d8542253df596a566e65eff3c 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/TestIOUtils.java 6fcdf5526927a3b8033c7a0c747b7480a5f199e7 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 6a9e4b4bfd50954c399dd094c3fee82c491b6800 
>   docs/src/site/twiki/DG_HCatalogIntegration.twiki 39df67c3f5c5df0bb0521f613a66862c1b0a0ed8 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   docs/src/site/twiki/WebServicesAPI.twiki a63a68cb27de05d03ad173caeb18f5b17e20867b 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 460f6e521cb8fe2e508ec7086081d49890efb037 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 98a2359a3ca476a0516a9746da2c1e91857143fd 
>   release-log.txt 4e7097b17a2ab1036135b27e9bdf3554fa2923ae 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java fcaadefd215368565fb126ca92515792e1f1a09b 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java 7edb40d9d7cba6ef0603de25c9cee55f39b75f3b 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java faeff2a7f5f24f1680df5e529bbb48f49c750e09 
>   sharelib/hcatalog/src/test/java/org/apache/oozie/util/TestHCatURI.java ce690da986f3bbbab7f74e771b1fe9e32f047b27 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bc8ab29c25405806f40232d32254ba2a7f5271e4 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 7ae48e168a310b4d55feda7cbce8b2d29c2ec505 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java ba393eee53e92d144159f83893d4baab00737a44 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 14ab65e7d00c426036ced3e80400c80deea0e282 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java c24d95c88be0dbf6fe80241d887a70100a5e1b4a 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb8f56e9b66de0eb2799e68bc56d0ee36d4ba317 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/5/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On jan. 6, 2017, 7:19 de, Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java, line 48
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597305#file1597305line48>
> >
> >     This isn't deprecated, is it?

This is a result of Attila's commit: https://issues.apache.org/jira/browse/OOZIE-2316 although I'm not sure why this is in the map, because both XOozieClient.RM and "yarn.resourcemanager.address" are the same. I'll ask him.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On jan. 6, 2017, 1:59 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated jan. 6, 2017, 1:59 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> Diff: https://reviews.apache.org/r/54705/diff/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On Jan. 6, 2017, 7:19 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 262-266 (patched)
> > <https://reviews.apache.org/r/54705/diff/2/?file=1597293#file1597293line281>
> >
> >     We'll probably need to change the ````OozieActionConfigurator```` interface to use a ````Configuration```` object instead of a ````JobConf```` object.

The OozieActionConfigurator is used only for MR actions based on it's apidoc. Is the launchermapper itself configurable via this? If so we might want to drop that option. If it's related only for MR actions, we can keep the JobConf here.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------


On Jan. 23, 2017, 11:43 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2017, 11:43 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6e30d7eb667fe74440281fab3ca06c9f4df7bcf7 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/FSUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml ad103864d7631e9d3678a828bf297ca1c80f09ca 
>   core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 8ec38e498f31bb69923125e34cda5abfc8e72aa9 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 15f4195074c935bfaf941d2d0dc502105c4294b1 
>   release-log.txt ca9cd43260ffd3b5468631ff3c6edf2d90457e1c 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 7de2e453e150d0cede961f9be622c4b55396d5d5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/4/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160656
-----------------------------------------------------------




client/src/main/java/org/apache/oozie/client/XOozieClient.java (line 119)
<https://reviews.apache.org/r/54705/#comment231818>

    "ResourceManager is not specified in conf"



core/src/main/java/org/apache/oozie/WorkflowJobBean.java (line 607)
<https://reviews.apache.org/r/54705/#comment231819>

    I was looking at http://stackoverflow.com/questions/933447/how-do-you-cast-a-list-of-supertypes-to-a-list-of-subtypes.
    
    This still generics an unchecked warning, so is there an advantage to this ugly double casting vs just doing ````(List)````?



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 250)
<https://reviews.apache.org/r/54705/#comment231821>

    This should be changed to use ````Configuration````.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 253)
<https://reviews.apache.org/r/54705/#comment231820>

    ````JobConf```` is an MR thing.  We should use a ````Configuration````



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 254)
<https://reviews.apache.org/r/54705/#comment231822>

    ?



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (lines 262 - 266)
<https://reviews.apache.org/r/54705/#comment231823>

    We'll probably need to change the ````OozieActionConfigurator```` interface to use a ````Configuration```` object instead of a ````JobConf```` object.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
<https://reviews.apache.org/r/54705/#comment231824>

    Yay!!!



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (lines 991 - 1001)
<https://reviews.apache.org/r/54705/#comment231825>

    Instead of doing this (which I know is equivalent to what we currently do), would it better to call ````check(...)```` instead?  I think it handles stuff better (e.g. fallback to data file, etc).  Or maybe we can extract out the common code into a new method and have them both call that, if we don't want all of ````check(...)```` here.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (lines 1113 - 1117)
<https://reviews.apache.org/r/54705/#comment231827>

    Do we need this?



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 1115)
<https://reviews.apache.org/r/54705/#comment231828>

    Does this mean the launcher logs can't be larger than 1GB?



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 1341)
<https://reviews.apache.org/r/54705/#comment231830>

    Do we still need this method?  We have ````createYarnClient```` now.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 1396)
<https://reviews.apache.org/r/54705/#comment231833>

    We should try to replace this with ````Configuration````



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 1538)
<https://reviews.apache.org/r/54705/#comment231836>

    Let's give a more helpful message here.



core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java (line 148)
<https://reviews.apache.org/r/54705/#comment231837>

    I don't think we need this anymore?  Probably not for a long time.



core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java (lines 156 - 158)
<https://reviews.apache.org/r/54705/#comment231843>

    We shouldn't need the MR jars in the Spark Action :)



core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java (line 48)
<https://reviews.apache.org/r/54705/#comment231846>

    This isn't deprecated, is it?



core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java (line 538)
<https://reviews.apache.org/r/54705/#comment231847>

    (I think I originally wrote this Javadoc but,) I think we should add an additional sentence:
    
    Return a YarnClient created with the provided user and configuration.  The caller is responsible for closing it when done.



core/src/main/resources/oozie-log4j.properties (line 35)
<https://reviews.apache.org/r/54705/#comment231855>

    This is the issue we discussed in today's meeting about the logger breaking things, right?
    
    In any case, I don't think we want to get rid of this.



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java (lines 268 - 269)
<https://reviews.apache.org/r/54705/#comment231857>

    ?



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java (lines 1640 - 1674)
<https://reviews.apache.org/r/54705/#comment231858>

    What's the plan here?



core/src/test/java/org/apache/oozie/client/TestOozieCLI.java (lines 137 - 138)
<https://reviews.apache.org/r/54705/#comment231859>

    9000 and 9001 are super old default ports.  Let's take this opportunity to update them to 8020 and 8032 here and elsewhere.



core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java (line 23)
<https://reviews.apache.org/r/54705/#comment231860>

    No * imports please



distro/src/main/bin/addtowar.sh (line 98)
<https://reviews.apache.org/r/54705/#comment231862>

    This is probably the same for Hadoop 3 as well.



docs/src/site/twiki/DG_QuickStart.twiki (line 44)
<https://reviews.apache.org/r/54705/#comment231863>

    This was changed to 2.6.0 in the pom



docs/src/site/twiki/DG_QuickStart.twiki (lines 45 - 46)
<https://reviews.apache.org/r/54705/#comment231864>

    Are these still separate?  If so, there's no reason to split these from ````hadoop.version```` anymore now that Hadoop 1 is gone.



docs/src/site/twiki/DG_QuickStart.twiki (line 47)
<https://reviews.apache.org/r/54705/#comment231865>

    0.12.1 < 0.16.0, right?



docs/src/site/twiki/ENG_Building.twiki (line 117)
<https://reviews.apache.org/r/54705/#comment231866>

    2.6.0



docs/src/site/twiki/ENG_Building.twiki (line 215)
<https://reviews.apache.org/r/54705/#comment231867>

    2.6.0



docs/src/site/twiki/ENG_Building.twiki (lines 216 - 217)
<https://reviews.apache.org/r/54705/#comment231868>

    Same comment as in DG_QuickStart



docs/src/site/twiki/ENG_Building.twiki (line 219)
<https://reviews.apache.org/r/54705/#comment231869>

    h2



findbugs-filter.xml (line 18)
<https://reviews.apache.org/r/54705/#comment231810>

    I'm fine with ignoring some findbugs stuff (Hadoop has a ton of these), but I think we should add a comment for each exclusion explaining it.  As it stands, I only know the class and findbugs pattern name.  It would be nice to at least narrow it down to the method and a reason why we can exclude it.



hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java (line 22)
<https://reviews.apache.org/r/54705/#comment231872>

    Why do we need a factory for this?  Creating one seems pretty straightforward.



pom.xml (line 1949)
<https://reviews.apache.org/r/54705/#comment231817>

    The wfgen profile was removed; we shouldn't add it back.



pom.xml (line 1950)
<https://reviews.apache.org/r/54705/#comment231816>

    I'm not sure if it's easier to do it separately or not, but I had created OOZIE-2761 to drop Hadoop 1 from trunk.  It is also supposed to change the project version from 4.4.0-SNAPSHOT to 5.0.0-SNAPSHOT.
    
    I'm fine with doing the changes here, but if we do them in OOZIE-2761, it would remove some of the changes here, making this patch smaller (i.e. hadooplibs, etc)



sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java (line 265)
<https://reviews.apache.org/r/54705/#comment231873>

    I'm not sure we need to print the security manager here



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java (lines 63 - 64)
<https://reviews.apache.org/r/54705/#comment231874>

    We're on Java 7 now so you can use try-with-resources



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java (lines 105 - 107)
<https://reviews.apache.org/r/54705/#comment231875>

    try-with-resources



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java (lines 118 - 127)
<https://reviews.apache.org/r/54705/#comment231876>

    try-with-resources



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java (lines 51 - 54)
<https://reviews.apache.org/r/54705/#comment231892>

    From what I can tell, they're just for a message when LocalFsOperations#getLocalFileContentAsString throws an Exception.  
    
    I think we can just use their counterparts that are used for the key in the map.  i.e. ACTION_DATA_NEW_ID instead of ACTIONOUTPUTTYPE_ID_SWAP, etc
    
    No reason to essentially duplicate all of these.
    
    Also, there's only two files we pass to getLocalFileContentAsString: action-data.seq, which has most things, and externalChildIDs, which has the external child IDs.  The Exception is thrown if we have too much data, but the message says it's for each of these file types, which is not true because there's only 2 files and 4 of these types.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java (line 157)
<https://reviews.apache.org/r/54705/#comment231891>

    launcherExecutedProperly



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java (lines 154 - 159)
<https://reviews.apache.org/r/54705/#comment231894>

    Can be simplified with a multi-catch (or whatever it's called).
    
    http://docs.oracle.com/javase/7/docs/technotes/guides/language/catch-multiple.html



sharelib/spark/pom.xml (line 254)
<https://reviews.apache.org/r/54705/#comment231895>

    ${spark.scala.binary.version}



sharelib/spark/pom.xml (lines 375 - 393)
<https://reviews.apache.org/r/54705/#comment231896>

    I don't think we need this anymore (the other actions certainly don't need it)


- Robert Kanter


On Jan. 5, 2017, 1:16 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 5, 2017, 1:16 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
>   core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> Diff: https://reviews.apache.org/r/54705/diff/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On Jan. 6, 2017, 4:37 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 104-110 (original), 129-135 (patched)
> > <https://reviews.apache.org/r/54705/diff/3/?file=1598594#file1598594line134>
> >
> >     Since we use our own App master, do we care about all these properties?
> >     
> >     I checked MapReduceActionExecutor which calls super.setupActionConf() and super.setupLauncherConf(). But even if we use these props, it should be relevant only for the MR action.
> >     
> >     Thoughts?
> 
> Robert Kanter wrote:
>     We had discussed doing some backwards compatibility for some of these things for the launcher.  It might make sense to delete these for now (or leave them?), and OOZIE-2687 can deal with them later.

Will be removed after https://issues.apache.org/jira/browse/OOZIE-2687


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160725
-----------------------------------------------------------


On March 22, 2017, 8:34 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated March 22, 2017, 8:34 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1779
>     https://issues.apache.org/jira/browse/OOZIE-1779
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   bin/test-patch-20-tests 22f472c07e80fdb38c632329a5707998ab352172 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/cli/ValidationUtil.java b1da2af070970add01031d6208a395520cd7055c 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 73708080e624e775f7d058cd3377ad69d8399264 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ca168e084998b5dd564536078a7a69efd7291828 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 8ddb1f873f0c69aadf5f4a5161d8b21f7392fa1e 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 39daca7b3050c259a36d54710b4c1279f3acdeac 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 5fd30f7858c5c4c70dcbdb6df86904518d92f7f3 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 2ba1999ba7d5e892bd7e37d5fd8d1f926a84b48b 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java 52e03d9839f01e8fcbc9c0fdee179467b9ac5c98 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 4cc284ae52ac50cce36c251536f045804e351f7a 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/PurgeService.java c2c966b2cd056b77ae52164cb3f40955e98ae69a 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java d263567eab9fc9b8763296c35e0a904499eabc11 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/servlet/JVMInfo.java bf2b01db340a014d8831c5186188741f866e6559 
>   core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java d4a098949534f417938a65c27e395c9ac1f71ca7 
>   core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java fc3a6bc19d6a04e489c3f65c8248478a58540ede 
>   core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java d08521ca28d9ae4c8582c6ce3a8e6ca35a26f8dd 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 83b455ebc2629fddccf44b61b1051f932e1d40a3 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 03f428a47e34f2d07ea8fff755f29d222d6aeb9a 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherHCatURIHandler.java 4288974b036b68b42d260d674aaaa0d345484123 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestScriptLanguageActionExecutor.java 0adc190d56ec163d454d64ae341b6a658ac14781 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 564db2a5150d949e2e9274984c1d67acce6b5447 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java e41df8719fa361adca5406ddf1da919f9e78016d 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 6fd1a1aa5eab59c085551be8e52fa0d01c79c4da 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java 1278efa0775c32c16cb38309ab8e92e08d4eb2ec 
>   core/src/test/java/org/apache/oozie/servlet/TestAdminServlet.java 8e3502c343744b7e70d2033643c9596c0c029778 
>   core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java a3f3b9c28c0f34381930a7457b5b622695948742 
>   core/src/test/java/org/apache/oozie/test/MiniHCatServer.java b2f7282a30bc8ed269d9c4208036421047647ef0 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 58f4beab9e707a3d8542253df596a566e65eff3c 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/TestIOUtils.java 6fcdf5526927a3b8033c7a0c747b7480a5f199e7 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 6a9e4b4bfd50954c399dd094c3fee82c491b6800 
>   docs/src/site/twiki/DG_HCatalogIntegration.twiki 39df67c3f5c5df0bb0521f613a66862c1b0a0ed8 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   docs/src/site/twiki/WebServicesAPI.twiki a63a68cb27de05d03ad173caeb18f5b17e20867b 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 460f6e521cb8fe2e508ec7086081d49890efb037 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 98a2359a3ca476a0516a9746da2c1e91857143fd 
>   release-log.txt 4e7097b17a2ab1036135b27e9bdf3554fa2923ae 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java fcaadefd215368565fb126ca92515792e1f1a09b 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java 7edb40d9d7cba6ef0603de25c9cee55f39b75f3b 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java faeff2a7f5f24f1680df5e529bbb48f49c750e09 
>   sharelib/hcatalog/src/test/java/org/apache/oozie/util/TestHCatURI.java ce690da986f3bbbab7f74e771b1fe9e32f047b27 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bc8ab29c25405806f40232d32254ba2a7f5271e4 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 7ae48e168a310b4d55feda7cbce8b2d29c2ec505 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java ba393eee53e92d144159f83893d4baab00737a44 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 14ab65e7d00c426036ced3e80400c80deea0e282 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java c24d95c88be0dbf6fe80241d887a70100a5e1b4a 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb8f56e9b66de0eb2799e68bc56d0ee36d4ba317 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/5/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.


> On Jan. 6, 2017, 4:37 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, lines 134-140
> > <https://reviews.apache.org/r/54705/diff/3/?file=1598594#file1598594line134>
> >
> >     Since we use our own App master, do we care about all these properties?
> >     
> >     I checked MapReduceActionExecutor which calls super.setupActionConf() and super.setupLauncherConf(). But even if we use these props, it should be relevant only for the MR action.
> >     
> >     Thoughts?

We had discussed doing some backwards compatibility for some of these things for the launcher.  It might make sense to delete these for now (or leave them?), and OOZIE-2687 can deal with them later.


> On Jan. 6, 2017, 4:37 p.m., Peter Bacsko wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, line 1289
> > <https://reviews.apache.org/r/54705/diff/3/?file=1598594#file1598594line1289>
> >
> >     This will be addressed in OOZIE-2687. For the time being IMO it's ok to have 0 here.

By the way, Yarn currently ignores the priority.  But I think we should still add it (via OOZIE-2687) in case Yarn starts honoring it in the future.


- Robert


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160725
-----------------------------------------------------------


On Jan. 6, 2017, 1:59 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 6, 2017, 1:59 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> Diff: https://reviews.apache.org/r/54705/diff/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review160725
-----------------------------------------------------------



Random question - Since we're abandoning LauncherMapper, I think we should delete it. But then how we'll provide fixes for this class in the future?


core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (lines 129 - 135)
<https://reviews.apache.org/r/54705/#comment231921>

    Since we use our own App master, do we care about all these properties?
    
    I checked MapReduceActionExecutor which calls super.setupActionConf() and super.setupLauncherConf(). But even if we use these props, it should be relevant only for the MR action.
    
    Thoughts?



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 1074)
<https://reviews.apache.org/r/54705/#comment231922>

    This will be addressed in OOZIE-2687. For the time being IMO it's ok to have 0 here.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java (line 1141)
<https://reviews.apache.org/r/54705/#comment231923>

    Same here -- OOZIE-2687



core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java (line 56)
<https://reviews.apache.org/r/54705/#comment231924>

    I'm going to rename this, it's still used but we don't have a LauncherMapper anymore.


estion

- Peter Bacsko


On jan. 6, 2017, 1:59 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated jan. 6, 2017, 1:59 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> Diff: https://reviews.apache.org/r/54705/diff/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.


> On March 20, 2017, 5:34 p.m., Abhishek Bafna wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
> > Line 122 (original), 113 (patched)
> > <https://reviews.apache.org/r/54705/diff/4/?file=1612706#file1612706line123>
> >
> >     I believe "yarn.resourcemanager.address" is defined as constant some where, can we use that. If not, please define as constant. Thanks.

There's sort of two camps on this, but for tests, I like to hardcode these sorts of things.  That way, if we (accidently) change the constant, it breaks the test so we notice.


> On March 20, 2017, 5:34 p.m., Abhishek Bafna wrote:
> > core/src/test/java/org/apache/oozie/client/TestOozieCLI.java
> > Line 137 (original), 137 (patched)
> > <https://reviews.apache.org/r/54705/diff/4/?file=1612714#file1612714line137>
> >
> >     Can you keep the port number same as previous, like it is kept below?

9000 is a really really old NN port.  I think we should use the normal ports.  This is also a problem with some of the examples in our docs.


- Robert


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review169460
-----------------------------------------------------------


On Jan. 23, 2017, 11:43 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2017, 11:43 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6e30d7eb667fe74440281fab3ca06c9f4df7bcf7 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/FSUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml ad103864d7631e9d3678a828bf297ca1c80f09ca 
>   core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 8ec38e498f31bb69923125e34cda5abfc8e72aa9 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 15f4195074c935bfaf941d2d0dc502105c4294b1 
>   release-log.txt ca9cd43260ffd3b5468631ff3c6edf2d90457e1c 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 7de2e453e150d0cede961f9be622c4b55396d5d5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/4/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On March 20, 2017, 5:34 p.m., Abhishek Bafna wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 1376 (patched)
> > <https://reviews.apache.org/r/54705/diff/4/?file=1612665#file1612665line1614>
> >
> >     should not this be a error level message instead of debug?

We have a secondary mechanism to check the action status based on a file on HDFS. We'll throw an exception in line 1391 if the fallback mechanism fails. I've increased the log level to info to make it easier to find out why the initial check failed.


> On March 20, 2017, 5:34 p.m., Abhishek Bafna wrote:
> > core/src/main/java/org/apache/oozie/service/Services.java
> > Lines 307 (patched)
> > <https://reviews.apache.org/r/54705/diff/4/?file=1612685#file1612685line308>
> >
> >     Can we use a log instance to capture the stack trace?

I think it gets logged because of the exeption later. I've added a log entry to make it sure.


> On March 20, 2017, 5:34 p.m., Abhishek Bafna wrote:
> > core/src/main/resources/META-INF/persistence.xml
> > Lines 79 (patched)
> > <https://reviews.apache.org/r/54705/diff/4/?file=1612694#file1612694line79>
> >
> >     Will this print user credential information as well?
> >     
> >     From here:https://openjpa.apache.org/builds/1.2.3/apache-openjpa/docs/ref_guide_dbsetup.html
> >     
> >     PrintParameters: A boolean indicating whether SQL parameter values will be included in exception text and when logging is enabled. Since the parameter values may contain sensitive information the default value is false.

Good catch! This was added for development reasons.


> On March 20, 2017, 5:34 p.m., Abhishek Bafna wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
> > Line 122 (original), 113 (patched)
> > <https://reviews.apache.org/r/54705/diff/4/?file=1612706#file1612706line123>
> >
> >     I believe "yarn.resourcemanager.address" is defined as constant some where, can we use that. If not, please define as constant. Thanks.
> 
> Robert Kanter wrote:
>     There's sort of two camps on this, but for tests, I like to hardcode these sorts of things.  That way, if we (accidently) change the constant, it breaks the test so we notice.

I think I'll create a constant for it in the test though.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review169460
-----------------------------------------------------------


On Jan. 23, 2017, 11:43 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2017, 11:43 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6e30d7eb667fe74440281fab3ca06c9f4df7bcf7 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/FSUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml ad103864d7631e9d3678a828bf297ca1c80f09ca 
>   core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 8ec38e498f31bb69923125e34cda5abfc8e72aa9 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 15f4195074c935bfaf941d2d0dc502105c4294b1 
>   release-log.txt ca9cd43260ffd3b5468631ff3c6edf2d90457e1c 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 7de2e453e150d0cede961f9be622c4b55396d5d5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/4/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Abhishek Bafna <ba...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review169460
-----------------------------------------------------------




core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java
Lines 37 (patched)
<https://reviews.apache.org/r/54705/#comment241803>

    Instead of '+' can we use '{0}' index based parameters.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 1376 (patched)
<https://reviews.apache.org/r/54705/#comment241804>

    should not this be a error level message instead of debug?



core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
Lines 319 (patched)
<https://reviews.apache.org/r/54705/#comment241805>

    Should we log previous value as well here before updating it?
    
    Can this be defined as constant "job.end.notification.url" and then used?



core/src/main/java/org/apache/oozie/service/Services.java
Lines 307 (patched)
<https://reviews.apache.org/r/54705/#comment241808>

    Can we use a log instance to capture the stack trace?



core/src/main/java/org/apache/oozie/service/Services.java
Line 308 (original), 308 (patched)
<https://reviews.apache.org/r/54705/#comment241810>

    Can this be converted into index and parameter based log statement?



core/src/main/resources/META-INF/persistence.xml
Lines 79 (patched)
<https://reviews.apache.org/r/54705/#comment241811>

    Will this print user credential information as well?
    
    From here:https://openjpa.apache.org/builds/1.2.3/apache-openjpa/docs/ref_guide_dbsetup.html
    
    PrintParameters: A boolean indicating whether SQL parameter values will be included in exception text and when logging is enabled. Since the parameter values may contain sensitive information the default value is false.



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
Line 122 (original), 113 (patched)
<https://reviews.apache.org/r/54705/#comment241812>

    I believe "yarn.resourcemanager.address" is defined as constant some where, can we use that. If not, please define as constant. Thanks.



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
Line 2890 (original), 2256 (patched)
<https://reviews.apache.org/r/54705/#comment241813>

    "yarn.resourcemanager.address" this is hardcoded many places, please replace it by constant.



core/src/test/java/org/apache/oozie/client/TestOozieCLI.java
Line 137 (original), 137 (patched)
<https://reviews.apache.org/r/54705/#comment241814>

    Can you keep the port number same as previous, like it is kept below?



core/src/test/java/org/apache/oozie/command/wf/HangServlet.java
Lines 49 (patched)
<https://reviews.apache.org/r/54705/#comment241815>

    Please use index and parameter based logging.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java
Line 51 (original), 51 (patched)
<https://reviews.apache.org/r/54705/#comment241820>

    Why have we moved back to old way instead os using the utility method?



sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java
Lines 35 (patched)
<https://reviews.apache.org/r/54705/#comment241821>

    Can this be converted into a switch case?
    Why not check lenght of the array equal to 1 instead of > 0.



sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java
Line 169 (original), 169 (patched)
<https://reviews.apache.org/r/54705/#comment241823>

    With this, we will loose the masking of the sensitive information like password etc?
    
    Any specific reason for using doing it this way and not using the utility method?


- Abhishek Bafna


On Jan. 23, 2017, 11:43 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2017, 11:43 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6e30d7eb667fe74440281fab3ca06c9f4df7bcf7 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/FSUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml ad103864d7631e9d3678a828bf297ca1c80f09ca 
>   core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 8ec38e498f31bb69923125e34cda5abfc8e72aa9 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 15f4195074c935bfaf941d2d0dc502105c4294b1 
>   release-log.txt ca9cd43260ffd3b5468631ff3c6edf2d90457e1c 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 7de2e453e150d0cede961f9be622c4b55396d5d5 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/4/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 11, 2017, 6:14 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java
> > Lines 528-532 (original), 535-540 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714662#file1714662line542>
> >
> >     There's no group

It mentioned user/group originally. UserGroupInformatio can have an implicit support for groups also. https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java#L495


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174591
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174591
-----------------------------------------------------------




core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java
Lines 528-532 (original), 535-540 (patched)
<https://reviews.apache.org/r/54705/#comment247785>

    There's no group



core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java
Lines 574-577 (patched)
<https://reviews.apache.org/r/54705/#comment247786>

    This can use the | syntax for multiple Exception types in a catch.



core/src/main/java/org/apache/oozie/service/Services.java
Line 308 (original), 307 (patched)
<https://reviews.apache.org/r/54705/#comment247812>

    Why printStackTrace?



core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java
Lines 43 (patched)
<https://reviews.apache.org/r/54705/#comment247814>

    Do we need this?



core/src/main/java/org/apache/oozie/util/ClasspathUtils.java
Lines 44-46 (patched)
<https://reviews.apache.org/r/54705/#comment247815>

    I'm not sure we need these 3.  We only localize files into the PWD in the Oozie AM.



core/src/main/java/org/apache/oozie/util/Instrumentation.java
Lines 35 (patched)
<https://reviews.apache.org/r/54705/#comment247816>

    Unnecessary change



core/src/main/java/org/apache/oozie/util/Instrumentation.java
Lines 699 (patched)
<https://reviews.apache.org/r/54705/#comment247817>

    Unnecessary change



core/src/main/resources/oozie-default.xml
Line 1863 (original), 1838 (patched)
<https://reviews.apache.org/r/54705/#comment247818>

    This can also be removed - it has no bearing now.


- Robert Kanter


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174829
-----------------------------------------------------------




core/src/test/java/org/apache/oozie/test/XTestCase.java
Lines 1238 (patched)
<https://reviews.apache.org/r/54705/#comment248050>

    We should use Configuration



docs/src/site/twiki/ENG_Building.twiki
Line 115 (original), 115 (patched)
<https://reviews.apache.org/r/54705/#comment248051>

    missing space in "version(Hadoop-2)"


- Robert Kanter


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 11, 2017, 12:41 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
> > Line 102 (original), 119 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714645#file1714645line121>
> >
> >     I think we still need this here, but it got me thinking - do we need to Oozie AM to cancel the tokens before it exits?
> 
> Rohini Palaniswamy wrote:
>     Yes. We do need to have the delegation tokens cancelled. AM itself should not be cancelling the tokens. It has tell the RM via ApplicationSubmissionContext.setCancelTokensWhenComplete and RM has to do it.

If we set appContext.setCancelTokensWhenComplete(true) in createAppSubmissionContext, it will cancel the tokens for MR action because the LauncherAM exits early in that case. I think it will cause issues there.
Created  OOZIE-2904 to address this.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174589
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 11, 2017, 12:41 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 1066 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line1307>
> >
> >     It's my understanding that the priority doesn't do anything in Yarn today.  YARNRunner currently uses 0 for the MR AM, so let's also use 0.
> >     https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java#L130
> >     
> >     I think we can get rid of the TODO, and replace it with a comment like "MR AM uses 0" or something like that so we remember why we set 0.  Should probably make it a constant too.
> 
> Rohini Palaniswamy wrote:
>     mapred.job.priority is used by some users. Oozie AM needs to support that. Please file a new jira for it

OOZIE-2892


> On May 11, 2017, 12:41 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 1069 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line1310>
> >
> >     Please file a followup JIRA for this.
> 
> Rohini Palaniswamy wrote:
>     This is important and needs to be fixed. 95% of our users configure a different queue using oozie.launcher settings.

OOZIE-2896


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174589
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174589
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.


> On May 11, 2017, 12:41 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Line 1166 (original), 989 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line1216>
> >
> >     Have we figured this out?

Yes it is needed. Many people launch MR jobs from java actions today.


> On May 11, 2017, 12:41 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 1066 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line1307>
> >
> >     It's my understanding that the priority doesn't do anything in Yarn today.  YARNRunner currently uses 0 for the MR AM, so let's also use 0.
> >     https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java#L130
> >     
> >     I think we can get rid of the TODO, and replace it with a comment like "MR AM uses 0" or something like that so we remember why we set 0.  Should probably make it a constant too.

mapred.job.priority is used by some users. Oozie AM needs to support that. Please file a new jira for it


> On May 11, 2017, 12:41 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 1069 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line1310>
> >
> >     Please file a followup JIRA for this.

This is important and needs to be fixed. 95% of our users configure a different queue using oozie.launcher settings.


> On May 11, 2017, 12:41 a.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
> > Line 102 (original), 119 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714645#file1714645line121>
> >
> >     I think we still need this here, but it got me thinking - do we need to Oozie AM to cancel the tokens before it exits?

Yes. We do need to have the delegation tokens cancelled. AM itself should not be cancelling the tokens. It has tell the RM via ApplicationSubmissionContext.setCancelTokensWhenComplete and RM has to do it.


- Rohini


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174589
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174589
-----------------------------------------------------------




core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Line 1166 (original), 989 (patched)
<https://reviews.apache.org/r/54705/#comment247769>

    Have we figured this out?



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Line 1239 (original), 1058 (patched)
<https://reviews.apache.org/r/54705/#comment247771>

    Can you create a followup JIRA to decide the name for the jobs?  We might want to have some discussion around this, so it would be easier to spin off into a new JIRA.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 1066 (patched)
<https://reviews.apache.org/r/54705/#comment247772>

    It's my understanding that the priority doesn't do anything in Yarn today.  YARNRunner currently uses 0 for the MR AM, so let's also use 0.
    https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java#L130
    
    I think we can get rid of the TODO, and replace it with a comment like "MR AM uses 0" or something like that so we remember why we set 0.  Should probably make it a constant too.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 1069 (patched)
<https://reviews.apache.org/r/54705/#comment247773>

    Please file a followup JIRA for this.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 1344 (patched)
<https://reviews.apache.org/r/54705/#comment247775>

    Also say what the default behavior is (i.e. returns false).



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 1382 (patched)
<https://reviews.apache.org/r/54705/#comment247776>

    Should we make this a warning?



core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java
Line 56 (original), 54 (patched)
<https://reviews.apache.org/r/54705/#comment247777>

    Remove TODO?



core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
Line 102 (original), 119 (patched)
<https://reviews.apache.org/r/54705/#comment247779>

    I think we still need this here, but it got me thinking - do we need to Oozie AM to cancel the tokens before it exits?



core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
Lines 136 (patched)
<https://reviews.apache.org/r/54705/#comment247780>

    Is this needed here?  We already inject it on line 120.



core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java
Line 39 (original), 39 (patched)
<https://reviews.apache.org/r/54705/#comment247783>

    I think we can remove this.  It's set on the Oozie AM's config, which doesn't do anything with this.



core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java
Lines 47 (patched)
<https://reviews.apache.org/r/54705/#comment247784>

    Delete


- Robert Kanter


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review176104
-----------------------------------------------------------


Ship it!




Ship It!

- Robert Kanter


On May 24, 2017, 12:05 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 24, 2017, 12:05 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/11/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review175946
-----------------------------------------------------------


Ship it!




Ship It!

- Rohini Palaniswamy


On May 24, 2017, 12:05 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 24, 2017, 12:05 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/11/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated May 24, 2017, 12:05 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Bugs: OOZIE-1770
    https://issues.apache.org/jira/browse/OOZIE-1770


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
  core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
  docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
  release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 


Diff: https://reviews.apache.org/r/54705/diff/11/

Changes: https://reviews.apache.org/r/54705/diff/10-11/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated May 23, 2017, 4:43 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Bugs: OOZIE-1770
    https://issues.apache.org/jira/browse/OOZIE-1770


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
  core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
  docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
  release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 


Diff: https://reviews.apache.org/r/54705/diff/10/

Changes: https://reviews.apache.org/r/54705/diff/9-10/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 22, 2017, 11:21 p.m., Rohini Palaniswamy wrote:
> > sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherMapper.java
> > Line 45 (original), 44 (patched)
> > <https://reviews.apache.org/r/54705/diff/9/?file=1726015#file1726015line45>
> >
> >     Why are they methods in this class renamed to start with testLauncherMapper when the test class itself is TestLauncherMapper?  
> >     
> >     Are these tests still relevant as LauncherMapper is not used anymore? Shouldn't they be changed to adapt to Oozie AM instead? Please revert current changes and create a jira for it if that is the case.

Opened up https://issues.apache.org/jira/browse/OOZIE-2918 .


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review175722
-----------------------------------------------------------


On May 20, 2017, 11:55 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 20, 2017, 11:55 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherMapper.java 51b1d6f8ccf1b64d2cb4b6a4da36b55c0888c259 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/9/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.


> On May 22, 2017, 11:21 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java
> > Line 49 (original), 50 (patched)
> > <https://reviews.apache.org/r/54705/diff/8-9/?file=1714632#file1714632line50>
> >
> >     Why would this require a ConcurrentHashMap?
> 
> Peter Cseh wrote:
>     It a precaution to avoid issues when getInstance() is called from multiple threads and the map is filled up from two threads. It shouldn't cause any issues with performance or threads this way.

Using a ConcurrentHashMap does not help in anyway. That variable is local to the new CredentialsProviderFactory being created. Two threads cannot fill the same map as the constructors will create two different instances. If two threads call getInstance at same time and if instance is null, both will create a instance and the last one will win. Ideally we should avoid that by synchronizing, but in this case based on the nature of data it is not a problem. Please switch it back to HashMap.


> On May 22, 2017, 11:21 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
> > Line 145 (original), 148 (patched)
> > <https://reviews.apache.org/r/54705/diff/8-9/?file=1714649#file1714649line148>
> >
> >     This method can be removed now.

It is still not removed.


- Rohini


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review175722
-----------------------------------------------------------


On May 23, 2017, 4:43 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 23, 2017, 4:43 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/10/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 22, 2017, 11:21 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java
> > Line 49 (original), 50 (patched)
> > <https://reviews.apache.org/r/54705/diff/8-9/?file=1714632#file1714632line50>
> >
> >     Why would this require a ConcurrentHashMap?

It a precaution to avoid issues when getInstance() is called from multiple threads and the map is filled up from two threads. It shouldn't cause any issues with performance or threads this way.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review175722
-----------------------------------------------------------


On May 20, 2017, 11:55 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 20, 2017, 11:55 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherMapper.java 51b1d6f8ccf1b64d2cb4b6a4da36b55c0888c259 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/9/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 22, 2017, 11:21 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
> > Line 145 (original), 148 (patched)
> > <https://reviews.apache.org/r/54705/diff/8-9/?file=1714649#file1714649line148>
> >
> >     This method can be removed now.
> 
> Rohini Palaniswamy wrote:
>     It is still not removed.

Hmm It's not there on the branch: https://github.com/apache/oozie/blob/oya/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
Let me double-check what's going on in the diff.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review175722
-----------------------------------------------------------


On May 23, 2017, 4:43 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 23, 2017, 4:43 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/10/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 22, 2017, 11:21 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
> > Line 145 (original), 148 (patched)
> > <https://reviews.apache.org/r/54705/diff/8-9/?file=1714649#file1714649line148>
> >
> >     This method can be removed now.
> 
> Rohini Palaniswamy wrote:
>     It is still not removed.
> 
> Peter Cseh wrote:
>     Hmm It's not there on the branch: https://github.com/apache/oozie/blob/oya/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
>     Let me double-check what's going on in the diff.

I looks like it's removed now
https://reviews.apache.org/r/54705/diff/11?page=2#25


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review175722
-----------------------------------------------------------


On May 24, 2017, 12:05 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 24, 2017, 12:05 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/11/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review175722
-----------------------------------------------------------




core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java
Line 49 (original), 50 (patched)
<https://reviews.apache.org/r/54705/#comment249074>

    Why would this require a ConcurrentHashMap?



core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java
Line 48 (original), 48 (patched)
<https://reviews.apache.org/r/54705/#comment249075>

    to credentials



core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java
Line 66 (original), 67 (patched)
<https://reviews.apache.org/r/54705/#comment249076>

    to credentials



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 959 (patched)
<https://reviews.apache.org/r/54705/#comment249077>

    Fix typo. Creating YarnClient



core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java
Line 145 (original), 148 (patched)
<https://reviews.apache.org/r/54705/#comment249079>

    This method can be removed now.



sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherMapper.java
Line 45 (original), 44 (patched)
<https://reviews.apache.org/r/54705/#comment249080>

    Why are they methods in this class renamed to start with testLauncherMapper when the test class itself is TestLauncherMapper?  
    
    Are these tests still relevant as LauncherMapper is not used anymore? Shouldn't they be changed to adapt to Oozie AM instead? Please revert current changes and create a jira for it if that is the case.


- Rohini Palaniswamy


On May 20, 2017, 11:55 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 20, 2017, 11:55 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherMapper.java 51b1d6f8ccf1b64d2cb4b6a4da36b55c0888c259 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/9/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated May 20, 2017, 11:55 a.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Bugs: OOZIE-1770
    https://issues.apache.org/jira/browse/OOZIE-1770


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
  core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
  docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
  release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherMapper.java 51b1d6f8ccf1b64d2cb4b6a4da36b55c0888c259 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 


Diff: https://reviews.apache.org/r/54705/diff/9/

Changes: https://reviews.apache.org/r/54705/diff/8-9/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 12, 2017, 6:35 p.m., Robert Kanter wrote:
> > sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java
> > Lines 312-318 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714737#file1714737line315>
> >
> >     Don't we need this to capture System.exit calls?

It's handled in LauncherAM https://github.com/apache/oozie/blob/oya/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java#L387


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174833
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174833
-----------------------------------------------------------




sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java
Lines 312-318 (original)
<https://reviews.apache.org/r/54705/#comment248052>

    Don't we need this to capture System.exit calls?



sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java
Lines 247-253 (original)
<https://reviews.apache.org/r/54705/#comment248053>

    Same as previous


- Robert Kanter


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 12, 2017, 8:27 p.m., Robert Kanter wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java
> > Lines 284 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714750#file1714750line284>
> >
> >     What should we do about this?  I don't think there's really a reason the user should play with this, so I'm thinking we should leave this as hardcoded and remove the TODO.
> 
> Rohini Palaniswamy wrote:
>     For cases like this, please use the value of mapreduce settings if it is set instead of coming up with a new setting name or trying to hard code some value. In our clusters we use a higher value to put less load on RM.

OOZIE-2894


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174834
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 12, 2017, 8:27 p.m., Robert Kanter wrote:
> > sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java
> > Lines 379 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714761#file1714761line379>
> >
> >     Please file a followup JIRA for this if we're not going to figure it out now.

https://issues.apache.org/jira/browse/OOZIE-2895


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174834
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.


> On May 12, 2017, 8:27 p.m., Robert Kanter wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java
> > Lines 284 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714750#file1714750line284>
> >
> >     What should we do about this?  I don't think there's really a reason the user should play with this, so I'm thinking we should leave this as hardcoded and remove the TODO.

For cases like this, please use the value of mapreduce settings if it is set instead of coming up with a new setting name or trying to hard code some value. In our clusters we use a higher value to put less load on RM.


- Rohini


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174834
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174834
-----------------------------------------------------------




sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java
Lines 61 (patched)
<https://reviews.apache.org/r/54705/#comment248054>

    Replace with finalPath.toUri() - no need to create a duplicate Path object.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java
Lines 284 (patched)
<https://reviews.apache.org/r/54705/#comment248059>

    What should we do about this?  I don't think there's really a reason the user should play with this, so I'm thinking we should leave this as hardcoded and remove the TODO.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java
Lines 222-226 (patched)
<https://reviews.apache.org/r/54705/#comment248060>

    Can be combined into a multi-catch



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java
Lines 260-264 (patched)
<https://reviews.apache.org/r/54705/#comment248061>

    Multicatch



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
Lines 57 (patched)
<https://reviews.apache.org/r/54705/#comment248062>

    We should delete this class.



sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java
Lines 379 (patched)
<https://reviews.apache.org/r/54705/#comment248063>

    Please file a followup JIRA for this if we're not going to figure it out now.


- Robert Kanter


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 10, 2017, 11:54 p.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java
> > Line 47 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714633#file1714633line47>
> >
> >     I think this is the only place where getClassNamesByType is called, in which case, we can also delete that.
> >     
> >     While I can see that this code isn't currently doing anything, so it should be safe to remove, it would be good to figure out what it was supposed to be doing originally.

I'm not that familiar with Distcp so I don't know if removing this will cause issues or not. Created https://issues.apache.org/jira/browse/OOZIE-2913


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174588
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 10, 2017, 11:54 p.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java
> > Lines 108 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714632#file1714632line108>
> >
> >     I imagine there's one of these in some Commons library or something?  At the very least, this should go into a Utils class.
> 
> Peter Cseh wrote:
>     This is comming from here: https://github.com/apache/oozie/blame/master/core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java#L91 
>     We'll look for a better place for it.

We've found this piece of code in DistCpActionExecutor as well.
 OOZIE-2914 - Consolidate Trim


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174588
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 10, 2017, 11:54 p.m., Robert Kanter wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java
> > Lines 108 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714632#file1714632line108>
> >
> >     I imagine there's one of these in some Commons library or something?  At the very least, this should go into a Utils class.

This is comming from here: https://github.com/apache/oozie/blame/master/core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java#L91 
We'll look for a better place for it.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174588
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174588
-----------------------------------------------------------




core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java
Lines 29 (patched)
<https://reviews.apache.org/r/54705/#comment247758>

    "... should modify the configuration which..."



core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java
Lines 108 (patched)
<https://reviews.apache.org/r/54705/#comment247759>

    I imagine there's one of these in some Commons library or something?  At the very least, this should go into a Utils class.



core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java
Line 47 (original)
<https://reviews.apache.org/r/54705/#comment247760>

    I think this is the only place where getClassNamesByType is called, in which case, we can also delete that.
    
    While I can see that this code isn't currently doing anything, so it should be safe to remove, it would be good to figure out what it was supposed to be doing originally.



core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java
Lines 189 (patched)
<https://reviews.apache.org/r/54705/#comment247761>

    Do we need this?



core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java
Line 49 (original), 49 (patched)
<https://reviews.apache.org/r/54705/#comment247763>

    Not "job conf" anymore.



core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java
Line 92 (original), 92 (patched)
<https://reviews.apache.org/r/54705/#comment247764>

    We should use the {0} syntax to avoid String concatination.



core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java
Line 102 (original), 102 (patched)
<https://reviews.apache.org/r/54705/#comment247765>

    Same here.  Also check for others, especially on busy code paths.



core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java
Lines 105-106 (patched)
<https://reviews.apache.org/r/54705/#comment247766>

    Unnecessary new lines


- Robert Kanter


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java
> > Line 51 (original), 45 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714690#file1714690line51>
> >
> >     Why is ./ required instead of just script.sh?

This was a tricky thing that I didn't understand before. We have to update $PATH to contain the current working directory. I can imagine there are users who depend on this.

OOZIE-2896 will address handling MR properties (ShellActionExecutor updates "mapred.map.env" but it's ignored right now).

But it's better to update ShellActionExecutor itself, that is, extend addActionSpecificEnvVars(). Will do this soon.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On máj. 10, 2017, 10:46 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated máj. 10, 2017, 10:46 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
> > Line 2076 (original), 1791 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714684#file1714684line2084>
> >
> >     -Xmx2048m should work if we are honoring the current configuration of users increasing launcher memory.

will be part of https://issues.apache.org/jira/browse/OOZIE-2896


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
> > Lines 413 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714645#file1714645line454>
> >
> >     This code will have race condition issues. super.check() will check the status of launcher AM if action data file is not found. While it starts the check, if it completes then it will read the NEW_ID from action data and mark the action as completed even though the mapreduce job is just launched and is still running.

That's right. I think the whole logic inside the check() methods have to be rewritten and refactored properly. It's too convoluted and responsibilities are in the wrong place.

Created follow-up JIRA: https://issues.apache.org/jira/browse/OOZIE-2915


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On máj. 10, 2017, 10:46 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated máj. 10, 2017, 10:46 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Line 423 (original), 321 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line463>
> >
> >     This should not be required anymore. The Oozie AM code should be doing the same thing as done in MAPREDUCE-6472.
> 
> Peter Bacsko wrote:
>     Right - deleted

Code to be added to Oozie AM is still not done. Refer https://issues.apache.org/jira/secure/attachment/12755788/MAPREDUCE-6472.20150914-2.patch


- Rohini


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 20, 2017, 11:55 a.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 20, 2017, 11:55 a.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 06ae5fd9a52099c5bc63b038d336ce96103c8819 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml b22b1ce64e5cdf50359479eec6944caf08f44c7b 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java b27b3d8a2b0ce55df70e55c2cb525be0c6573700 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaMain.java bb2df43d0d0f9e1630b53ff2c9525e2f949bbcbd 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml c4a1c25295b264e6cae66db3a3c84fdde0109f13 
>   release-log.txt ccf8a125df15a1e0140a8c860eb9f0b75080efbb 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8657c678c9786e1af20a2124214bc5106fe31223 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherMapper.java 51b1d6f8ccf1b64d2cb4b6a4da36b55c0888c259 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/9/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
> > Line 1788 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714684#file1714684line1796>
> >
> >     Oozie AM needs to support the mapreduce view and modify acls. This feature is heavily used by users for security in a multi-tenant environment. Please create a new jira for this to be addressed after merge.

Created follow-up JIRA: https://issues.apache.org/jira/browse/OOZIE-2897


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
> > Line 37 (original), 24 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714722#file1714722line37>
> >
> >     Any reason for not having this method as static and instead creating a new instance of LauncherAMCallbackNotifierFactory everytime?

Unit tests. We insert a mock factory because we don't want to invoke methods on a real callback notifier instance.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
> > Line 44 (original), 26 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714723#file1714723line45>
> >
> >     Create a new class instead of moving HadoopShims as they are totally unrelated

This wasn't a move, I suppose Git thought it was a file rename/move.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java
> > Lines 64 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714748#file1714748line64>
> >
> >     Why do we need a factory for creating a SequenceFile? It is just a few lines of code and it is a direct Hadoop API. If there were different formats for the file  or different ways to create a sequence file it would make sense to have a factory.

This helps greatly with writing unit tests for LauncherAM. An implementation of SequenceFile requires a Hadoop configuration, wants to create a FileSystem object, etc. Much easier with mocks.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java
> > Line 41 (original), 41-42 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714756#file1714756line41>
> >
> >     This is a internal class. Why not totally get rid of it instead of deprecating?

Created follow-up JIRA: https://issues.apache.org/jira/browse/OOZIE-2907


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java
> > Lines 49 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714760#file1714760line49>
> >
> >     Do we need this test class? uploadActionDataToHDFS is already tested during execution of many tests - mapreduce action tests or when we check for external ids. Just trying to avoid unnecessary tests as test run time is constantly going up.

This is a super-fast unit test (doesn't extend XTestCase). IMO it's worth keeping it.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On máj. 10, 2017, 10:46 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated máj. 10, 2017, 10:46 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/command/XCommand.java
> > Line 247 (original), 247 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714653#file1714653line247>
> >
> >     Unnecessary thread local call when there are no commands to be queued. Can we make getInterruptTypes as static method in CallableQueueService keeping the HashSet as private static?

The set is only populated when CallableQueueService.init is called on an instance. I would prefer to keep it an instance variable so we can't call it when the service is not initialized properly. I know that it's a rare case but calling a map.get() should not result in performance degradation.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > Have asked for new jiras for enhancements, missing scenarios or changes that require more work. Please reply with the new jira link before closing those issues in review board for easier referencing. Also add a TODO in the code with the jira number.
> > 
> > Some more action items:
> > 1) Getting tokens for ATS (Application Timeline Server) is missing. Please create a separate jira for that.
> > 2) Getting tokens for oozie.launcher.mapreduce.job.hdfs-servers is missing. Cannot do cross cluster distcp without that. Please create a separate jira for that.
> > 3) There should be code to get the Oozie AM application state from AHS (Application History Server) in check() if the completed job is gone from RM memory. For mapreduce launcher jobs, JobClient goes to Job History server to fetch that information. Please create a separate jira for that.

https://issues.apache.org/jira/browse/OOZIE-2899
https://issues.apache.org/jira/browse/OOZIE-2900
https://issues.apache.org/jira/browse/OOZIE-2901


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
> > Line 2110 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714684#file1714684line2118>
> >
> >     Instead of getting rid of the testUpdateConfForUberMode* tests altogether, it would be better to adapt them to the Oozie AM (once it is fixed to handle oozie.launcher configurations) as the tests are still valid. Comment them out for now with a TODO and address in the new jira after the merge.

https://issues.apache.org/jira/browse/OOZIE-2912


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java
> > Line 436 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714690#file1714690line436>
> >
> >     Please adapt this test to the Oozie AM instead of removing it similar to testUpdateConfForUberMode tests

https://issues.apache.org/jira/browse/OOZIE-2912


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java
> > Lines 39-44 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714751#file1714751line39>
> >
> >     Would be better to reuse mapreduce.job.end-notification. as prefix. Else will break backward compatibility and existing users will have to change if they have configured them.

This will be handled in OOZIE-2896


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On máj. 10, 2017, 10:46 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated máj. 10, 2017, 10:46 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java
> > Lines 777 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714662#file1714662line790>
> >
> >     os needs to be closed.

It's closed by java in try-with-resouces https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Line 325 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line365>
> >
> >     We still need the java opts, memory.mb and env settings specified by the user using oozie.launcher. settings applied to the Oozie AM. Please create a new jira for this.
> 
> Rohini Palaniswamy wrote:
>     This is different from OOZIE-2687 which introduces a option xml section to simplify specification of those launcher settings. Existing oozie.launcher MR settings specified by users should continute to work as is without user having to rewrite their workflow.xml. Else this will be a major backward incompatibility.

OOZIE-2896


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java
> > Line 25 (original), 28 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714630#file1714630line29>
> >
> >     Shouldn't this be just a new class instead of moving Credentials as they are totally unrelated?

This was not a rename, git is messing up something. We'll check it.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
> > Line 546 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714684#file1714684line548>
> >
> >     Why is this test removed? This is important.

Yes, it is - good catch. It's not obvious to me how this disappeared and why.
Created follow-up JIRA: https://issues.apache.org/jira/browse/OOZIE-2910


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java
> > Line 120 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714702#file1714702line124>
> >
> >     Why remove this test? This is important.

Created follow-up JIRA: https://issues.apache.org/jira/browse/OOZIE-2911


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java
> > Lines 55 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714748#file1714748line55>
> >
> >     Why are we doing ugi.doAs for all filesytem operations in this class? If they are all being called inside the launcher, ugi.doAs should not be required because Oozie AM should be running as the user ugi.  If it is broken for some reason then it is really bad as user calls in java action will also be impacted.

Yep, single doAs is enough -> OOZIE-2909


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On máj. 10, 2017, 10:46 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated máj. 10, 2017, 10:46 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
> > Line 30 (original), 22 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714722#file1714722line30>
> >
> >     Shouldn't this be just a new class instead of moving LauncherMainHadoopUtils as they are totally unrelated?

Will check.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java
> > Lines 295 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714750#file1714750line295>
> >
> >     Can't we use log4j (syslog instead of stdout) for Oozie AM messages? There will be no timestamp with S.O.P and debugging will be really hard. Can be a new jira.

https://issues.apache.org/jira/browse/OOZIE-2702 and https://issues.apache.org/jira/browse/OOZIE-2898 are opened to address this


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java
> > Lines 93 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714751#file1714751line93>
> >
> >     Use log4j instead of S.O.P and S.E.P

https://issues.apache.org/jira/browse/OOZIE-2898


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java
> > Lines 172-175 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714772#file1714772line172>
> >
> >     Shouldn't we retain this?

It's handled in LauncherAM for every kind of Main class.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Bacsko <pb...@cloudera.com>.


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Line 423 (original), 321 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line463>
> >
> >     This should not be required anymore. The Oozie AM code should be doing the same thing as done in MAPREDUCE-6472.

Right - deleted


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java
> > Line 130 (original), 126 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714647#file1714647line137>
> >
> >     Can you rename method as addMapreduceToClasspath?

Renamed to needToAddMapReduceToClassPath()


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > sharelib/oozie/pom.xml
> > Line 66 (original), 66 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714743#file1714743line66>
> >
> >     Shouldn't scope be test?

Yep, added


> On máj. 14, 2017, 4:16 de, Rohini Palaniswamy wrote:
> > sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java
> > Lines 119-127 (patched)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714750#file1714750line119>
> >
> >     We cannot be doing createRemoteUser with submitter.user. Oozie AM should be launched as the ugi the whole process should run as similar to current mapreduce launcher. Proxy users only are supported when run from specific machines as they are super users (For eg: Oozie is specified as a super user in NN, RM and JT configs and oozie server hosts are specified there so that do as from any other host is rejected). Basically there should not be any ugi.doAs() in launcher code.

I don't completely follow this reasoning. This is very similar to MRAppMaster: https://github.com/apache/hadoop/blob/3ed3062fe3979ff55a411b730a8eee2b2c96d6b3/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java#L1718-L1726

As far as I understand, proxy user support are only required on hosts where Oozie is running so that Oozie can retrieve the tokens for the impersonated user. 

There are two things which we probably should do differently:
1) For some reason, I had to use getLoginUser() in a Kerberized cluster because createRemoteUser() didn't work (RPC call got stuck in RetryInvocationHandler). I haven't been able to figure out what's missing, could be something minor
2) As you pointed out in a different comment, a single doAs() call should be sufficient for the whole LauncherAM, there's no need for separate calls

I opened a JIRA for these improvements: https://issues.apache.org/jira/browse/OOZIE-2909


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On máj. 10, 2017, 10:46 du, Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated máj. 10, 2017, 10:46 du)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.


> On May 14, 2017, 4:16 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Line 325 (original)
> > <https://reviews.apache.org/r/54705/diff/8/?file=1714643#file1714643line365>
> >
> >     We still need the java opts, memory.mb and env settings specified by the user using oozie.launcher. settings applied to the Oozie AM. Please create a new jira for this.

This is different from OOZIE-2687 which introduces a option xml section to simplify specification of those launcher settings. Existing oozie.launcher MR settings specified by users should continute to work as is without user having to rewrite their workflow.xml. Else this will be a major backward incompatibility.


- Rohini


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review174883
-----------------------------------------------------------



Have asked for new jiras for enhancements, missing scenarios or changes that require more work. Please reply with the new jira link before closing those issues in review board for easier referencing. Also add a TODO in the code with the jira number.

Some more action items:
1) Getting tokens for ATS (Application Timeline Server) is missing. Please create a separate jira for that.
2) Getting tokens for oozie.launcher.mapreduce.job.hdfs-servers is missing. Cannot do cross cluster distcp without that. Please create a separate jira for that.
3) There should be code to get the Oozie AM application state from AHS (Application History Server) in check() if the completed job is gone from RM memory. For mapreduce launcher jobs, JobClient goes to Job History server to fetch that information. Please create a separate jira for that.


core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java
Line 25 (original), 28 (patched)
<https://reviews.apache.org/r/54705/#comment248115>

    Shouldn't this be just a new class instead of moving Credentials as they are totally unrelated?



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 156-158 (original), 177-179 (patched)
<https://reviews.apache.org/r/54705/#comment248144>

    Do we still need the input and output format classes?



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Line 325 (original)
<https://reviews.apache.org/r/54705/#comment248146>

    We still need the java opts, memory.mb and env settings specified by the user using oozie.launcher. settings applied to the Oozie AM. Please create a new jira for this.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Line 423 (original), 321 (patched)
<https://reviews.apache.org/r/54705/#comment248145>

    This should not be required anymore. The Oozie AM code should be doing the same thing as done in MAPREDUCE-6472.



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Line 1147 (original)
<https://reviews.apache.org/r/54705/#comment248150>

    Can you retain this debug message? Has come in useful debugging slowness in app submissions



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 1133 (patched)
<https://reviews.apache.org/r/54705/#comment248151>

    You have to translate them from mapreduce settings instead of setting own defaults. Else it will break lot of existing user jobs which have been tuned to have more memory.  Please create a new jira for this (same issue as mentioned in updateConfForUberMode comment)



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 1292-1293 (original), 1187-1188 (patched)
<https://reviews.apache.org/r/54705/#comment248154>

    Nitpick. Unnecessary local variable assignment. CredentialsProviderFactory.getInstance().createCredentialsProvider(credProps.getType());



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Line 1622 (original), 1514 (patched)
<https://reviews.apache.org/r/54705/#comment248163>

    Put just the kill call within the loop inside try catch block and log a warning if it cannot be killed. Killing an already completed child application may throw exception and may prevent other child applications from being killed.



core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
Lines 413 (patched)
<https://reviews.apache.org/r/54705/#comment248162>

    This code will have race condition issues. super.check() will check the status of launcher AM if action data file is not found. While it starts the check, if it completes then it will read the NEW_ID from action data and mark the action as completed even though the mapreduce job is just launched and is still running.



core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java
Line 130 (original), 126 (patched)
<https://reviews.apache.org/r/54705/#comment248110>

    Can you rename method as addMapreduceToClasspath?



core/src/main/java/org/apache/oozie/command/XCommand.java
Line 247 (original), 247 (patched)
<https://reviews.apache.org/r/54705/#comment248111>

    Unnecessary thread local call when there are no commands to be queued. Can we make getInterruptTypes as static method in CallableQueueService keeping the HashSet as private static?



core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java
Lines 777 (patched)
<https://reviews.apache.org/r/54705/#comment248113>

    os needs to be closed.



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
Line 546 (original)
<https://reviews.apache.org/r/54705/#comment248116>

    Why is this test removed? This is important.



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
Line 1788 (original)
<https://reviews.apache.org/r/54705/#comment248117>

    Oozie AM needs to support the mapreduce view and modify acls. This feature is heavily used by users for security in a multi-tenant environment. Please create a new jira for this to be addressed after merge.



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
Line 2076 (original), 1791 (patched)
<https://reviews.apache.org/r/54705/#comment248118>

    -Xmx2048m should work if we are honoring the current configuration of users increasing launcher memory.



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
Line 2110 (original)
<https://reviews.apache.org/r/54705/#comment248119>

    Instead of getting rid of the testUpdateConfForUberMode* tests altogether, it would be better to adapt them to the Oozie AM (once it is fixed to handle oozie.launcher configurations) as the tests are still valid. Comment them out for now with a TODO and address in the new jira after the merge.



core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java
Line 51 (original), 45 (patched)
<https://reviews.apache.org/r/54705/#comment248120>

    Why is ./ required instead of just script.sh?



core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java
Line 436 (original)
<https://reviews.apache.org/r/54705/#comment248121>

    Please adapt this test to the Oozie AM instead of removing it similar to testUpdateConfForUberMode tests



core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java
Line 120 (original)
<https://reviews.apache.org/r/54705/#comment248122>

    Why remove this test? This is important.



core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java
Lines 144-146 (original), 153-155 (patched)
<https://reviews.apache.org/r/54705/#comment248123>

    Why remove?



docs/src/site/twiki/ENG_Building.twiki
Line 115 (original), 115 (patched)
<https://reviews.apache.org/r/54705/#comment248125>

    Remove (Hadoop-2) from here.



docs/src/site/twiki/ENG_Building.twiki
Line 117 (original), 117 (patched)
<https://reviews.apache.org/r/54705/#comment248124>

    The default version is 2.6.0 and that is the minimum supported Hadoop version.



hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
Line 30 (original), 22 (patched)
<https://reviews.apache.org/r/54705/#comment248137>

    Shouldn't this be just a new class instead of moving LauncherMainHadoopUtils as they are totally unrelated?



hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
Line 37 (original), 24 (patched)
<https://reviews.apache.org/r/54705/#comment248138>

    Any reason for not having this method as static and instead creating a new instance of LauncherAMCallbackNotifierFactory everytime?



hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
Line 44 (original), 26 (patched)
<https://reviews.apache.org/r/54705/#comment248142>

    Create a new class instead of moving HadoopShims as they are totally unrelated



sharelib/oozie/pom.xml
Line 66 (original), 66 (patched)
<https://reviews.apache.org/r/54705/#comment248126>

    Shouldn't scope be test?



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java
Lines 25 (patched)
<https://reviews.apache.org/r/54705/#comment248164>

    static



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java
Lines 55 (patched)
<https://reviews.apache.org/r/54705/#comment248166>

    Why are we doing ugi.doAs for all filesytem operations in this class? If they are all being called inside the launcher, ugi.doAs should not be required because Oozie AM should be running as the user ugi.  If it is broken for some reason then it is really bad as user calls in java action will also be impacted.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java
Lines 64 (patched)
<https://reviews.apache.org/r/54705/#comment248165>

    Why do we need a factory for creating a SequenceFile? It is just a few lines of code and it is a direct Hadoop API. If there were different formats for the file  or different ways to create a sequence file it would make sense to have a factory.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java
Lines 119-127 (patched)
<https://reviews.apache.org/r/54705/#comment248167>

    We cannot be doing createRemoteUser with submitter.user. Oozie AM should be launched as the ugi the whole process should run as similar to current mapreduce launcher. Proxy users only are supported when run from specific machines as they are super users (For eg: Oozie is specified as a super user in NN, RM and JT configs and oozie server hosts are specified there so that do as from any other host is rejected). Basically there should not be any ugi.doAs() in launcher code.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java
Lines 295 (patched)
<https://reviews.apache.org/r/54705/#comment248169>

    Can't we use log4j (syslog instead of stdout) for Oozie AM messages? There will be no timestamp with S.O.P and debugging will be really hard. Can be a new jira.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java
Lines 39-44 (patched)
<https://reviews.apache.org/r/54705/#comment248170>

    Would be better to reuse mapreduce.job.end-notification. as prefix. Else will break backward compatibility and existing users will have to change if they have configured them.



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java
Lines 93 (patched)
<https://reviews.apache.org/r/54705/#comment248171>

    Use log4j instead of S.O.P and S.E.P



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
Line 142 (original), 136 (patched)
<https://reviews.apache.org/r/54705/#comment248139>

    Remove this method and directly call ActionUtils



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
Lines 141 (patched)
<https://reviews.apache.org/r/54705/#comment248140>

    Remove this method and directly call ActionUtils



sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java
Line 41 (original), 41-42 (patched)
<https://reviews.apache.org/r/54705/#comment248141>

    This is a internal class. Why not totally get rid of it instead of deprecating?



sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java
Lines 49 (patched)
<https://reviews.apache.org/r/54705/#comment248172>

    Do we need this test class? uploadActionDataToHDFS is already tested during execution of many tests - mapreduce action tests or when we check for external ids. Just trying to avoid unnecessary tests as test run time is constantly going up.



sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java
Lines 172-175 (original)
<https://reviews.apache.org/r/54705/#comment248143>

    Shouldn't we retain this?


- Rohini Palaniswamy


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated May 10, 2017, 10:46 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Bugs: OOZIE-1770
    https://issues.apache.org/jira/browse/OOZIE-1770


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
  core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
  docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
  release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 


Diff: https://reviews.apache.org/r/54705/diff/8/

Changes: https://reviews.apache.org/r/54705/diff/7-8/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated May 10, 2017, 9:24 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Bugs: OOZIE-1770
    https://issues.apache.org/jira/browse/OOZIE-1770


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
  core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki fdc837417cda19dc5daa05793eba6136531b5d8e 
  docs/src/site/twiki/ENG_Building.twiki 019f8f1169fdc7fbfdcc0bc78ebf2f2b87192e4d 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml 5c6edf140048bdc32199444630ed0c13302a7d6f 
  release-log.txt 0128b3ba6e7b598dbf7043bbe3940576938c312f 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 


Diff: https://reviews.apache.org/r/54705/diff/7/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated May 10, 2017, 9:13 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Bugs: OOZIE-1779
    https://issues.apache.org/jira/browse/OOZIE-1779


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
  core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki fdc837417cda19dc5daa05793eba6136531b5d8e 
  docs/src/site/twiki/ENG_Building.twiki 019f8f1169fdc7fbfdcc0bc78ebf2f2b87192e4d 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml 5c6edf140048bdc32199444630ed0c13302a7d6f 
  release-log.txt 0128b3ba6e7b598dbf7043bbe3940576938c312f 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 


Diff: https://reviews.apache.org/r/54705/diff/7/

Changes: https://reviews.apache.org/r/54705/diff/6-7/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated May 9, 2017, 10:48 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Bugs: OOZIE-1779
    https://issues.apache.org/jira/browse/OOZIE-1779


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
  core/src/main/java/org/apache/oozie/CoordinatorEngine.java.orig 2c04beaa5be1d9deca8effbd138f50c84d8b85a6 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java f62c99700f715d7afaa35b5657a837f9dc884103 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 969336d27ce5cd6535c926b0883ff5b7f610ce72 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml e7a48a03f4b4a3c6c00274c0650933057091e537 
  core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java dcc97ad070a7a0d30c5f93e9347f187a32e4a52b 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  core/src/test/resources/coord-invalid-el-function.xml PRE-CREATION 
  core/src/test/resources/coord-invalid-output-instance.xml PRE-CREATION 
  core/src/test/resources/coord-multiple-input-instance4.xml 023df26a341991c07d81c0d0d27d1fe65c03d29f 
  core/src/test/resources/coord-multiple-input-start-instance2.xml 83d37733f98732b1a4042d879198cd0284da0c94 
  core/src/test/resources/coord-multiple-output-instance4.xml 32d07865417dd5d227cc23bf6143eb29d311faa3 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/CoordinatorFunctionalSpec.twiki 3828eb5bca9aded801a01e13f09e42ead08cc6e0 
  docs/src/site/twiki/DG_QuickStart.twiki fdc837417cda19dc5daa05793eba6136531b5d8e 
  docs/src/site/twiki/ENG_Building.twiki 019f8f1169fdc7fbfdcc0bc78ebf2f2b87192e4d 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml 5c6edf140048bdc32199444630ed0c13302a7d6f 
  release-log.txt 38762bdcadb37803a2e2ac6854a95ca8c3cdf57c 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 


Diff: https://reviews.apache.org/r/54705/diff/6/

Changes: https://reviews.apache.org/r/54705/diff/5-6/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java
> > Lines 112-114 (original)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671955#file1671955line113>
> >
> >     You can't remove this. Will breaking existing users. Since it starts with oozie.launcher, config deprecation will not handle it. Basically any oozie.launcher hadoop 1.x settings cannot be removed because of that.
> 
> Peter Cseh wrote:
>     This is setting "mapreduce.task.classpath.user.precedence" in the launcher job conf, it was parsed by the launcher Mapper and the LauncherAM is not doing anything with this property at the moment. Should we add similar functionality into LauncherAM? Is this a must have to support without MR launcher?
> 
> Robert Kanter wrote:
>     I'd rather not have to support an equivalent of "mapreduce.task.classpath.user.precedence" in the Oozie AM itself.  It's a really ugly hacky thing.  If we want to go down this route, a better fix would be to provide complete classpath isolation somehow; but I think that could be a future feature.

We need to provide feature parity for both mapreduce.task.classpath.user.precedence (simple hack, but very useful and addressed most of cases) and mapreduce.job.classloader/mapreduce.job.classloader.system.classes (this provided the isolation in the mapreduce world). Please create a separate jira for this issue that can be addressed after the merge. Have seen a lot of users use this feature due to one conflict or another. One of the most common cases that requires this is avro.


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java
> > Line 89 (original), 87 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671956#file1671956line95>
> >
> >     Revert back to old code. Just remove the final instead of creating another temporary list.
> 
> Peter Cseh wrote:
>     merge issue. will check

You can totally avoid the array list to array conversion by using String.split() instead of StringTokenizer. 

https://docs.oracle.com/javase/7/docs/api/java/util/StringTokenizer.html
StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code.


- Rohini


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review170061
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java
> > Line 89 (original), 87 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671956#file1671956line95>
> >
> >     Revert back to old code. Just remove the final instead of creating another temporary list.
> 
> Peter Cseh wrote:
>     merge issue. will check
> 
> Rohini Palaniswamy wrote:
>     You can totally avoid the array list to array conversion by using String.split() instead of StringTokenizer. 
>     
>     https://docs.oracle.com/javase/7/docs/api/java/util/StringTokenizer.html
>     StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code.

This is not in the latest patch.


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review170061
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Robert Kanter <rk...@cloudera.com>.


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java
> > Lines 112-114 (original)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671955#file1671955line113>
> >
> >     You can't remove this. Will breaking existing users. Since it starts with oozie.launcher, config deprecation will not handle it. Basically any oozie.launcher hadoop 1.x settings cannot be removed because of that.
> 
> Peter Cseh wrote:
>     This is setting "mapreduce.task.classpath.user.precedence" in the launcher job conf, it was parsed by the launcher Mapper and the LauncherAM is not doing anything with this property at the moment. Should we add similar functionality into LauncherAM? Is this a must have to support without MR launcher?

I'd rather not have to support an equivalent of "mapreduce.task.classpath.user.precedence" in the Oozie AM itself.  It's a really ugly hacky thing.  If we want to go down this route, a better fix would be to provide complete classpath isolation somehow; but I think that could be a future feature.


- Robert


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review170061
-----------------------------------------------------------


On March 22, 2017, 8:34 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated March 22, 2017, 8:34 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1779
>     https://issues.apache.org/jira/browse/OOZIE-1779
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   bin/test-patch-20-tests 22f472c07e80fdb38c632329a5707998ab352172 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/cli/ValidationUtil.java b1da2af070970add01031d6208a395520cd7055c 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 73708080e624e775f7d058cd3377ad69d8399264 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ca168e084998b5dd564536078a7a69efd7291828 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 8ddb1f873f0c69aadf5f4a5161d8b21f7392fa1e 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 39daca7b3050c259a36d54710b4c1279f3acdeac 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 5fd30f7858c5c4c70dcbdb6df86904518d92f7f3 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 2ba1999ba7d5e892bd7e37d5fd8d1f926a84b48b 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java 52e03d9839f01e8fcbc9c0fdee179467b9ac5c98 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 4cc284ae52ac50cce36c251536f045804e351f7a 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/PurgeService.java c2c966b2cd056b77ae52164cb3f40955e98ae69a 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java d263567eab9fc9b8763296c35e0a904499eabc11 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/servlet/JVMInfo.java bf2b01db340a014d8831c5186188741f866e6559 
>   core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java d4a098949534f417938a65c27e395c9ac1f71ca7 
>   core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java fc3a6bc19d6a04e489c3f65c8248478a58540ede 
>   core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java d08521ca28d9ae4c8582c6ce3a8e6ca35a26f8dd 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 83b455ebc2629fddccf44b61b1051f932e1d40a3 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 03f428a47e34f2d07ea8fff755f29d222d6aeb9a 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherHCatURIHandler.java 4288974b036b68b42d260d674aaaa0d345484123 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestScriptLanguageActionExecutor.java 0adc190d56ec163d454d64ae341b6a658ac14781 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 564db2a5150d949e2e9274984c1d67acce6b5447 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java e41df8719fa361adca5406ddf1da919f9e78016d 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 6fd1a1aa5eab59c085551be8e52fa0d01c79c4da 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java 1278efa0775c32c16cb38309ab8e92e08d4eb2ec 
>   core/src/test/java/org/apache/oozie/servlet/TestAdminServlet.java 8e3502c343744b7e70d2033643c9596c0c029778 
>   core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java a3f3b9c28c0f34381930a7457b5b622695948742 
>   core/src/test/java/org/apache/oozie/test/MiniHCatServer.java b2f7282a30bc8ed269d9c4208036421047647ef0 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 58f4beab9e707a3d8542253df596a566e65eff3c 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/TestIOUtils.java 6fcdf5526927a3b8033c7a0c747b7480a5f199e7 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 6a9e4b4bfd50954c399dd094c3fee82c491b6800 
>   docs/src/site/twiki/DG_HCatalogIntegration.twiki 39df67c3f5c5df0bb0521f613a66862c1b0a0ed8 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   docs/src/site/twiki/WebServicesAPI.twiki a63a68cb27de05d03ad173caeb18f5b17e20867b 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 460f6e521cb8fe2e508ec7086081d49890efb037 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 98a2359a3ca476a0516a9746da2c1e91857143fd 
>   release-log.txt 4e7097b17a2ab1036135b27e9bdf3554fa2923ae 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java fcaadefd215368565fb126ca92515792e1f1a09b 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java 7edb40d9d7cba6ef0603de25c9cee55f39b75f3b 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java faeff2a7f5f24f1680df5e529bbb48f49c750e09 
>   sharelib/hcatalog/src/test/java/org/apache/oozie/util/TestHCatURI.java ce690da986f3bbbab7f74e771b1fe9e32f047b27 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bc8ab29c25405806f40232d32254ba2a7f5271e4 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 7ae48e168a310b4d55feda7cbce8b2d29c2ec505 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java ba393eee53e92d144159f83893d4baab00737a44 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 14ab65e7d00c426036ced3e80400c80deea0e282 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java c24d95c88be0dbf6fe80241d887a70100a5e1b4a 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb8f56e9b66de0eb2799e68bc56d0ee36d4ba317 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/5/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java
> > Lines 112-114 (original)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671955#file1671955line113>
> >
> >     You can't remove this. Will breaking existing users. Since it starts with oozie.launcher, config deprecation will not handle it. Basically any oozie.launcher hadoop 1.x settings cannot be removed because of that.

This is setting "mapreduce.task.classpath.user.precedence" in the launcher job conf, it was parsed by the launcher Mapper and the LauncherAM is not doing anything with this property at the moment. Should we add similar functionality into LauncherAM? Is this a must have to support without MR launcher?


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java
> > Line 89 (original), 87 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671956#file1671956line95>
> >
> >     Revert back to old code. Just remove the final instead of creating another temporary list.

merge issue. will check


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java
> > Lines 109-111 (original)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671956#file1671956line119>
> >
> >     Why is this condition not necessary any more?

merge issue. will check


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java
> > Line 69 (original), 69 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671957#file1671957line69>
> >
> >     Revert. UNKNOWN_ERROR is correct

merge issue. will check


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/command/PurgeXCommand.java
> > Line 410 (original), 410 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671959#file1671959line410>
> >
> >     Why remove entity key?

merge issue. will check


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/command/PurgeXCommand.java
> > Line 418 (original), 418 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671959#file1671959line418>
> >
> >     Why is lock not required anymore?

merge issue. will check


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
> > Line 182 (original)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671967#file1671967line182>
> >
> >     Why is support for dropping table being removed?

merge issue. will check


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
> > Line 345 (original)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671967#file1671967line345>
> >
> >     Why is support for checking exists on a table being removed?

merge issue. will check


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/servlet/JVMInfo.java
> > Line 123 (original), 123 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671977#file1671977line123>
> >
> >     Revert. The original spelling of Daemon is correct.

merge issue. will check


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java
> > Line 386 (original), 386 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671980#file1671980line386>
> >
> >     Revert. threw exception is correct.

merge issue. will check


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review170061
-----------------------------------------------------------


On March 22, 2017, 8:34 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated March 22, 2017, 8:34 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1779
>     https://issues.apache.org/jira/browse/OOZIE-1779
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   bin/test-patch-20-tests 22f472c07e80fdb38c632329a5707998ab352172 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/cli/ValidationUtil.java b1da2af070970add01031d6208a395520cd7055c 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 73708080e624e775f7d058cd3377ad69d8399264 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ca168e084998b5dd564536078a7a69efd7291828 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 8ddb1f873f0c69aadf5f4a5161d8b21f7392fa1e 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 39daca7b3050c259a36d54710b4c1279f3acdeac 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 5fd30f7858c5c4c70dcbdb6df86904518d92f7f3 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 2ba1999ba7d5e892bd7e37d5fd8d1f926a84b48b 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java 52e03d9839f01e8fcbc9c0fdee179467b9ac5c98 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 4cc284ae52ac50cce36c251536f045804e351f7a 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/PurgeService.java c2c966b2cd056b77ae52164cb3f40955e98ae69a 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java d263567eab9fc9b8763296c35e0a904499eabc11 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/servlet/JVMInfo.java bf2b01db340a014d8831c5186188741f866e6559 
>   core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java d4a098949534f417938a65c27e395c9ac1f71ca7 
>   core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java fc3a6bc19d6a04e489c3f65c8248478a58540ede 
>   core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java d08521ca28d9ae4c8582c6ce3a8e6ca35a26f8dd 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 83b455ebc2629fddccf44b61b1051f932e1d40a3 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 03f428a47e34f2d07ea8fff755f29d222d6aeb9a 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherHCatURIHandler.java 4288974b036b68b42d260d674aaaa0d345484123 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestScriptLanguageActionExecutor.java 0adc190d56ec163d454d64ae341b6a658ac14781 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 564db2a5150d949e2e9274984c1d67acce6b5447 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java e41df8719fa361adca5406ddf1da919f9e78016d 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 6fd1a1aa5eab59c085551be8e52fa0d01c79c4da 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java 1278efa0775c32c16cb38309ab8e92e08d4eb2ec 
>   core/src/test/java/org/apache/oozie/servlet/TestAdminServlet.java 8e3502c343744b7e70d2033643c9596c0c029778 
>   core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java a3f3b9c28c0f34381930a7457b5b622695948742 
>   core/src/test/java/org/apache/oozie/test/MiniHCatServer.java b2f7282a30bc8ed269d9c4208036421047647ef0 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 58f4beab9e707a3d8542253df596a566e65eff3c 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/TestIOUtils.java 6fcdf5526927a3b8033c7a0c747b7480a5f199e7 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 6a9e4b4bfd50954c399dd094c3fee82c491b6800 
>   docs/src/site/twiki/DG_HCatalogIntegration.twiki 39df67c3f5c5df0bb0521f613a66862c1b0a0ed8 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   docs/src/site/twiki/WebServicesAPI.twiki a63a68cb27de05d03ad173caeb18f5b17e20867b 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 460f6e521cb8fe2e508ec7086081d49890efb037 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 98a2359a3ca476a0516a9746da2c1e91857143fd 
>   release-log.txt 4e7097b17a2ab1036135b27e9bdf3554fa2923ae 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java fcaadefd215368565fb126ca92515792e1f1a09b 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java 7edb40d9d7cba6ef0603de25c9cee55f39b75f3b 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java faeff2a7f5f24f1680df5e529bbb48f49c750e09 
>   sharelib/hcatalog/src/test/java/org/apache/oozie/util/TestHCatURI.java ce690da986f3bbbab7f74e771b1fe9e32f047b27 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bc8ab29c25405806f40232d32254ba2a7f5271e4 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 7ae48e168a310b4d55feda7cbce8b2d29c2ec505 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java ba393eee53e92d144159f83893d4baab00737a44 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 14ab65e7d00c426036ced3e80400c80deea0e282 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java c24d95c88be0dbf6fe80241d887a70100a5e1b4a 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb8f56e9b66de0eb2799e68bc56d0ee36d4ba317 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/5/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java
> > Lines 112-114 (original)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671955#file1671955line113>
> >
> >     You can't remove this. Will breaking existing users. Since it starts with oozie.launcher, config deprecation will not handle it. Basically any oozie.launcher hadoop 1.x settings cannot be removed because of that.
> 
> Peter Cseh wrote:
>     This is setting "mapreduce.task.classpath.user.precedence" in the launcher job conf, it was parsed by the launcher Mapper and the LauncherAM is not doing anything with this property at the moment. Should we add similar functionality into LauncherAM? Is this a must have to support without MR launcher?
> 
> Robert Kanter wrote:
>     I'd rather not have to support an equivalent of "mapreduce.task.classpath.user.precedence" in the Oozie AM itself.  It's a really ugly hacky thing.  If we want to go down this route, a better fix would be to provide complete classpath isolation somehow; but I think that could be a future feature.
> 
> Rohini Palaniswamy wrote:
>     We need to provide feature parity for both mapreduce.task.classpath.user.precedence (simple hack, but very useful and addressed most of cases) and mapreduce.job.classloader/mapreduce.job.classloader.system.classes (this provided the isolation in the mapreduce world). Please create a separate jira for this issue that can be addressed after the merge. Have seen a lot of users use this feature due to one conflict or another. One of the most common cases that requires this is avro.

https://issues.apache.org/jira/browse/OOZIE-2891


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review170061
-----------------------------------------------------------


On May 10, 2017, 10:46 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated May 10, 2017, 10:46 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1770
>     https://issues.apache.org/jira/browse/OOZIE-1770
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   core/pom.xml 14aa03483e107db326628ba627cc3ee9f5fd3f3d 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java d60a5c7b996a4085f9bc77d82f12ad6925280b1a 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 72ed2f150be73f0a5e8b71b0fb6cc3594dc23078 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 076401d8c984a7a8a70b639f19ea96aa3b3daaa7 
>   core/src/main/resources/oozie-log4j.properties 05fb37af2a04cfcc464ae98940767d0e5e802880 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java 386fef3b89a6342cd09d8e7841d8997e06af46c4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java d66d9c9d5c1f84ea11f4bf43e19bd9523933c6eb 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java d1f53fee46e6168ce4b6277af441ab06995c4502 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java f12927b89f6e4be36a5d3f6d8677684e503295ae 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java dbc160f0fa971fdce2a81b073a1af17ada3c6591 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java c071000b22d87c76fad3f18597ded604f8c2f732 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 3c6525dc7e1049726eebafe056efdc49851ea62a 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_QuickStart.twiki 7ea87c7a8d6fdcb3d016b63851c9b75fa80515f1 
>   docs/src/site/twiki/ENG_Building.twiki 535f1b04a4331409e0721bf9f1b4946e79443e10 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml ebe1d686a7c840b5713a8fe506337b4b3daf9d2b 
>   release-log.txt 5800715f223d7dd455b04ef36867e86fdede94d2 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 65e7c5ec9eff5e5bb8cb943ca9a8dfc1d82f67d9 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java 6a600fa0621f5aee697c15d931568742ba5215da 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 9a411acc2adf2907aae75fd02f4010aaec7cf8ef 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 11cc7ee83af5b7d9f1ea0cc6b6331952711f35cc 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 503d0ebb8801d8de01a490bce5fd14e7947fb66b 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java ffa934aa4d216ad57199cb30f625595228244444 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java 6672ffbc4460ade84f1a4090fd15217d4881e296 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml e4fdfb78589d76b0438e027b51dc563e11f1c58e 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/8/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.


> On March 26, 2017, 12:37 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java
> > Lines 86-87 (original), 71 (patched)
> > <https://reviews.apache.org/r/54705/diff/5/?file=1671975#file1671975line86>
> >
> >     You should retain the code of checking the resource name and throwing error here which is currently commented out instead of doing it in setOozieMode()

Merge issue


- Peter


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review170061
-----------------------------------------------------------


On March 22, 2017, 8:34 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated March 22, 2017, 8:34 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1779
>     https://issues.apache.org/jira/browse/OOZIE-1779
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   bin/test-patch-20-tests 22f472c07e80fdb38c632329a5707998ab352172 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/cli/ValidationUtil.java b1da2af070970add01031d6208a395520cd7055c 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 73708080e624e775f7d058cd3377ad69d8399264 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ca168e084998b5dd564536078a7a69efd7291828 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 8ddb1f873f0c69aadf5f4a5161d8b21f7392fa1e 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 39daca7b3050c259a36d54710b4c1279f3acdeac 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 5fd30f7858c5c4c70dcbdb6df86904518d92f7f3 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 2ba1999ba7d5e892bd7e37d5fd8d1f926a84b48b 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java 52e03d9839f01e8fcbc9c0fdee179467b9ac5c98 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 4cc284ae52ac50cce36c251536f045804e351f7a 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/PurgeService.java c2c966b2cd056b77ae52164cb3f40955e98ae69a 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java d263567eab9fc9b8763296c35e0a904499eabc11 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/servlet/JVMInfo.java bf2b01db340a014d8831c5186188741f866e6559 
>   core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java d4a098949534f417938a65c27e395c9ac1f71ca7 
>   core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java fc3a6bc19d6a04e489c3f65c8248478a58540ede 
>   core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java d08521ca28d9ae4c8582c6ce3a8e6ca35a26f8dd 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 83b455ebc2629fddccf44b61b1051f932e1d40a3 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 03f428a47e34f2d07ea8fff755f29d222d6aeb9a 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherHCatURIHandler.java 4288974b036b68b42d260d674aaaa0d345484123 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestScriptLanguageActionExecutor.java 0adc190d56ec163d454d64ae341b6a658ac14781 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 564db2a5150d949e2e9274984c1d67acce6b5447 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java e41df8719fa361adca5406ddf1da919f9e78016d 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 6fd1a1aa5eab59c085551be8e52fa0d01c79c4da 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java 1278efa0775c32c16cb38309ab8e92e08d4eb2ec 
>   core/src/test/java/org/apache/oozie/servlet/TestAdminServlet.java 8e3502c343744b7e70d2033643c9596c0c029778 
>   core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java a3f3b9c28c0f34381930a7457b5b622695948742 
>   core/src/test/java/org/apache/oozie/test/MiniHCatServer.java b2f7282a30bc8ed269d9c4208036421047647ef0 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 58f4beab9e707a3d8542253df596a566e65eff3c 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/TestIOUtils.java 6fcdf5526927a3b8033c7a0c747b7480a5f199e7 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 6a9e4b4bfd50954c399dd094c3fee82c491b6800 
>   docs/src/site/twiki/DG_HCatalogIntegration.twiki 39df67c3f5c5df0bb0521f613a66862c1b0a0ed8 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   docs/src/site/twiki/WebServicesAPI.twiki a63a68cb27de05d03ad173caeb18f5b17e20867b 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 460f6e521cb8fe2e508ec7086081d49890efb037 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 98a2359a3ca476a0516a9746da2c1e91857143fd 
>   release-log.txt 4e7097b17a2ab1036135b27e9bdf3554fa2923ae 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java fcaadefd215368565fb126ca92515792e1f1a09b 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java 7edb40d9d7cba6ef0603de25c9cee55f39b75f3b 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java faeff2a7f5f24f1680df5e529bbb48f49c750e09 
>   sharelib/hcatalog/src/test/java/org/apache/oozie/util/TestHCatURI.java ce690da986f3bbbab7f74e771b1fe9e32f047b27 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bc8ab29c25405806f40232d32254ba2a7f5271e4 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 7ae48e168a310b4d55feda7cbce8b2d29c2ec505 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java ba393eee53e92d144159f83893d4baab00737a44 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 14ab65e7d00c426036ced3e80400c80deea0e282 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java c24d95c88be0dbf6fe80241d887a70100a5e1b4a 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb8f56e9b66de0eb2799e68bc56d0ee36d4ba317 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/5/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Rohini Palaniswamy <ro...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/#review170061
-----------------------------------------------------------



Reviews for Pages 1-3 except JavaActionExecutor

Note: On a time crunch and doing review of couple of pages at a time to avoid postponing this for too long. Have also not had time to go through history of previous review comments from others. So there might be duplicates.


core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java
Lines 34 (patched)
<https://reviews.apache.org/r/54705/#comment242833>

    If you are calling this a factory, you should look into removing the constructor by type and making it a true Factory. Cache the credential type and the corresponding Class. createCredentialsProvider() will then become createCredentialsProvider(String type) which will just look up the cache and return a new instance of the Class. 
    
    Parsing ConfigurationService.getStrings(CRED_KEY) and loading the class everytime is a lot of waste.



core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java
Lines 54 (patched)
<https://reviews.apache.org/r/54705/#comment242898>

    You could just call it LauncherHelper.



core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
Lines 319 (patched)
<https://reviews.apache.org/r/54705/#comment242899>

    Typo. original



core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
Lines 370 (patched)
<https://reviews.apache.org/r/54705/#comment242900>

    Remove this warn and add e to the next warning logging.



core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
Lines 406-407 (patched)
<https://reviews.apache.org/r/54705/#comment242902>

    1) Can we kill directly if externalChild id is not empty (MR job already launched) else call  LauncherMainHadoopUtils.killChildYarnJobs? If it is empty we have to call killChildYarnJobs as the launcher AM could have just launched the mapreduce job before kill reached it.
    2) Why are we duplicating the killChildYarnJobs code here?



core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java
Lines 112-114 (original)
<https://reviews.apache.org/r/54705/#comment242903>

    You can't remove this. Will breaking existing users. Since it starts with oozie.launcher, config deprecation will not handle it. Basically any oozie.launcher hadoop 1.x settings cannot be removed because of that.



core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java
Line 89 (original), 87 (patched)
<https://reviews.apache.org/r/54705/#comment242904>

    Revert back to old code. Just remove the final instead of creating another temporary list.



core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java
Lines 109-111 (original)
<https://reviews.apache.org/r/54705/#comment242905>

    Why is this condition not necessary any more?



core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java
Line 69 (original), 69 (patched)
<https://reviews.apache.org/r/54705/#comment242906>

    Revert. UNKNOWN_ERROR is correct



core/src/main/java/org/apache/oozie/command/PurgeXCommand.java
Line 410 (original), 410 (patched)
<https://reviews.apache.org/r/54705/#comment242907>

    Why remove entity key?



core/src/main/java/org/apache/oozie/command/PurgeXCommand.java
Line 418 (original), 418 (patched)
<https://reviews.apache.org/r/54705/#comment242908>

    Why is lock not required anymore?



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
Line 182 (original)
<https://reviews.apache.org/r/54705/#comment242835>

    Why is support for dropping table being removed?



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
Line 345 (original)
<https://reviews.apache.org/r/54705/#comment242836>

    Why is support for checking exists on a table being removed?



core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java
Lines 776-777 (patched)
<https://reviews.apache.org/r/54705/#comment242893>

    This is going to fill up the temp disk space pretty quickly. Please avoid creating the local file and write directly to the FSDataOutputStream after creating the file in HDFS.



core/src/main/java/org/apache/oozie/service/Services.java
Lines 308-309 (patched)
<https://reviews.apache.org/r/54705/#comment242894>

    log.fatal("Runtime Exception during Services Load. Check your list of '{0}' or '{1}'", CONF_SERVICE_CLASSES, CONF_SERVICE_EXT_CLASSES, rex);



core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java
Lines 86-87 (original), 71 (patched)
<https://reviews.apache.org/r/54705/#comment242895>

    You should retain the code of checking the resource name and throwing error here which is currently commented out instead of doing it in setOozieMode()



core/src/main/java/org/apache/oozie/servlet/JVMInfo.java
Line 123 (original), 123 (patched)
<https://reviews.apache.org/r/54705/#comment242896>

    Revert. The original spelling of Daemon is correct.



core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java
Line 386 (original), 386 (patched)
<https://reviews.apache.org/r/54705/#comment242897>

    Revert. threw exception is correct.


- Rohini Palaniswamy


On March 22, 2017, 8:34 p.m., Peter Cseh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54705/
> -----------------------------------------------------------
> 
> (Updated March 22, 2017, 8:34 p.m.)
> 
> 
> Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.
> 
> 
> Bugs: OOZIE-1779
>     https://issues.apache.org/jira/browse/OOZIE-1779
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Replacing the launcher MR job with a launcher AM
> 
> 
> Diffs
> -----
> 
>   bin/test-patch-20-tests 22f472c07e80fdb38c632329a5707998ab352172 
>   client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
>   client/src/main/java/org/apache/oozie/cli/ValidationUtil.java b1da2af070970add01031d6208a395520cd7055c 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 73708080e624e775f7d058cd3377ad69d8399264 
>   client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ca168e084998b5dd564536078a7a69efd7291828 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 8ddb1f873f0c69aadf5f4a5161d8b21f7392fa1e 
>   core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
>   core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
>   core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
>   core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
>   core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
>   core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 39daca7b3050c259a36d54710b4c1279f3acdeac 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
>   core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
>   core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
>   core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
>   core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
>   core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
>   core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
>   core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 5fd30f7858c5c4c70dcbdb6df86904518d92f7f3 
>   core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
>   core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 2ba1999ba7d5e892bd7e37d5fd8d1f926a84b48b 
>   core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
>   core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
>   core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
>   core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java 52e03d9839f01e8fcbc9c0fdee179467b9ac5c98 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 4cc284ae52ac50cce36c251536f045804e351f7a 
>   core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
>   core/src/main/java/org/apache/oozie/service/PurgeService.java c2c966b2cd056b77ae52164cb3f40955e98ae69a 
>   core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
>   core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java d263567eab9fc9b8763296c35e0a904499eabc11 
>   core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
>   core/src/main/java/org/apache/oozie/servlet/JVMInfo.java bf2b01db340a014d8831c5186188741f866e6559 
>   core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java d4a098949534f417938a65c27e395c9ac1f71ca7 
>   core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java fc3a6bc19d6a04e489c3f65c8248478a58540ede 
>   core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java d08521ca28d9ae4c8582c6ce3a8e6ca35a26f8dd 
>   core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
>   core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
>   core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
>   core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 83b455ebc2629fddccf44b61b1051f932e1d40a3 
>   core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
>   core/src/main/resources/oozie-default.xml 03f428a47e34f2d07ea8fff755f29d222d6aeb9a 
>   core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
>   core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
>   core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherHCatURIHandler.java 4288974b036b68b42d260d674aaaa0d345484123 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestScriptLanguageActionExecutor.java 0adc190d56ec163d454d64ae341b6a658ac14781 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 564db2a5150d949e2e9274984c1d67acce6b5447 
>   core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
>   core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
>   core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
>   core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
>   core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
>   core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java e41df8719fa361adca5406ddf1da919f9e78016d 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 6fd1a1aa5eab59c085551be8e52fa0d01c79c4da 
>   core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
>   core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
>   core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
>   core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java 1278efa0775c32c16cb38309ab8e92e08d4eb2ec 
>   core/src/test/java/org/apache/oozie/servlet/TestAdminServlet.java 8e3502c343744b7e70d2033643c9596c0c029778 
>   core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java a3f3b9c28c0f34381930a7457b5b622695948742 
>   core/src/test/java/org/apache/oozie/test/MiniHCatServer.java b2f7282a30bc8ed269d9c4208036421047647ef0 
>   core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
>   core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
>   core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 58f4beab9e707a3d8542253df596a566e65eff3c 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
>   core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/util/TestIOUtils.java 6fcdf5526927a3b8033c7a0c747b7480a5f199e7 
>   core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
>   distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 6a9e4b4bfd50954c399dd094c3fee82c491b6800 
>   docs/src/site/twiki/DG_HCatalogIntegration.twiki 39df67c3f5c5df0bb0521f613a66862c1b0a0ed8 
>   docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
>   docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
>   docs/src/site/twiki/WebServicesAPI.twiki a63a68cb27de05d03ad173caeb18f5b17e20867b 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 460f6e521cb8fe2e508ec7086081d49890efb037 
>   findbugs-filter.xml PRE-CREATION 
>   hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
>   hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
>   hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
>   hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
>   hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
>   hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
>   hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
>   hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
>   hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
>   hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
>   hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
>   pom.xml 98a2359a3ca476a0516a9746da2c1e91857143fd 
>   release-log.txt 4e7097b17a2ab1036135b27e9bdf3554fa2923ae 
>   sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
>   sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java fcaadefd215368565fb126ca92515792e1f1a09b 
>   sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java 7edb40d9d7cba6ef0603de25c9cee55f39b75f3b 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java faeff2a7f5f24f1680df5e529bbb48f49c750e09 
>   sharelib/hcatalog/src/test/java/org/apache/oozie/util/TestHCatURI.java ce690da986f3bbbab7f74e771b1fe9e32f047b27 
>   sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
>   sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bc8ab29c25405806f40232d32254ba2a7f5271e4 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
>   sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
>   sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
>   sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
>   sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 7ae48e168a310b4d55feda7cbce8b2d29c2ec505 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
>   sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
>   sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java ba393eee53e92d144159f83893d4baab00737a44 
>   sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 14ab65e7d00c426036ced3e80400c80deea0e282 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
>   sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
>   sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
>   sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java c24d95c88be0dbf6fe80241d887a70100a5e1b4a 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
>   sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
>   sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
>   sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb8f56e9b66de0eb2799e68bc56d0ee36d4ba317 
>   sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
>   sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
>   sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
>   sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
>   webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 
> 
> 
> Diff: https://reviews.apache.org/r/54705/diff/5/
> 
> 
> Testing
> -------
> 
> Junit tests are passing.
> Submitted jobs to a hadoop cluster successfully.
> 
> 
> Thanks,
> 
> Peter Cseh
> 
>

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated March 22, 2017, 8:34 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  bin/test-patch-20-tests 22f472c07e80fdb38c632329a5707998ab352172 
  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 38fb84e8a8918aee6ae57d19a29cdf181aba1955 
  client/src/main/java/org/apache/oozie/cli/ValidationUtil.java b1da2af070970add01031d6208a395520cd7055c 
  client/src/main/java/org/apache/oozie/client/OozieClient.java 73708080e624e775f7d058cd3377ad69d8399264 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  client/src/main/java/org/apache/oozie/client/rest/JsonTags.java ca168e084998b5dd564536078a7a69efd7291828 
  client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 8ddb1f873f0c69aadf5f4a5161d8b21f7392fa1e 
  core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 39daca7b3050c259a36d54710b4c1279f3acdeac 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java 92e149d5cac6509893c8507b776263a38312a53b 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 1a3197abb52f80b545dcc8634a8cac7a281a9eac 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 22e28746a9a39ae6c1e22326cc4f8f759d820b20 
  core/src/main/java/org/apache/oozie/action/ssh/SshActionExecutor.java 5fd30f7858c5c4c70dcbdb6df86904518d92f7f3 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 2ba1999ba7d5e892bd7e37d5fd8d1f926a84b48b 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java ace120d10e3546201aa4145df56abf7933fb24b8 
  core/src/main/java/org/apache/oozie/coord/input/logic/CoordInputLogicEvaluatorPhaseOne.java 52e03d9839f01e8fcbc9c0fdee179467b9ac5c98 
  core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 4cc284ae52ac50cce36c251536f045804e351f7a 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 6f0abf6e094ed04158652191f2b8bb3b0cd32e90 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 23a9d92bf56a625469ec247cfb9727972511b101 
  core/src/main/java/org/apache/oozie/service/PurgeService.java c2c966b2cd056b77ae52164cb3f40955e98ae69a 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/BaseAdminServlet.java d263567eab9fc9b8763296c35e0a904499eabc11 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/servlet/JVMInfo.java bf2b01db340a014d8831c5186188741f866e6559 
  core/src/main/java/org/apache/oozie/servlet/V0AdminServlet.java d4a098949534f417938a65c27e395c9ac1f71ca7 
  core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java fc3a6bc19d6a04e489c3f65c8248478a58540ede 
  core/src/main/java/org/apache/oozie/sla/listener/SLAEmailEventListener.java d08521ca28d9ae4c8582c6ce3a8e6ca35a26f8dd 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 45219a9238c09ff290e94121aaec46a2c7ea27ac 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/java/org/apache/oozie/util/db/SqlStatement.java 83b455ebc2629fddccf44b61b1051f932e1d40a3 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml 03f428a47e34f2d07ea8fff755f29d222d6aeb9a 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 09d723ac96ad8f06e3006a7ded3d9b7f8289bb45 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherHCatURIHandler.java 4288974b036b68b42d260d674aaaa0d345484123 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestScriptLanguageActionExecutor.java 0adc190d56ec163d454d64ae341b6a658ac14781 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 564db2a5150d949e2e9274984c1d67acce6b5447 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java e41df8719fa361adca5406ddf1da919f9e78016d 
  core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 6fd1a1aa5eab59c085551be8e52fa0d01c79c4da 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java 1278efa0775c32c16cb38309ab8e92e08d4eb2ec 
  core/src/test/java/org/apache/oozie/servlet/TestAdminServlet.java 8e3502c343744b7e70d2033643c9596c0c029778 
  core/src/test/java/org/apache/oozie/servlet/TestV1AdminServlet.java a3f3b9c28c0f34381930a7457b5b622695948742 
  core/src/test/java/org/apache/oozie/test/MiniHCatServer.java b2f7282a30bc8ed269d9c4208036421047647ef0 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 58f4beab9e707a3d8542253df596a566e65eff3c 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/util/TestIOUtils.java 6fcdf5526927a3b8033c7a0c747b7480a5f199e7 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_CommandLineTool.twiki 6a9e4b4bfd50954c399dd094c3fee82c491b6800 
  docs/src/site/twiki/DG_HCatalogIntegration.twiki 39df67c3f5c5df0bb0521f613a66862c1b0a0ed8 
  docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
  docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
  docs/src/site/twiki/WebServicesAPI.twiki a63a68cb27de05d03ad173caeb18f5b17e20867b 
  docs/src/site/twiki/WorkflowFunctionalSpec.twiki 460f6e521cb8fe2e508ec7086081d49890efb037 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml 98a2359a3ca476a0516a9746da2c1e91857143fd 
  release-log.txt 4e7097b17a2ab1036135b27e9bdf3554fa2923ae 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java fcaadefd215368565fb126ca92515792e1f1a09b 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hcatalog/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java 7edb40d9d7cba6ef0603de25c9cee55f39b75f3b 
  sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java faeff2a7f5f24f1680df5e529bbb48f49c750e09 
  sharelib/hcatalog/src/test/java/org/apache/oozie/util/TestHCatURI.java ce690da986f3bbbab7f74e771b1fe9e32f047b27 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bc8ab29c25405806f40232d32254ba2a7f5271e4 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 2ba0da7b1bc342d7d644a914763621940bd34532 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java a3a07bdedfe311fbfb8ec2236d8a7d6db55e82d8 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 7ae48e168a310b4d55feda7cbce8b2d29c2ec505 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java d376057f010dc359b44c6d4fe69ec0493261f037 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java ba393eee53e92d144159f83893d4baab00737a44 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java 14ab65e7d00c426036ced3e80400c80deea0e282 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java c24d95c88be0dbf6fe80241d887a70100a5e1b4a 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb8f56e9b66de0eb2799e68bc56d0ee36d4ba317 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 3dfd606afb9a54791796cb3742998f35cec2f0ce 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java 5f9f38eb2c30edd234b2a523e5ca6c5c2f2ae832 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 


Diff: https://reviews.apache.org/r/54705/diff/5/

Changes: https://reviews.apache.org/r/54705/diff/4-5/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated Jan. 23, 2017, 11:43 a.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6e30d7eb667fe74440281fab3ca06c9f4df7bcf7 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProviderFactory.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentials.java d7689a924059e553abaeed374eed4072933292cb 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java bd2f1f768c5b04bef4a33613baede9202144c747 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/FSUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
  core/src/main/java/org/apache/oozie/util/JobUtils.java 63f88acd6fb91668baa6cd0a80b2d45435e7b213 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml ad103864d7631e9d3678a828bf297ca1c80f09ca 
  core/src/main/resources/oozie-log4j.properties c86b30143f5f58135493d9e6bdba29f402a0b1fd 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/CredentialForTest.java 06298916e797890731134370412f51f3456aa70c 
  core/src/test/java/org/apache/oozie/action/hadoop/InsertTestToken.java 9da8fbe18923899489c71f40c99a01fe6522e143 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
  core/src/test/java/org/apache/oozie/action/hadoop/TestHCatCredentials.java e6d43ca820330a5f00cc5d82e8cfb88f3a814fb4 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 1c4b4296682865f96857c1038e0b26c4877b6dc8 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java 8ec38e498f31bb69923125e34cda5abfc8e72aa9 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java 42ffdbeb41e5829cdf814cf13ada331973fc33bd 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
  docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml 15f4195074c935bfaf941d2d0dc502105c4294b1 
  release-log.txt ca9cd43260ffd3b5468631ff3c6edf2d90457e1c 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml f3ea0716a6719c44916bb50255f5f31a873fb514 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifierFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 8edebac1147d6e918c7bec921ef9ee94fb16e8e4 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/SequenceFileWriterFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 99148d73e19a42531275789e4079e11a916bbaeb 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 7de2e453e150d0cede961f9be622c4b55396d5d5 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 

Diff: https://reviews.apache.org/r/54705/diff/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh

Re: Review Request 54705: OOZIE-1770: Create Oozie Application Master for YARN

Posted by Peter Cseh <ge...@cloudera.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54705/
-----------------------------------------------------------

(Updated Jan. 6, 2017, 1:59 p.m.)


Review request for oozie, Purshotam Shah, Robert Kanter, and Rohini Palaniswamy.


Repository: oozie-git


Description
-------

Replacing the launcher MR job with a launcher AM


Diffs (updated)
-----

  client/pom.xml f99366c90887a5a45f3942291c90e1cdbfd84b3f 
  client/src/main/java/org/apache/oozie/cli/OozieCLI.java e67fae9e2c7b32c1f209c24b4aba98daba96c06e 
  client/src/main/java/org/apache/oozie/client/XOozieClient.java 8538ec7c742cc013f7c18eaff3ecf61c2835cc8a 
  core/pom.xml 1dd104581ccab7976d788b674d2200e48a70c27f 
  core/src/main/java/org/apache/oozie/WorkflowJobBean.java 55d79a5d5e5e4efab9eb832e9341a16bcf8959cc 
  core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1d6456b236bf9e27c0f55d9f9e3b4bd94517c797 
  core/src/main/java/org/apache/oozie/action/hadoop/Credentials.java eadb47b07d157f45f086517ab41c4b9826f003f0 
  core/src/main/java/org/apache/oozie/action/hadoop/CredentialsProvider.java 6fe22fbd2fca5d873d58ece60f20d0f131371475 
  core/src/main/java/org/apache/oozie/action/hadoop/DistcpActionExecutor.java 78cd257a836707c1efeac13566d6285d42970545 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 27654741101d2056b25626fbbe345183eafd612d 
  core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 801bfe6eb212b549888f0e9fb7df7b65bcf59393 
  core/src/main/java/org/apache/oozie/action/hadoop/HCatCredentialHelper.java 36ad1df727e869b8051481ad969771e91bc6ae07 
  core/src/main/java/org/apache/oozie/action/hadoop/HadoopELFunctions.java ad2a71d641caa71efc098ad00bfbab3cec4d9506 
  core/src/main/java/org/apache/oozie/action/hadoop/HbaseCredentials.java 307f565786ad77f5a35b5790bd5b052b04c6eac8 
  core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 2aed936af75da38c433be5d202bdc01c2f612c76 
  core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 8e2453ef4f78f4a5f1f95b1ca2793f5d7e07ad81 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 4beeb965a461e246479c3e8ad88f92ecafca13c1 
  core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 9609fdc2f3440fb176b659ef16c2a23451724cf9 
  core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java 55c937249ac35e14bd63dd0c1fa962ea636afa3a 
  core/src/main/java/org/apache/oozie/action/hadoop/OozieJobInfo.java 581d3b3219eff882f961c8e495611e873c6a5a52 
  core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 8b2dc16db80ad602911f1f64e60cfad41b198922 
  core/src/main/java/org/apache/oozie/action/hadoop/ScriptLanguageActionExecutor.java f2541265ca71fb7f3b053c59775ef5e23d0d8f94 
  core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java b9ffa7ab7ec87db8a76a3653c74974d296c8cd17 
  core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java 8e8d7d308b294efd91e3057e0d88ee06643ab2d4 
  core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 6cee32aa8f5b8b36c9c7a98bb738f50b18b64624 
  core/src/main/java/org/apache/oozie/client/rest/BulkResponseImpl.java fb021bd022d199f93fc5674843514abf45ae12a5 
  core/src/main/java/org/apache/oozie/command/XCommand.java bdf13f65ea25e8e0f5125e2798568a2c03fffe81 
  core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java d2a2742ae418070a1cd0a889f14194224bc944a6 
  core/src/main/java/org/apache/oozie/command/wf/SubmitMRXCommand.java cc61d3de813c75c7f1cf4d405fc043253e3ca4ea 
  core/src/main/java/org/apache/oozie/command/wf/SubmitScriptLanguageXCommand.java 9d413059e99bc676feb63e02a2f704048eb4a355 
  core/src/main/java/org/apache/oozie/command/wf/SubmitSqoopXCommand.java 51b739e21c802e09f4110b740138f4ff5c341fe4 
  core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java 0da60ec70373399414a637c22753da7e2a7e6618 
  core/src/main/java/org/apache/oozie/dependency/hcat/EhcacheHCatDependencyCache.java 3bc467535202e13387b1d29ac678573f4154c522 
  core/src/main/java/org/apache/oozie/service/CallableQueueService.java a86a8d0889138cd1564ca9253e5b319ea52e55cc 
  core/src/main/java/org/apache/oozie/service/EventHandlerService.java 22c6fb0965f45c5cf2322c251d4a8aa77d263458 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 5377127736cf189bcf4aa3d62bbbeab6b3a29d52 
  core/src/main/java/org/apache/oozie/service/Services.java 829d5f5cd90c86e6cba9854dc0b4b7df570b8900 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java b79bd37bb2b44348f220128334497d788dd922d4 
  core/src/main/java/org/apache/oozie/servlet/CallbackServlet.java 612302173e0f485345558b0c96a8a75d2f43d238 
  core/src/main/java/org/apache/oozie/store/OozieSchema.java 23dd1e508b6c721a10c9fe828baa5cd0d95064b4 
  core/src/main/java/org/apache/oozie/util/ClasspathUtils.java PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/IOUtils.java a6c2fdae049d1df936a0345f5ffc6609ea40a5c0 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 55e00d4dfc1009b6724918a328e254228a2f6240 
  core/src/main/java/org/apache/oozie/util/JobUtils.java a7a53b355d194b55959ede20c85d38e02294853d 
  core/src/main/resources/META-INF/persistence.xml edda2d5e22856b0250a53bfde7c73fd96f85c6e9 
  core/src/main/resources/oozie-default.xml 2d7650acee73af713e2e3ea70975e74d1866cfd4 
  core/src/test/java/org/apache/oozie/QueryServlet.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/BlockingMapper.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/LauncherMainTester.java 4baed6e42beccd2fdb55ab1e6fb150c0d8821a9c 
  core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 8f08ddd9fa077fc9526814fd25e7477850c51737 
  core/src/test/java/org/apache/oozie/action/hadoop/TestDistCpActionExecutor.java e8a140f330062c5ce8324112aae83b3d0ae1f772 
  core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java b27ede7fae8297d982a5dee612fe714cf61f673e 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 8965cdff8627cb2ae0d2b167d7e48eab9b7825c8 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java aa938d01b7273d6db273908e5480b71613598135 
  core/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAMCallbackNotifier.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java 4cda6151ca5bb0d9007f11bfe712de74b1a0da26 
  core/src/test/java/org/apache/oozie/action/hadoop/TestOozieJobInfo.java 5bc7d00ff6c6c2d29806633761cba4a940ce512a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java df9e939cb236e8b65e7b3aa9ab683c33f5fa02b7 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java 1531ed994740d8d509c5429f761f4aa2e83c3227 
  core/src/test/java/org/apache/oozie/action/hadoop/TestShellMain.java e757e542cb6256906f43a36f7d30c8e8b8aa04e2 
  core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ce95ff34030c97b56ed9257ebe027833493a9fa1 
  core/src/test/java/org/apache/oozie/client/TestWorkflowXClient.java 51ae9e87d060c9e276ca74b45b24a5f5ae65e722 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java 8034bbe9e6caa1672a0018daf48d649c5e5d2410 
  core/src/test/java/org/apache/oozie/command/wf/HangServlet.java 3344cf97762c77f5660d94341e72df4243f76eb2 
  core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java 5898d1af79631a1b7e5021d9ca05685eab7225a5 
  core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java ea90c087fc145650dfb77ac35a03ba1203ffb0f2 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitHiveXCommand.java 43edf5efad0932aa5089ac8a6f6cfcf9f120d966 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitMRXCommand.java 5bc5747786774766e7156cb241d5444fefe6d7cb 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitPigXCommand.java 5a1de25c85129e0b9567d0fd045133bffc700c5d 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitSqoopXCommand.java 49b5028bcaff62e752a9582f1ddb5fb2ec56433c 
  core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java 72f01148ea457085f293689eda30509a1ca694d6 
  core/src/test/java/org/apache/oozie/service/TestConfigurationService.java f18a8c073d280b9570f815be30d72c5b349ca7f5 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java bbca4794b693f865290adaa915d60019fc1218dd 
  core/src/test/java/org/apache/oozie/service/TestRecoveryService.java 8fd0c2d5080b416470a8d886f69c7e3de8fa3ffc 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java 2d31f9a86c25c15ee50bd420fb7300ee81a20835 
  core/src/test/java/org/apache/oozie/test/XDataTestCase.java ea778bd19f272876f1f8d38614ecba47649526ef 
  core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1d399e4ce921af8cb2cb2e6375d425dd5f7780f7 
  core/src/test/java/org/apache/oozie/test/XTestCase.java e1d9068f3ae0ee2ee01eb1e8f6e7164fd3c865b8 
  core/src/test/java/org/apache/oozie/util/TestClasspathUtils.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java a3c7b142d08101e2bb03737871db939133bd3cd2 
  distro/src/main/bin/addtowar.sh 399040936bde3d091d42557f6a64a0aee2b139e2 
  docs/src/site/twiki/DG_QuickStart.twiki 2b9dc3bc753e7bf712117a92b76cdf29d11446cc 
  docs/src/site/twiki/ENG_Building.twiki dda829b7eb9c1683f5999e436dafaa531be0920a 
  findbugs-filter.xml PRE-CREATION 
  hadooplibs/hadoop-auth-1/pom.xml 85e2298c25fc13bfc29ef8e0226d4eda2a95a073 
  hadooplibs/hadoop-auth-2/pom.xml 7202b2e036bb49964435d089d9f9ff1e9237a7f7 
  hadooplibs/hadoop-distcp-1/pom.xml bbc536b5901540e8f0835f75e185b7bb40e02e3c 
  hadooplibs/hadoop-distcp-2/pom.xml 2c21b12068ecc9782e2d12c4376ee16bfba16af0 
  hadooplibs/hadoop-distcp-3/pom.xml 39cf9f26c71dc5a93bceb1f750de46002d2c7f80 
  hadooplibs/hadoop-utils-1/pom.xml a53e9bca3024d924b7e9e79f2199d30b18506132 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java c0a237777f44e4d0a9061fc468af98cec2b92aa9 
  hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java 799dffbd7dc04654e72fa8de484143732f725fa1 
  hadooplibs/hadoop-utils-2/pom.xml 7c4a0a10a56a79ce0513bfa2f790ffbbc5eb960a 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java a0b7d62c9432614f33baa3631660e6cb7918cd94 
  hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/hadoop-utils-3/pom.xml 2f36f8bea921ad5ff74663728b377352daf0b2ca 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java 0cf2e909283712f6e93653241f62ba601ae85706 
  hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java c21884c7f94dab78be4a6dea20b698476479dc17 
  hadooplibs/pom.xml 06801cc6d8dcfe1fa21367c3bb26c3f8dcaf29ec 
  pom.xml 0cb5c7888d0cc534f7245b1d183228c3feacc377 
  sharelib/distcp/pom.xml 1f3c0cc6da4e996045fcce39e0e6477df2589f63 
  sharelib/distcp/src/main/java/org/apache/oozie/action/hadoop/DistcpMain.java 2b84f92ce0f7028e4a7f8d1b520b9dfcee6a83bd 
  sharelib/hcatalog/pom.xml 6eb88ef85e2f8c9398df64908e7777f251c1dbfa 
  sharelib/hive/pom.xml 13312192b634be802b3bc52726770d5c7b4057e9 
  sharelib/hive/src/main/java/org/apache/oozie/action/hadoop/HiveMain.java bbcaefa2b363fcf8dd7ce481fc7db71fda69bdad 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java 12e1e91cf83a109d77b6a20b022dee851447525d 
  sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveMain.java 302631182a3e70f7188272d1260c2a393b283d31 
  sharelib/hive2/pom.xml e81bfbe0a96a490b349a2660a0b00381d56785e1 
  sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 54f703926e360e1942f4da41d712c29c02a3332a 
  sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java 4818bb6399a6c9d315af8fee7667db3691704840 
  sharelib/oozie/pom.xml 4a89934f73012a89f9e56b791352bdeec4bafded 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMCallBackHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/AMRMClientAsyncFactory.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ActionUtils.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ErrorHolder.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/HdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/JavaMain.java 30d68e2a55dd2e6b695c27ec18e5e47a48eb9322 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAM.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherAMCallbackNotifier.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 785ca5ea2fdaef3b414bd96c6872035b7a10c680 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java 7271486646edabab19066a0a6986398f12691516 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LocalFsOperations.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java 23447cf87427af2de8623d3715696d38c429f962 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java 21ae456b768fb1cb5334a9e23e6c6082240f8dff 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsHandler.java PRE-CREATION 
  sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java f109318d949c68d4738e65de5a63de7c950a7888 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/LauncherAMTestMainClass.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestHdfsOperations.java PRE-CREATION 
  sharelib/oozie/src/test/java/org/apache/oozie/action/hadoop/TestLauncherAM.java PRE-CREATION 
  sharelib/pig/pom.xml 233873c87091adef31bd0f40ecdb18b757c8d0f5 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMain.java 87c4cc4cca2d2d299b814a82c108d87574bc6946 
  sharelib/pig/src/main/java/org/apache/oozie/action/hadoop/PigMainWithOldAPI.java b84e2b5ee37df078e48959584eb41ec1a90200ca 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java 25092ce55d8407384837b07d6d58b0218d1de7ab 
  sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigMainWithOldAPI.java e52e6fddd474da1bd3c4b03376b7b060df2ab3b5 
  sharelib/spark/pom.xml abef24fe7f8a95f35347c2515385f9f582864559 
  sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java 0da74d4ac55710a541eb28ee0ac8ab36bfd4fa00 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestPyspark.java 458baaa8b6f5bc4ec5dd8395d428a7e4215fb27c 
  sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java 8c77be0bdd7bf8cae07c4d4acd803df5730826e7 
  sharelib/sqoop/pom.xml d5afa370b19ac3895e99c4bf594d3d32a4711d5b 
  sharelib/sqoop/src/main/java/org/apache/oozie/action/hadoop/SqoopMain.java cb65eb82077b6ab4a08f25866386435fcb0b107c 
  sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java 166d939b6016aa0f8a1739adc2ea17aed0b7ee7b 
  sharelib/streaming/pom.xml 4f732724846411f13bd0204cdc2bef198b44f0c6 
  sharelib/streaming/src/main/java/org/apache/oozie/action/hadoop/StreamingMain.java 991bf7e849171cb7847fa6d64781ceb0782dae7c 
  sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java c7860bed2ee46d88a9ce7768b5d2c4d6cabc3094 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 9aa4cb6bf47a632b87b187882a8e105c40f38daf 
  webapp/pom.xml 2c4dfc263f90c2167ff76aac1297ceb19cd32ae9 

Diff: https://reviews.apache.org/r/54705/diff/


Testing
-------

Junit tests are passing.
Submitted jobs to a hadoop cluster successfully.


Thanks,

Peter Cseh