You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by vramachan <gi...@git.apache.org> on 2016/06/30 00:36:28 UTC
[GitHub] falcon pull request #200: Falcon 2051-Falcon post-processing services are no...
GitHub user vramachan opened a pull request:
https://github.com/apache/falcon/pull/200
Falcon 2051-Falcon post-processing services are not getting invoked
Basically, reverted FALCON-1926 and ran the tests to make sure REPL metrics are populated.
After 0.10 release, we will figure out a mechanism so that Post Processing does not need to send JMS message.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vramachan/falcon FALCON-2051.PostProcessingNotInvoked.0.10
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/falcon/pull/200.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #200
----
commit 16a07cb3150e929d6334a7e7ccee809cec255c5a
Author: yzheng-hortonworks <yz...@hortonworks.com>
Date: 2016-05-27T21:08:33Z
FALCON-1983 Upgrade jackson core and databind versions to fix dependency incompatibility with higher-version Hive
Tested it works and generates consistent jackson jars with Hive 1.2.1x.
Author: yzheng-hortonworks <yz...@hortonworks.com>
Reviewers: "Balu Vellanki <ba...@apache.org>, Venkat Ranganathan <ve...@hortonworks.com>"
Closes #163 from yzheng-hortonworks/FALCON-1983
(cherry picked from commit 2923c04c1d6434cae27f8a4e6b35934d11f1f4ab)
Signed-off-by: Sowmya Ramesh <sr...@hortonworks.com>
commit 33a37a6380f16830265d23c47d444cd3b1d7ddbe
Author: Peeyush <pe...@apache.org>
Date: 2016-05-30T05:16:02Z
FALCON-1980 Change input and output argument order for Spark process workflow
Details to perform this change has been mentioned in the FALCON-1980.
Author: peeyush b <pb...@hortonworks.com>
Reviewers: Pavan <pa...@gmail.com>
Closes #160 from peeyushb/FALCON-1980 and squashes the following commits:
23d4e24 [peeyush b] Added comments for input and output argument
3d86f58 [peeyush b] FALCON-1980 : Change input and output argument order for Spark process workflow
(cherry picked from commit 400ef329d1816b408e759990b880c9dd1970dae2)
Signed-off-by: peeyush b <pb...@hortonworks.com>
commit da3e37b25d8d49fca2d54c4fe61425824101ef0e
Author: peeyush b <pb...@hortonworks.com>
Date: 2016-05-30T09:14:42Z
FALCON-1893 : Add documentation and examples for spark workflow engine
Please review the pull request which contains documentation with examples how to use the Spark application with Falcon.
Author: peeyush b <pb...@hortonworks.com>
Reviewers: "Pavan Kumar Kolamuri <pa...@gmail.com>, Venkat Ranganathan <vr...@hortonworks.com>"
Closes #150 from peeyushb/FALCON-1893
(cherry picked from commit ff2f9e27fb96e01a1c03598ba936eff77fd3b630)
Signed-off-by: pavankumar526 <pa...@gmail.com>
commit ca90089e3a7752799429005449bd0f6d2e04a0c1
Author: Peeyush <pe...@apache.org>
Date: 2016-06-02T14:41:39Z
FALCON-2007 Hive DR Replication failing with - Can not create a Path from a null string
Following is the pull request that will check the existence of the file oozie.action.conf.xml. If file is not there, then it will not get into the code block that adds it to the configuration object, thus to avoid the exception.
Author: peeyush b <pb...@hortonworks.com>
Reviewers: Venkat <n....@live.com>
Closes #167 from peeyushb/FALCON-2007
(cherry picked from commit 629bcfbdbd6833cf590a6f622ccde7813e5556dd)
Signed-off-by: peeyush b <pb...@hortonworks.com>
commit a8d6dcdb3e672fecebf2abab19dca09a3ef864dd
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Date: 2016-06-03T00:07:51Z
FALCON-1981 Remove runtime jar dependencies from pom files
+ Remove jar dependency from pom files that are not required during runtime, but provided by Hadoop
+ End to End test to make sure few flows work
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Reviewers: "Ying Zheng <yz...@hortonworks.com>, Venkat Ranganathan <ve...@hortonworks.com>"
Closes #161 from vramachan/FALCON-1982.PomCleanUp
(cherry picked from commit 85ed40eba15039c9ed15ab9785f74a991bd50f41)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit 5b06e5f6bb3483aec8aa1c7dac749a1803b63897
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Date: 2016-06-03T14:50:13Z
FALCON-1859 Feed export instances are not added to Graph DB
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Reviewers: "Balu Vellanki <ba...@apache.org>"
Closes #165 from vramachan/FALCON-1859.Export.GraphDB
(cherry picked from commit 16d2b39b24f30b3562874585fcdc58ecf4460ad0)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit 0e16f5c67c896658dfdc01d2142099e2ba0a1de6
Author: yzheng-hortonworks <yz...@hortonworks.com>
Date: 2016-06-03T14:53:16Z
FALCON-1984 Provide proper hint and documentation if required titan storage backend is not configured in startup.properties
Added validation of required configuration properties for graph db storage backend.
Added error hints on the missing property in Falcon server log.
Reorganized the order of the configuration properties in startup.properties so relevant properties for each backend are grouped in consecutive lines.
Author: yzheng-hortonworks <yz...@hortonworks.com>
Reviewers: "Balu Vellanki <ba...@apache.org>, Venkat Ranganathan <ve...@hortonworks.com>"
Closes #166 from yzheng-hortonworks/FALCON-1984
(cherry picked from commit e84fca7ff2ce235a7d804b3c4806091a2a2c3037)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit f53f99a7272d4241f221f759eac60db358671dbd
Author: yzheng-hortonworks <yz...@hortonworks.com>
Date: 2016-06-06T18:56:59Z
FALCON-1965 Update ActiveMQ version to 5.13.3 to avoid Falcon start error after rolling upgrade
Upgraded ActiveMQ version to the latest 5.13.3. Tested Falcon server upgrade with ActiveMQ history works with 0.6, 0.7 and trunk.
Split the fix for two test failures due to upgrade to https://github.com/apache/falcon/pull/169, per vrangan 's suggestion.
Author: yzheng-hortonworks <yz...@hortonworks.com>
Reviewers: "Balu Vellanki <ba...@apache.org>, Venkat Ranganathan <ve...@hortonworks.com>, Pallavi Rao <pa...@apache.org>"
Closes #168 from yzheng-hortonworks/FALCON-1965
(cherry picked from commit f8707fdc6f512fdfc386b010e4dee0812f45cbd7)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit 0b922edbfa01c2f37a158fa63883ba7931b37bb0
Author: yzheng-hortonworks <yz...@hortonworks.com>
Date: 2016-06-06T18:58:56Z
FALCON-2010 Fix UT errors due to ActiveMQ upgrade
Fixed two test failures due to the upgrade:
1. JMSMessageConsumerTest
After upgrade, some messages are consumed faster and so the initial expectation on no message being dequeued right after the message sending doesn't hold. Also confirmed with bvellanki that this kind of tests are flaky and have caused intermittent test errors before. Therefore, removed getTotalDequeueCount assertion. In addition, the expected message consumptions are tested in existing code "Mockito.verify(jobEndService,...".
2. FalconPostProcessingTest
After upgrade, Falcon will run into an error if it tries to delete messages when ActiveMQ service is stopped. Therefore, need to call start before deleting the message during teardown.
Author: yzheng-hortonworks <yz...@hortonworks.com>
Reviewers: "Balu Vellanki <ba...@apache.org>, Venkat Ranganathan <ve...@hortonworks.com>, Pallavi Rao <pa...@apache.org>"
Closes #169 from yzheng-hortonworks/FALCON-2010
(cherry picked from commit fb4d0b4955429e6cef413ab073814fc800361bd4)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit 1689a098580349e37d4ab478c06b03dee4af91cb
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Date: 2016-06-06T22:21:15Z
FALCON-1982 Configuration steps for HBase standalone mode
Step to run HBase in standalone mode to support titan backend.
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Reviewers: "Balu Vellanki <ba...@apache.org>, Venkat Ranganathan <ve...@hortonworks.com>"
Closes #164 from vramachan/FALCON-1982.HBaseStandAlone
(cherry picked from commit ea2122b6c699bdb6429e6dda6d91f3d6bdd697d7)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit 377452395f328bb2ca2e4ec5ac22f6b669944279
Author: bvellanki <bv...@hortonworks.com>
Date: 2016-06-06T23:38:35Z
FALCON-1996 Upgrade falcon POM for 0.10 release
Author: bvellanki <bv...@hortonworks.com>
Reviewers: "Sowmya Ramesh <so...@apache.org>"
Closes #172 from bvellanki/0.10
commit 7d7b1fb16492f8d3d2cf8bd4b5e1642622738345
Author: Sowmya Ramesh <sr...@hortonworks.com>
Date: 2016-06-07T21:06:52Z
FALCON-2017 Fix HiveDR extension issues
Author: Sowmya Ramesh <sr...@hortonworks.com>
Reviewers: "Venkat Ranganathan <ve...@hortonworks.com>, Balu Vellanki <ba...@apache.org>"
Closes #175 from sowmyaramesh/FALCON-2017
(cherry picked from commit ee644dd2a6639085b469f8fae96580671f4d0fe9)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit 248c92935915f3c1f1e8bcaa942c15760c3ffdf5
Author: Praveen Adlakha <ad...@gmail.com>
Date: 2016-06-08T18:10:15Z
FALCON-2018 WorkflowJobNotification sends incorrect message for kille\u2026
\u2026d instances
Author: Praveen Adlakha <ad...@gmail.com>
Reviewers: "Balu Vellanki <ba...@apache.org>"
Closes #179 from PraveenAdlakha/2018
(cherry picked from commit a4e18a4d428569c37e65e281c3c36bf3ed737eed)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit 49df8da45db6948aa374838b530095d68a6c3324
Author: Pallavi Rao <pa...@inmobi.com>
Date: 2016-06-09T03:23:07Z
FALCON-2016 maven assembly:single fails on MacOS
Author: Pallavi Rao <pa...@inmobi.com>
Reviewers: @sandeepSamudrala, @bvellanki
Closes #174 from pallavi-rao/2016 and squashes the following commits:
35e1506 [Pallavi Rao] Merge remote-tracking branch 'upstream/master' into 2016
d76f24b [Pallavi Rao] FALCON-2016 maven assembly:single fails on MacOS
(cherry picked from commit b2f085c7cb21b6c5208eaa3aaa8fac331e2cd05a)
Signed-off-by: Pallavi Rao <pa...@inmobi.com>
commit 9dce4e7f98d9603733bab4fe1c2b8ff9685939b0
Author: Praveen Adlakha <ad...@gmail.com>
Date: 2016-06-10T07:05:39Z
FALCON-2020 Changes in Graphite Notification Plugin
Author: Praveen Adlakha <ad...@gmail.com>
Reviewers: @bvellanki, @pallavi-rao
Closes #180 from PraveenAdlakha/2020 and squashes the following commits:
ecfae9a [Praveen Adlakha] comments addressed
5f3b962 [Praveen Adlakha] FALCON-2020 Changes in Graphite Notification Plugin
(cherry picked from commit 54430ed279909c3e7d488e4ef98d05f56c4ecf45)
Signed-off-by: Pallavi Rao <pa...@inmobi.com>
commit fc9070c8123ee048ccd7570e17f98f3387b0e41d
Author: Praveen Adlakha <ad...@gmail.com>
Date: 2016-06-10T07:08:21Z
FALCON-2008 Add documentation for Graphite Notification Plugin
Author: Praveen Adlakha <ad...@gmail.com>
Reviewers: @pallavi-rao
Closes #181 from PraveenAdlakha/2008
(cherry picked from commit 65c804cbac7884f28d60d18116625938a8811456)
Signed-off-by: Pallavi Rao <pa...@inmobi.com>
commit af7407f4c05a43baa15b98ab6b1ecc1b1f1d51c2
Author: Praveen Adlakha <ad...@gmail.com>
Date: 2016-06-08T18:18:35Z
FALCON-2009 Falcon Aspect weaving issue
Author: Praveen Adlakha <ad...@gmail.com>
Reviewers: "Balu <ba...@apache.org>"
Closes #178 from PraveenAdlakha/2009
(cherry picked from commit 52425d19c98c160cc1b6fff4065dc10fb9b20a6a)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit df6b53c83520c9cd015dfbd0d752f25d18cd9c3f
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Date: 2016-06-13T22:19:05Z
FALCON-2023 Feed eviction fails when feed locations stats and meta \u2026
\u2026does not have time pattern
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Reviewers: " sandeepSamudrala <sa...@gmail.com>, Ying Zheng <yz...@hortonworks.com>, Balu Vellanki <ba...@apache.org>"
Closes #182 from vramachan/FALCON-2023
commit 11ca2a0eb51821a03bd31b192c89e36530fb0961
Author: bvellanki <bv...@hortonworks.com>
Date: 2016-06-14T15:57:51Z
FALCON-2025 Periodic revalidation of kerberos credentials should be done on loginUser
Author: bvellanki <bv...@hortonworks.com>
Reviewers: "sandeepSamudrala <sa...@gmail.com>, Praveen Adlakha <ad...@gmail.com>, Sowmya Ramesh <so...@apache.org>, Venkat Ranganathan <ve...@hortonworks.com>"
Closes #183 from bvellanki/FALCON-2025
(cherry picked from commit 00a07d561fdee4aba5be24cfe841c438b17a0e69)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit b2e1296bdabdc58c104a2858aef483719cf5bc42
Author: Sowmya Ramesh <so...@apache.org>
Date: 2016-06-16T00:25:51Z
FALCON-2032 Update the extension documentation to add ExtensionService before ConfigurationStore in startup properties
Author: Sowmya Ramesh <sr...@hortonworks.com>
Reviewers: Balu Vellanki <ba...@apache.org>
Closes #186 from sowmyaramesh/FALCON-2032
(cherry picked from commit df10d5ac333813edc9a6025b8f3098bd8d3e79db)
Signed-off-by: Sowmya Ramesh <sr...@hortonworks.com>
commit 196a76bfd932b7e4997a443ec76d8f29ddc0b186
Author: Peeyush <pe...@apache.org>
Date: 2016-06-16T07:03:59Z
FALCON-2031 Hcat Retention test cases are failing NoClassDefFoundError
Please review the following pull request, in which we have added the "oozie.action.sharelib.for.java" = "hcatalog,hive" in eviction-action.xml to avoid the failure of HCat retention test cases with "NoClassDefFoundError" when using latest Hive.
Author: peeyush b <pb...@hortonworks.com>
Reviewers: Pallavi Rao <pa...@inmobi.com>, Praveen Adlakha <ad...@gmail.com>, sandeep <sa...@gmail.com>, Balu <ba...@apache.org>
Closes #185 from peeyushb/FALCON-2031
(cherry picked from commit e7cdda742f90744519f522fcf499f79debfeb945)
Signed-off-by: peeyush b <pb...@hortonworks.com>
commit bdda78ca83e7778acda2f80f65d0e635abe3d043
Author: yzheng-hortonworks <yz...@hortonworks.com>
Date: 2016-06-17T18:05:37Z
FALCON-2027 Enhance documentation on data replication from HDP to Azure
Also fixed a typo in FalconDocumentation.twiki.
Author: yzheng-hortonworks <yz...@hortonworks.com>
Reviewers: "Balu Vellanki <balu@apache.org?"
Closes #187 from yzheng-hortonworks/FALCON-2027
(cherry picked from commit 037e6821b6eb7dbad1c0b1a3508aa6715e77e454)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit b67dd535ea6c57c4008fd3165a339a6bc1581839
Author: sandeep <sa...@gmail.com>
Date: 2016-06-21T04:51:14Z
FALCON-2034 Make numThreads and timeOut configurable In ConfigurationStore init
Author: sandeep <sa...@gmail.com>
Reviewers: @pallavi-rao, @peeyushb
Closes #192 from sandeepSamudrala/FALCON-2034 and squashes the following commits:
78b98d5 [sandeep] FALCON-2034. Make numThreads and timeOut configurable In ConfigurationStore init
9d00722 [sandeep] FALCON-2034. Make numThreads and timeOut configurable In ConfigurationStore init
(cherry picked from commit d3ebf0b3d826c32e068c0c786d51dd4d39a7f08f)
Signed-off-by: Pallavi Rao <pa...@inmobi.com>
commit 81ae46de1c63da80498854bb1eb4a5d10ec806db
Author: yzheng-hortonworks <yz...@hortonworks.com>
Date: 2016-06-21T04:54:04Z
FALCON-2036 Update twiki on entity list operation with up-to-date REST API path
1. Mark entity type as optional.
2. Reorganize the order of examples.
Author: yzheng-hortonworks <yz...@hortonworks.com>
Reviewers: @pallavi-rao,@bvellanki
Closes #190 from yzheng-hortonworks/FALCON-2036
(cherry picked from commit dca46a916fb5dba84c324d2a456dd1248a6176b5)
Signed-off-by: Pallavi Rao <pa...@inmobi.com>
commit 820052464aed7df1213295ab19e58584a4a3dd5e
Author: Pallavi Rao <pa...@inmobi.com>
Date: 2016-06-21T04:57:56Z
FALCON-2038 When all Optional input instances are missing, partition should not be suffixed.
Updated code and UT.
Author: Pallavi Rao <pa...@inmobi.com>
Reviewers: @PraveenAdlakha, @sandeepSamudrala, @peeyushb
Closes #191 from pallavi-rao/2038
(cherry picked from commit 3abefccf5d311e885051cf7abc6592c19af1b67d)
Signed-off-by: Pallavi Rao <pa...@inmobi.com>
commit 641d57afa68b0d8ae091595f7eca560ec1d13f1b
Author: yzheng-hortonworks <yz...@hortonworks.com>
Date: 2016-06-21T06:24:09Z
FALCON-2035 Entity list operation without type parameter doesn't work when authorization is enabled
Test entity list operation without type parameter works when authorization is enabled.
Author: yzheng-hortonworks <yz...@hortonworks.com>
Reviewers: @pallavi-rao, @bvellanki
Closes #189 from yzheng-hortonworks/FALCON-2035 and squashes the following commits:
0cbd7a4 [yzheng-hortonworks] review by pallavi
4f7c4a4 [yzheng-hortonworks] review by balu
7aa034b [yzheng-hortonworks] FALCON-2035 Entity list operation without type parameter doesn't work when authorization is enabled
(cherry picked from commit aaf33261f32ba87d95c268e09a593f9ebae1f46b)
Signed-off-by: Pallavi Rao <pa...@inmobi.com>
commit b4b32daa181fa4e33b57de52f89f2934c3501545
Author: Murali Ramasami <murali dot msse at gmail dot com>
Date: 2016-06-23T20:48:38Z
FALCON-1931 : Adding multiCluster tag for Multiple Cluster scenarios
In the regression test suite, there are lot of places multiCluster annotation has been defined to denote it uses multiple cluster. It is very useful annotation since we require additional cluster and setup to test the test case . Some of the major test cases such as feed replication and Hive DR are missing this annotation.
Author: Murali Ramasami <mr...@Murali-Ramasami-MBP.local>
Author: Murali Ramasami <mr...@hortonworks.com>
Reviewers: "Raghav Gautam <ra...@apache.org>"
Closes #123 from muraliramasami/multiClusterTag
(cherry picked from commit 7dab15f0036c56c06a29d36a5df0527823a930f8)
Signed-off-by: Sowmya Ramesh <sr...@hortonworks.com>
commit 600b6bcc231b0f5e9516b1b60fa9a93eae4da137
Author: yzheng-hortonworks <yz...@hortonworks.com>
Date: 2016-06-24T22:23:17Z
FALCON-2045 Enhance document on registry point in cluster specifiction for Hive HA mode
Author: yzheng-hortonworks <yz...@hortonworks.com>
Reviewers: "Balu Vellanki <ba...@apache.org>"
Closes #197 from yzheng-hortonworks/FALCON-2045
(cherry picked from commit e8d1de35d60d32d499530412fc4592abaf16f544)
Signed-off-by: bvellanki <bv...@hortonworks.com>
commit c0b9b4d1ce6036818acf5951bd17017b16f988f6
Author: Venkatesan Ramachandran <vr...@hortonworks.com>
Date: 2016-06-30T00:34:21Z
FALCON-2051 : PostProcessing needs to send JMS message so that REPL metrics can be added to the GraphDB
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] falcon pull request #200: Falcon 2051-Falcon post-processing services are no...
Posted by vramachan <gi...@git.apache.org>.
Github user vramachan closed the pull request at:
https://github.com/apache/falcon/pull/200
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---