[SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

Oozie is starting the release process for Oozie 4.3.0. This thread is for discussion about the scope of the release. Please provide your suggestion about what should be included (or any exclusions) in the release.

Currently, we have around 158 patches submitted to trunk. Based on the scope, we will include (exclude) more if required.
Version Upgrade 
Java 1.7 (OOZIE-2036 <https://issues.apache.org/jira/browse/OOZIE-2036>)
OpenJPA (OOZIE-2488 <https://issues.apache.org/jira/browse/OOZIE-2488>), 
Curator  (OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231>), 
HttpClient  (OOZIE-2538 <https://issues.apache.org/jira/browse/OOZIE-2538>), 
Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602>).
New Features/Improvements: Add default JT and NN, Database schema service check, instrumental metrics, Tomcat upgrade etc.
Bug Fixes.
Provide your opinion on inclusion/exclusion of the above points or more for the scope of the release. We can also, take some work which is already done and review is pending for them (depending on the availability of the reviewers).

Thanks,
Abhishek

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

All the 4.3.0 release marked jiras are completed. Release candidate tag is created and pushed. Now working on creating release artifacts.

Regards,
Abhishek
> On Oct 19, 2016, at 9:28 PM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hi All,
> 
> Currently we have three Oozie Blocker issues marked for 4.3.0 release.
> OOZIE-2621
> OOZIE-2705
> OOZIE-2710
> 
> Robert, Rohini, Jaydeep, Purshotam : Would it be possible to review these them.
> 
> Regards,
> Abhishek
>> On Oct 11, 2016, at 7:42 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>> 
>> Hi Robert,
>> 
>> OOZIE-2658 has been rebased and OOZIE-2613 is also ready with the discussed workaround in the issue thread.
>> Keeping those for 4.3 and moving all the others.
>> 
>> Thanks,
>> Abhishek
>>> On Oct 8, 2016, at 5:01 AM, Robert Kanter <rk...@cloudera.com> wrote:
>>> 
>>> - I took care of OOZIE-1814 yesterday.
>>> - OOZIE-2658 should be good after it gets rebased (doesn't apply cleanly);
>>> I'm guessing that will have to wait until Monday PDT due to timezones.
>>> - I'm not sure what's the current status of OOZIE-2613.  test-patch looks
>>> like it's ready, but from subsequent comments, it sounds like it's not?
>>> 
>>> - Robert
>>> 
>>> On Fri, Oct 7, 2016 at 5:18 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>>> 
>>>> Hi All,
>>>> 
>>>> Today EOD (UTC), we will freeze the scope and further commits into
>>>> branch-4.3 expect any blocker issues.
>>>> 
>>>> By early next week, we will have our first release candidate for review.
>>>> 
>>>> Regards,
>>>> Abhishek
>>>>> On Sep 23, 2016, at 11:11 PM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> The new branch creation and update of the version in the current branch
>>>> will be done on Monday (IST).
>>>>> 
>>>>> Thanks,
>>>>> Abhishek
>>>>>> On Sep 21, 2016, at 10:56 PM, Robert Kanter <rk...@cloudera.com>
>>>> wrote:
>>>>>> 
>>>>>> --94eb2c04982ae57b7b053d07d72b
>>>>>> Content-Type: text/plain; charset=UTF-8
>>>>>> 
>>>>>> Oh, right.  I forgot that Shwetha was going to help you with that.
>>>> That's
>>>>>> fine with me.
>>>>>> 
>>>>>> On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna <ba...@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Hello Robert,
>>>>>>> 
>>>>>>> Yes. I do not have permissions to do it. I was planning to take
>>>> Shwetha's
>>>>>>> help. I am fine with you doing it.
>>>>>>> Please let me know if there is any change in the plan.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Abhishek
>>>>>>>> On Sep 21, 2016, at 2:11 AM, Robert Kanter <rk...@cloudera.com>
>>>> wrote:
>>>>>>>> 
>>>>>>>> Abhishek,
>>>>>>>> 
>>>>>>>> You don't have git permissions to create the release branch.  I can
>>>> take
>>>>>>>> care of that on Thursday if you want.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> - Robert
>>>>>>>> 
>>>>>>>> On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <bafna.iitr@gmail.com
>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hello All,
>>>>>>>>> 
>>>>>>>>> I think we can create the new branch by Thursday (IST) and update
>>>>>>> version
>>>>>>>>> for new branch and master. If we have any important pending Jira's by
>>>>>>> then
>>>>>>>>> we will commit them to both new branch and master.
>>>>>>>>> 
>>>>>>>>> I will start working on the next steps for release from release doc.
>>>>>>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
>>>>>>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
>>>>>>>>> 
>>>>>>>>> Please do provide if you have any feedback or suggestion.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Abhishek
>>>>>>>>>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> Sorry Abhishek for the long delay. Was on vacation and then busy
>>>>>>> catching
>>>>>>>>>> up with stuff. I did review a bunch of patches today which were
>>>> marked
>>>>>>>>>> 4.3.0 and Patch Available apart from those which Robert or others
>>>> have
>>>>>>>>>> already not commented on. Skipping those as they have better
>>>> context.
>>>>>>> Tag
>>>>>>>>>> my name if there is something else that needs to be looked at.
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Rohini
>>>>>>>>>> 
>>>>>>>>>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <
>>>> bafna.iitr@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hello Robert, Rohini, Purshotam, Jaydeep
>>>>>>>>>>> 
>>>>>>>>>>> As we discussed earlier, will be bumping the priority of some
>>>> Jira's
>>>>>>> to
>>>>>>>>>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try
>>>> to
>>>>>>>>>>> complete as much possible out of that. (It would to good complete
>>>>>>> all.)
>>>>>>>>>>> 
>>>>>>>>>>> @All, If I miss something, Please do priorities or send a message
>>>>>>> here.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks a lot for all the efforts and time we have put in last few
>>>>>>> weeks.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Abhishek
>>>>>>>>>>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <bafna.iitr@gmail.com
>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>> 
>>>>>>>>>>>> It has been a long time since we started the scope discussion for
>>>> the
>>>>>>>>>>> Oozie-4.3.0 release.
>>>>>>>>>>>> Since then a lot of patches got reviewed and committed and I
>>>> would to
>>>>>>>>>>> thank each one you for contributing your efforts for that.
>>>>>>>>>>>> 
>>>>>>>>>>>> I think we should move to the next stage and go into blocker only
>>>>>>> mode
>>>>>>>>>>> from next week (one week). After that 7-10 days for preparing and
>>>>>>>>>>> completing release condidate.
>>>>>>>>>>>> 
>>>>>>>>>>>> We did plan to update some of the old tech and some of them are
>>>> still
>>>>>>>>>>> pending for review. If can spend time into reviewing them, it
>>>> would be
>>>>>>>>>>> good, else we might have to move without them.
>>>>>>>>>>>> 
>>>>>>>>>>>> There are also some good to have patches available for review and
>>>>>>>>>>> commit, I will bump up the priority for them. If I miss something,
>>>>>>>>> please
>>>>>>>>>>> feel free to do the same.
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Abhishek
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <
>>>> bafna.iitr@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Can we please look into these upgrade related bugs?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
>>>>>>>>>>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
>>>>>>>>>>>>> OpenJPA              https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2488
>>>>>>>>>>>>> Curator              https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2231
>>>>>>>>>>>>> Hive                 https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2613
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Are the current patches good and feedback if there is any.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Also, If we can do some reviews collectively.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <
>>>> bafna.iitr@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hello All,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> How is it looking now?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I think we should move to blocker and critical patch mode. If
>>>>>>>>>>> required, we can take couple of days extra there.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <
>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Sure Robert. We can use some more time.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <
>>>> rkanter@cloudera.com
>>>>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Do you think we could wait one more week?  I've been really
>>>>>>>>> hammered
>>>>>>>>>>> with
>>>>>>>>>>>>>>>> some other work and there's more that I'd like to review here
>>>>>>> that
>>>>>>>>> I
>>>>>>>>>>>>>>>> haven't been able to get to.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Sure. I guess we can one extra week and complete some more
>>>> work.
>>>>>>>>>>>>>>>>> Thanks everyone for responding.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
>>>>>>>>>>> purushah@yahoo-inc.com.INVALID>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Yes. I also have some pending patches.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>>>>>>>>>>>>>> rkanter@cloudera.com> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Ya, I could use some more time as well.  I have some
>>>> deadlines
>>>>>>>>>>> early next
>>>>>>>>>>>>>>>>>> week, which has kept me from doing more reviews.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>>>>>>>>>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Let see if we can wait for another week. So we can merge
>>>> few
>>>>>>>>> more
>>>>>>>>>>>>>>>>> important
>>>>>>>>>>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
>>>>>>>>> merged.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Jaydeep
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> As previously discussed, we will be moving with whatever
>>>>>>>>> patches
>>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>>> reviewed/merged till end of this week.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> But there are some which needs to be reviewed and merged,
>>>>>>>>> mainly
>>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I
>>>>>>> think
>>>>>>>>> we
>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".
>>>>>>> Patch
>>>>>>>>>>>>>>>>>>> Available
>>>>>>>>>>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%
>>>> 20%3D%204.3.0%20and%20status%
>>>>>>>>>>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%
>>>> 20DESC
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next
>>>> week
>>>>>>> to
>>>>>>>>>>> see if
>>>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a
>>>> few
>>>>>>>>>>> which are
>>>>>>>>>>>>>>>>>>>>> unassigned.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> If there is any jira, someone would like to include in
>>>> 4.3.0
>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>>> update
>>>>>>>>>>>>>>>>>>>>> the jira.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Unset the open ones without sending emails. Will update
>>>> the
>>>>>>>>>>> resolved,
>>>>>>>>>>>>>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I
>>>> agree,
>>>>>>>>> we
>>>>>>>>>>> need
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review
>>>> Patch
>>>>>>>>>>> Available
>>>>>>>>>>>>>>>>>>>> JIRAs.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop
>>>> workflowgenerator.
>>>>>>>>>>> We can
>>>>>>>>>>>>>>>>>>>> discuss
>>>>>>>>>>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
>>>>>>>>>>> OOZIE-2625 about
>>>>>>>>>>>>>>>>>>>> why.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>>>>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>>>>>>>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset
>>>>>>> fix
>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of
>>>> them
>>>>>>>>>>> being
>>>>>>>>>>>>>>>>>>> worked
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and
>>>> close
>>>>>>>>>>> many of
>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>>>>> renaming
>>>>>>>>>>> it to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> release number.
>>>>>>>>>>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we
>>>> go
>>>>>>>>> with
>>>>>>>>>>>>>>>>>>> unsetting
>>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
>>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>>> 4.4.0
>>>>>>>>>>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras
>>>> from
>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> I think it would be better to send out email for the
>>>>>>> bulk
>>>>>>>>>>> change.
>>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
>>>>>>>>> respond.
>>>>>>>>>>> We do
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks
>>>> come
>>>>>>>>>>> back with
>>>>>>>>>>>>>>>>>>>>>>>> feedback
>>>>>>>>>>>>>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please
>>>>>>> remember
>>>>>>>>>>> to tell
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
>>>>>>>>> we're
>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> "master"
>>>>>>>>>>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a
>>>>>>> little
>>>>>>>>>>>>>>>>>>> confusing.
>>>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>>>>>>>>> renaming it
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label
>>>> any
>>>>>>>>>>> current
>>>>>>>>>>>>>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine
>>>> that
>>>>>>>>> the
>>>>>>>>>>> next
>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we
>>>> can
>>>>>>>>>>> always
>>>>>>>>>>>>>>>>>>>>>>> rename it
>>>>>>>>>>>>>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
>>>>>>>>>>> suppose we
>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make
>>>>>>> things
>>>>>>>>>>> official
>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send
>>>> an
>>>>>>>>>>> email?
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I
>>>>>>> think
>>>>>>>>>>>>>>>>> Rohini's
>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next
>>>>>>> step.
>>>>>>>>>>> You're
>>>>>>>>>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of
>>>>>>> doing
>>>>>>>>>>> each
>>>>>>>>>>>>>>>>> JIRA
>>>>>>>>>>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
>>>>>>>>>>> reviews this
>>>>>>>>>>>>>>>>>>>>>>> week,
>>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>>> actually already started.  I should have some time
>>>> to
>>>>>>>>>>> continue
>>>>>>>>>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty
>>>> slow
>>>>>>>>> about
>>>>>>>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
>>>>>>>>>>> Satish was
>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini
>>>> Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to
>>>> trunk as
>>>>>>>>>>> 4.3.0
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will
>>>>>>> help
>>>>>>>>>>> filter
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc.
>>>> I
>>>>>>> can
>>>>>>>>>>> pick up
>>>>>>>>>>>>>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try
>>>> to
>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You
>>>> can
>>>>>>>>>>> then use
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the
>>>> trunk
>>>>>>>>>>> version
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release numbers when committing from the
>>>> beginning
>>>>>>>>> like
>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to
>>>> release
>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will
>>>> delete
>>>>>>>>>>> the trunk
>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to
>>>>>>> include
>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed
>>>> some
>>>>>>> of
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my
>>>> efforts to
>>>>>>>>>>> review
>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers
>>>>>>> and
>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>>> committed.
>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of
>>>> the
>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some
>>>>>>> fixed
>>>>>>>>>>>>>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly.
>>>>>>> Because,
>>>>>>>>>>> If we
>>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then
>>>> it
>>>>>>>>>>> might take
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week
>>>> of
>>>>>>>>>>> August to
>>>>>>>>>>>>>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and
>>>> suggestions.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and
>>>> prepared
>>>>>>> the
>>>>>>>>>>> list. I
>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now,
>>>>>>> into
>>>>>>>>> a
>>>>>>>>>>> word
>>>>>>>>>>>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and
>>>>>>> comment).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
>>>>>>>>> Added
>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
>>>>>>>>>>> property).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below)
>>>>>>> and
>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
>>>>>>>>> process.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm
>>>> not
>>>>>>>>>>> sure of
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and
>>>> it
>>>>>>>>>>> might make
>>>>>>>>>>>>>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's
>>>>>>> quite
>>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside
>>>> of
>>>>>>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support
>>>> for
>>>>>>>>>>> Java 1.6)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting
>>>> for
>>>>>>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
>>>>>>>>> security
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available
>>>> JIRAs
>>>>>>> not
>>>>>>>>>>> on your
>>>>>>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't
>>>> need
>>>>>>>>> to
>>>>>>>>>>>>>>>>> include
>>>>>>>>>>>>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have
>>>>>>> the
>>>>>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the
>>>> list?
>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-
>>>>>>> patch-available
>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>>>>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek
>>>> Bafna <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from
>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If
>>>> there
>>>>>>> is
>>>>>>>>>>>>>>>>> anything,
>>>>>>>>>>>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on
>>>> the
>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review
>>>> and
>>>>>>>>>>> commit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
>>>>>>>>>>> ridiculously
>>>>>>>>>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for
>>>> Coord
>>>>>>>>>>> action
>>>>>>>>>>>>>>>>> rerun
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception
>>>> when
>>>>>>>>>>> reference
>>>>>>>>>>>>>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids
>>>>>>> for
>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for
>>>> startCreatedTime
>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns
>>>>>>> incorrect
>>>>>>>>>>> path for
>>>>>>>>>>>>>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown
>>>> when
>>>>>>>>>>> job is
>>>>>>>>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex
>>>> consume
>>>>>>>>>>> more than
>>>>>>>>>>>>>>>>>>>>>>> 90%
>>>>>>>>>>>>>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
>>>>>>>>>>> replication-enabled mysql
>>>>>>>>>>>>>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService
>>>> for
>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains
>>>>>>> spaces
>>>>>>>>>>> can not
>>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
>>>>>>>>>>> hdfs-site and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify
>>>>>>> group,it
>>>>>>>>>>> does`t
>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is
>>>> used
>>>>>>> for
>>>>>>>>>>> 'hive
>>>>>>>>>>>>>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does
>>>> not
>>>>>>>>>>> return
>>>>>>>>>>>>>>>>> port
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
>>>>>>>>> "Changing
>>>>>>>>>>>>>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration
>>>> for
>>>>>>>>>>>>>>>>> downstream
>>>>>>>>>>>>>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service
>>>> documentation
>>>>>>>>>>> for jobs
>>>>>>>>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option
>>>> mentioned
>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> minioozie
>>>>>>>>>>>>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not
>>>> be
>>>>>>>>>>> used for
>>>>>>>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and
>>>> Bundle
>>>>>>>>> job
>>>>>>>>>>>>>>>>>>>>>>> querying
>>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files
>>>> and
>>>>>>>>>>>>>>>>>>>>>>> directories
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
>>>>>>>>> failed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
>>>>>>>>> connection
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
>>>>>>>>>>> workflow
>>>>>>>>>>>>>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib
>>>> create'
>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option
>>>>>>> removing
>>>>>>>>>>>>>>>>> completed
>>>>>>>>>>>>>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into
>>>> Failed
>>>>>>>>>>> state when
>>>>>>>>>>>>>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords
>>>> in
>>>>>>> the
>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
>>>>>>>>>>> looked good
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there
>>>>>>> are
>>>>>>>>>>> things,
>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which
>>>> are
>>>>>>> in
>>>>>>>>>>>>>>>>> progress
>>>>>>>>>>>>>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
>>>>>>>>>>> (discontinued)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
>>>>>>>>> upgrades.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release.
>>>> We
>>>>>>>>>>> will start
>>>>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek
>>>> Bafna <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to
>>>> 1.7.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
>>>>>>>>>>> Curator,
>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we
>>>> have
>>>>>>>>>>> done
>>>>>>>>>>>>>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2231
>>>>>>>>>> ,
>>>>>>>>>>>>>>>>> curator
>>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert
>>>> Kanter <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat
>>>> based
>>>>>>>>> on
>>>>>>>>>>> 1.1.0
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment
>>>>>>> out a
>>>>>>>>>>> few
>>>>>>>>>>>>>>>>>>>>>>> lines
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
>>>>>>>>> MiniHCatServer
>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>>>> resetDefaultDBCreation()
>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will
>>>> not
>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is
>>>> no
>>>>>>>>>>> database
>>>>>>>>>>>>>>>>>>>>>>> named
>>>>>>>>>>>>>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(
>>>> true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need
>>>> this
>>>>>>> for
>>>>>>>>>>> Hive
>>>>>>>>>>>>>>>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(
>>>>>>> true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null,
>>>> false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>>>>>>>>> resetHiveConfStaticVariables()
>>>>>>>>>>>>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/
>>>> apache/oozie/test/hive/
>>>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2
>>>>>>> extends
>>>>>>>>>>>>>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_
>>>> SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>>>>>>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>>> HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
>>>>>>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this
>>>> for
>>>>>>>>> Hive
>>>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
>>>>>>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
>>>>>>>>>>> "scratch");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
>>>>>>>>>>> Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
>>>>>>>>>>> version also
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines
>>>> defined
>>>>>>>>>>> anywhere
>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
>>>>>>>>> perhaps
>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said,
>>>> Hadoop
>>>>>>>>>>> did this
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine
>>>>>>> with
>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>>>>>> it in
>>>>>>>>>>>>>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure
>>>> to
>>>>>>>>> call
>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for
>>>>>>> anything
>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is
>>>> not
>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action
>>>>>>> JIRAs;
>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending
>>>> ones.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you
>>>> think
>>>>>>>>>>> needs more
>>>>>>>>>>>>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the
>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM,
>>>> Abhishek
>>>>>>>>> Bafna
>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:
>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the
>>>> scope
>>>>>>> for
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek
>>>>>>> Bafna
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process
>>>> for
>>>>>>>>>>> Oozie
>>>>>>>>>>>>>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
>>>>>>>>> release.
>>>>>>>>>>> Please
>>>>>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be
>>>> included
>>>>>>> (or
>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
>>>>>>>>>>> submitted to
>>>>>>>>>>>>>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more
>>>> if
>>>>>>>>>>> required.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>> jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2036
>>>>>>>>>>> )
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>> jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2488
>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>> jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>> jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2538
>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>> jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add
>>>> default JT
>>>>>>>>> and
>>>>>>>>>>> NN,
>>>>>>>>>>>>>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics,
>>>>>>> Tomcat
>>>>>>>>>>> upgrade
>>>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on
>>>>>>> inclusion/exclusion
>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>> above
>>>>>>>>>>>>>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can
>>>> also,
>>>>>>>>>>> take some
>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
>>>>>>>>> (depending
>>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

Currently we have three Oozie Blocker issues marked for 4.3.0 release.
OOZIE-2621
OOZIE-2705
OOZIE-2710

Robert, Rohini, Jaydeep, Purshotam : Would it be possible to review these them.

Regards,
Abhishek
> On Oct 11, 2016, at 7:42 AM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hi Robert,
> 
> OOZIE-2658 has been rebased and OOZIE-2613 is also ready with the discussed workaround in the issue thread.
> Keeping those for 4.3 and moving all the others.
> 
> Thanks,
> Abhishek
>> On Oct 8, 2016, at 5:01 AM, Robert Kanter <rk...@cloudera.com> wrote:
>> 
>> - I took care of OOZIE-1814 yesterday.
>> - OOZIE-2658 should be good after it gets rebased (doesn't apply cleanly);
>> I'm guessing that will have to wait until Monday PDT due to timezones.
>> - I'm not sure what's the current status of OOZIE-2613.  test-patch looks
>> like it's ready, but from subsequent comments, it sounds like it's not?
>> 
>> - Robert
>> 
>> On Fri, Oct 7, 2016 at 5:18 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>> 
>>> Hi All,
>>> 
>>> Today EOD (UTC), we will freeze the scope and further commits into
>>> branch-4.3 expect any blocker issues.
>>> 
>>> By early next week, we will have our first release candidate for review.
>>> 
>>> Regards,
>>> Abhishek
>>>> On Sep 23, 2016, at 11:11 PM, Abhishek Bafna <ba...@gmail.com>
>>> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> The new branch creation and update of the version in the current branch
>>> will be done on Monday (IST).
>>>> 
>>>> Thanks,
>>>> Abhishek
>>>>> On Sep 21, 2016, at 10:56 PM, Robert Kanter <rk...@cloudera.com>
>>> wrote:
>>>>> 
>>>>> --94eb2c04982ae57b7b053d07d72b
>>>>> Content-Type: text/plain; charset=UTF-8
>>>>> 
>>>>> Oh, right.  I forgot that Shwetha was going to help you with that.
>>> That's
>>>>> fine with me.
>>>>> 
>>>>> On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna <ba...@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> Hello Robert,
>>>>>> 
>>>>>> Yes. I do not have permissions to do it. I was planning to take
>>> Shwetha's
>>>>>> help. I am fine with you doing it.
>>>>>> Please let me know if there is any change in the plan.
>>>>>> 
>>>>>> Thanks,
>>>>>> Abhishek
>>>>>>> On Sep 21, 2016, at 2:11 AM, Robert Kanter <rk...@cloudera.com>
>>> wrote:
>>>>>>> 
>>>>>>> Abhishek,
>>>>>>> 
>>>>>>> You don't have git permissions to create the release branch.  I can
>>> take
>>>>>>> care of that on Thursday if you want.
>>>>>>> 
>>>>>>> 
>>>>>>> - Robert
>>>>>>> 
>>>>>>> On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <bafna.iitr@gmail.com
>>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hello All,
>>>>>>>> 
>>>>>>>> I think we can create the new branch by Thursday (IST) and update
>>>>>> version
>>>>>>>> for new branch and master. If we have any important pending Jira's by
>>>>>> then
>>>>>>>> we will commit them to both new branch and master.
>>>>>>>> 
>>>>>>>> I will start working on the next steps for release from release doc.
>>>>>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
>>>>>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
>>>>>>>> 
>>>>>>>> Please do provide if you have any feedback or suggestion.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Abhishek
>>>>>>>>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Sorry Abhishek for the long delay. Was on vacation and then busy
>>>>>> catching
>>>>>>>>> up with stuff. I did review a bunch of patches today which were
>>> marked
>>>>>>>>> 4.3.0 and Patch Available apart from those which Robert or others
>>> have
>>>>>>>>> already not commented on. Skipping those as they have better
>>> context.
>>>>>> Tag
>>>>>>>>> my name if there is something else that needs to be looked at.
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Rohini
>>>>>>>>> 
>>>>>>>>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <
>>> bafna.iitr@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hello Robert, Rohini, Purshotam, Jaydeep
>>>>>>>>>> 
>>>>>>>>>> As we discussed earlier, will be bumping the priority of some
>>> Jira's
>>>>>> to
>>>>>>>>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try
>>> to
>>>>>>>>>> complete as much possible out of that. (It would to good complete
>>>>>> all.)
>>>>>>>>>> 
>>>>>>>>>> @All, If I miss something, Please do priorities or send a message
>>>>>> here.
>>>>>>>>>> 
>>>>>>>>>> Thanks a lot for all the efforts and time we have put in last few
>>>>>> weeks.
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> Abhishek
>>>>>>>>>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <bafna.iitr@gmail.com
>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi All,
>>>>>>>>>>> 
>>>>>>>>>>> It has been a long time since we started the scope discussion for
>>> the
>>>>>>>>>> Oozie-4.3.0 release.
>>>>>>>>>>> Since then a lot of patches got reviewed and committed and I
>>> would to
>>>>>>>>>> thank each one you for contributing your efforts for that.
>>>>>>>>>>> 
>>>>>>>>>>> I think we should move to the next stage and go into blocker only
>>>>>> mode
>>>>>>>>>> from next week (one week). After that 7-10 days for preparing and
>>>>>>>>>> completing release condidate.
>>>>>>>>>>> 
>>>>>>>>>>> We did plan to update some of the old tech and some of them are
>>> still
>>>>>>>>>> pending for review. If can spend time into reviewing them, it
>>> would be
>>>>>>>>>> good, else we might have to move without them.
>>>>>>>>>>> 
>>>>>>>>>>> There are also some good to have patches available for review and
>>>>>>>>>> commit, I will bump up the priority for them. If I miss something,
>>>>>>>> please
>>>>>>>>>> feel free to do the same.
>>>>>>>>>>> 
>>>>>>>>>>> Regards,
>>>>>>>>>>> Abhishek
>>>>>>>>>>> 
>>>>>>>>>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <
>>> bafna.iitr@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>> 
>>>>>>>>>>>> Can we please look into these upgrade related bugs?
>>>>>>>>>>>> 
>>>>>>>>>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
>>>>>>>>>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
>>>>>>>>>>>> OpenJPA              https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2488
>>>>>>>>>>>> Curator              https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2231
>>>>>>>>>>>> Hive                 https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2613
>>>>>>>>>>>> 
>>>>>>>>>>>> Are the current patches good and feedback if there is any.
>>>>>>>>>>>> 
>>>>>>>>>>>> Also, If we can do some reviews collectively.
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Abhishek
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <
>>> bafna.iitr@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello All,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> How is it looking now?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I think we should move to blocker and critical patch mode. If
>>>>>>>>>> required, we can take couple of days extra there.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <
>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Sure Robert. We can use some more time.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <
>>> rkanter@cloudera.com
>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Do you think we could wait one more week?  I've been really
>>>>>>>> hammered
>>>>>>>>>> with
>>>>>>>>>>>>>>> some other work and there's more that I'd like to review here
>>>>>> that
>>>>>>>> I
>>>>>>>>>>>>>>> haven't been able to get to.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Sure. I guess we can one extra week and complete some more
>>> work.
>>>>>>>>>>>>>>>> Thanks everyone for responding.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
>>>>>>>>>> purushah@yahoo-inc.com.INVALID>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Yes. I also have some pending patches.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>>>>>>>>>>>>> rkanter@cloudera.com> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Ya, I could use some more time as well.  I have some
>>> deadlines
>>>>>>>>>> early next
>>>>>>>>>>>>>>>>> week, which has kept me from doing more reviews.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>>>>>>>>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Let see if we can wait for another week. So we can merge
>>> few
>>>>>>>> more
>>>>>>>>>>>>>>>> important
>>>>>>>>>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
>>>>>>>> merged.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Jaydeep
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> As previously discussed, we will be moving with whatever
>>>>>>>> patches
>>>>>>>>>> can be
>>>>>>>>>>>>>>>>>>> reviewed/merged till end of this week.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> But there are some which needs to be reviewed and merged,
>>>>>>>> mainly
>>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I
>>>>>> think
>>>>>>>> we
>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".
>>>>>> Patch
>>>>>>>>>>>>>>>>>> Available
>>>>>>>>>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%
>>> 20%3D%204.3.0%20and%20status%
>>>>>>>>>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%
>>> 20DESC
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next
>>> week
>>>>>> to
>>>>>>>>>> see if
>>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a
>>> few
>>>>>>>>>> which are
>>>>>>>>>>>>>>>>>>>> unassigned.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> If there is any jira, someone would like to include in
>>> 4.3.0
>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>> update
>>>>>>>>>>>>>>>>>>>> the jira.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Unset the open ones without sending emails. Will update
>>> the
>>>>>>>>>> resolved,
>>>>>>>>>>>>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I
>>> agree,
>>>>>>>> we
>>>>>>>>>> need
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review
>>> Patch
>>>>>>>>>> Available
>>>>>>>>>>>>>>>>>>> JIRAs.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop
>>> workflowgenerator.
>>>>>>>>>> We can
>>>>>>>>>>>>>>>>>>> discuss
>>>>>>>>>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
>>>>>>>>>> OOZIE-2625 about
>>>>>>>>>>>>>>>>>>> why.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>>>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>>>>>>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset
>>>>>> fix
>>>>>>>>>> version
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of
>>> them
>>>>>>>>>> being
>>>>>>>>>>>>>>>>>> worked
>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and
>>> close
>>>>>>>>>> many of
>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>>>> renaming
>>>>>>>>>> it to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> release number.
>>>>>>>>>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we
>>> go
>>>>>>>> with
>>>>>>>>>>>>>>>>>> unsetting
>>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
>>>>>>>>>> create a
>>>>>>>>>>>>>>>>>> 4.4.0
>>>>>>>>>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras
>>> from
>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> I think it would be better to send out email for the
>>>>>> bulk
>>>>>>>>>> change.
>>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
>>>>>>>> respond.
>>>>>>>>>> We do
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks
>>> come
>>>>>>>>>> back with
>>>>>>>>>>>>>>>>>>>>>>> feedback
>>>>>>>>>>>>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please
>>>>>> remember
>>>>>>>>>> to tell
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
>>>>>>>> we're
>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> "master"
>>>>>>>>>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a
>>>>>> little
>>>>>>>>>>>>>>>>>> confusing.
>>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>>>>>>>> renaming it
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label
>>> any
>>>>>>>>>> current
>>>>>>>>>>>>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine
>>> that
>>>>>>>> the
>>>>>>>>>> next
>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we
>>> can
>>>>>>>>>> always
>>>>>>>>>>>>>>>>>>>>>> rename it
>>>>>>>>>>>>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
>>>>>>>>>> suppose we
>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make
>>>>>> things
>>>>>>>>>> official
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send
>>> an
>>>>>>>>>> email?
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I
>>>>>> think
>>>>>>>>>>>>>>>> Rohini's
>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next
>>>>>> step.
>>>>>>>>>> You're
>>>>>>>>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of
>>>>>> doing
>>>>>>>>>> each
>>>>>>>>>>>>>>>> JIRA
>>>>>>>>>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
>>>>>>>>>> reviews this
>>>>>>>>>>>>>>>>>>>>>> week,
>>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>>> actually already started.  I should have some time
>>> to
>>>>>>>>>> continue
>>>>>>>>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty
>>> slow
>>>>>>>> about
>>>>>>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy
>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
>>>>>>>>>> Satish was
>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini
>>> Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to
>>> trunk as
>>>>>>>>>> 4.3.0
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will
>>>>>> help
>>>>>>>>>> filter
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc.
>>> I
>>>>>> can
>>>>>>>>>> pick up
>>>>>>>>>>>>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try
>>> to
>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You
>>> can
>>>>>>>>>> then use
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the
>>> trunk
>>>>>>>>>> version
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release numbers when committing from the
>>> beginning
>>>>>>>> like
>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to
>>> release
>>>>>>>>>> version
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will
>>> delete
>>>>>>>>>> the trunk
>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to
>>>>>> include
>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed
>>> some
>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my
>>> efforts to
>>>>>>>>>> review
>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers
>>>>>> and
>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>>> committed.
>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of
>>> the
>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some
>>>>>> fixed
>>>>>>>>>>>>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly.
>>>>>> Because,
>>>>>>>>>> If we
>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then
>>> it
>>>>>>>>>> might take
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week
>>> of
>>>>>>>>>> August to
>>>>>>>>>>>>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and
>>> suggestions.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and
>>> prepared
>>>>>> the
>>>>>>>>>> list. I
>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now,
>>>>>> into
>>>>>>>> a
>>>>>>>>>> word
>>>>>>>>>>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and
>>>>>> comment).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
>>>>>>>> Added
>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
>>>>>>>>>> property).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below)
>>>>>> and
>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
>>>>>>>> process.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm
>>> not
>>>>>>>>>> sure of
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and
>>> it
>>>>>>>>>> might make
>>>>>>>>>>>>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's
>>>>>> quite
>>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside
>>> of
>>>>>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support
>>> for
>>>>>>>>>> Java 1.6)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting
>>> for
>>>>>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
>>>>>>>> security
>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available
>>> JIRAs
>>>>>> not
>>>>>>>>>> on your
>>>>>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't
>>> need
>>>>>>>> to
>>>>>>>>>>>>>>>> include
>>>>>>>>>>>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have
>>>>>> the
>>>>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the
>>> list?
>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-
>>>>>> patch-available
>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>>>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek
>>> Bafna <
>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from
>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If
>>> there
>>>>>> is
>>>>>>>>>>>>>>>> anything,
>>>>>>>>>>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on
>>> the
>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review
>>> and
>>>>>>>>>> commit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
>>>>>>>>>> ridiculously
>>>>>>>>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for
>>> Coord
>>>>>>>>>> action
>>>>>>>>>>>>>>>> rerun
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception
>>> when
>>>>>>>>>> reference
>>>>>>>>>>>>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids
>>>>>> for
>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for
>>> startCreatedTime
>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns
>>>>>> incorrect
>>>>>>>>>> path for
>>>>>>>>>>>>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown
>>> when
>>>>>>>>>> job is
>>>>>>>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex
>>> consume
>>>>>>>>>> more than
>>>>>>>>>>>>>>>>>>>>>> 90%
>>>>>>>>>>>>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
>>>>>>>>>> replication-enabled mysql
>>>>>>>>>>>>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService
>>> for
>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains
>>>>>> spaces
>>>>>>>>>> can not
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
>>>>>>>>>> hdfs-site and
>>>>>>>>>>>>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify
>>>>>> group,it
>>>>>>>>>> does`t
>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is
>>> used
>>>>>> for
>>>>>>>>>> 'hive
>>>>>>>>>>>>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does
>>> not
>>>>>>>>>> return
>>>>>>>>>>>>>>>> port
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
>>>>>>>> "Changing
>>>>>>>>>>>>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration
>>> for
>>>>>>>>>>>>>>>> downstream
>>>>>>>>>>>>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service
>>> documentation
>>>>>>>>>> for jobs
>>>>>>>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option
>>> mentioned
>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> minioozie
>>>>>>>>>>>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not
>>> be
>>>>>>>>>> used for
>>>>>>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and
>>> Bundle
>>>>>>>> job
>>>>>>>>>>>>>>>>>>>>>> querying
>>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files
>>> and
>>>>>>>>>>>>>>>>>>>>>> directories
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
>>>>>>>> failed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
>>>>>>>> connection
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
>>>>>>>>>> workflow
>>>>>>>>>>>>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib
>>> create'
>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option
>>>>>> removing
>>>>>>>>>>>>>>>> completed
>>>>>>>>>>>>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into
>>> Failed
>>>>>>>>>> state when
>>>>>>>>>>>>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords
>>> in
>>>>>> the
>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna
>>> <
>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
>>>>>>>>>> looked good
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there
>>>>>> are
>>>>>>>>>> things,
>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which
>>> are
>>>>>> in
>>>>>>>>>>>>>>>> progress
>>>>>>>>>>>>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
>>>>>>>>>> (discontinued)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
>>>>>>>> upgrades.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release.
>>> We
>>>>>>>>>> will start
>>>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek
>>> Bafna <
>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to
>>> 1.7.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
>>>>>>>>>> Curator,
>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we
>>> have
>>>>>>>>>> done
>>>>>>>>>>>>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2231
>>>>>>>>> ,
>>>>>>>>>>>>>>>> curator
>>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert
>>> Kanter <
>>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat
>>> based
>>>>>>>> on
>>>>>>>>>> 1.1.0
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment
>>>>>> out a
>>>>>>>>>> few
>>>>>>>>>>>>>>>>>>>>>> lines
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
>>>>>>>> MiniHCatServer
>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>>> resetDefaultDBCreation()
>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will
>>> not
>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is
>>> no
>>>>>>>>>> database
>>>>>>>>>>>>>>>>>>>>>> named
>>>>>>>>>>>>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(
>>> true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need
>>> this
>>>>>> for
>>>>>>>>>> Hive
>>>>>>>>>>>>>>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(
>>>>>> true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null,
>>> false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>>>>>>>> resetHiveConfStaticVariables()
>>>>>>>>>>>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/
>>> apache/oozie/test/hive/
>>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2
>>>>>> extends
>>>>>>>>>>>>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_
>>> SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>>>>>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>> HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
>>>>>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this
>>> for
>>>>>>>> Hive
>>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
>>>>>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
>>>>>>>>>> "scratch");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
>>>>>>>>>> Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
>>>>>>>>>> version also
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines
>>> defined
>>>>>>>>>> anywhere
>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
>>>>>>>> perhaps
>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said,
>>> Hadoop
>>>>>>>>>> did this
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine
>>>>>> with
>>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>>>>> it in
>>>>>>>>>>>>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure
>>> to
>>>>>>>> call
>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for
>>>>>> anything
>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is
>>> not
>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action
>>>>>> JIRAs;
>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending
>>> ones.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you
>>> think
>>>>>>>>>> needs more
>>>>>>>>>>>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the
>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM,
>>> Abhishek
>>>>>>>> Bafna
>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:
>>>>>> bafna.iitr@gmail.com
>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the
>>> scope
>>>>>> for
>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek
>>>>>> Bafna
>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process
>>> for
>>>>>>>>>> Oozie
>>>>>>>>>>>>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
>>>>>>>> release.
>>>>>>>>>> Please
>>>>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be
>>> included
>>>>>> (or
>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
>>>>>>>>>> submitted to
>>>>>>>>>>>>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more
>>> if
>>>>>>>>>> required.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>> jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2036
>>>>>>>>>> )
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>> jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2488
>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>> jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>> jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2538
>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>> jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add
>>> default JT
>>>>>>>> and
>>>>>>>>>> NN,
>>>>>>>>>>>>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics,
>>>>>> Tomcat
>>>>>>>>>> upgrade
>>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on
>>>>>> inclusion/exclusion
>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>> above
>>>>>>>>>>>>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can
>>> also,
>>>>>>>>>> take some
>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
>>>>>>>> (depending
>>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>> 
>>> 
>>> 
>>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi Robert,

OOZIE-2658 has been rebased and OOZIE-2613 is also ready with the discussed workaround in the issue thread.
Keeping those for 4.3 and moving all the others.

Thanks,
Abhishek
> On Oct 8, 2016, at 5:01 AM, Robert Kanter <rk...@cloudera.com> wrote:
> 
> - I took care of OOZIE-1814 yesterday.
> - OOZIE-2658 should be good after it gets rebased (doesn't apply cleanly);
> I'm guessing that will have to wait until Monday PDT due to timezones.
> - I'm not sure what's the current status of OOZIE-2613.  test-patch looks
> like it's ready, but from subsequent comments, it sounds like it's not?
> 
> - Robert
> 
> On Fri, Oct 7, 2016 at 5:18 AM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
>> Hi All,
>> 
>> Today EOD (UTC), we will freeze the scope and further commits into
>> branch-4.3 expect any blocker issues.
>> 
>> By early next week, we will have our first release candidate for review.
>> 
>> Regards,
>> Abhishek
>>> On Sep 23, 2016, at 11:11 PM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>>> 
>>> Hi All,
>>> 
>>> The new branch creation and update of the version in the current branch
>> will be done on Monday (IST).
>>> 
>>> Thanks,
>>> Abhishek
>>>> On Sep 21, 2016, at 10:56 PM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>>>> 
>>>> --94eb2c04982ae57b7b053d07d72b
>>>> Content-Type: text/plain; charset=UTF-8
>>>> 
>>>> Oh, right.  I forgot that Shwetha was going to help you with that.
>> That's
>>>> fine with me.
>>>> 
>>>> On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>> 
>>>>> Hello Robert,
>>>>> 
>>>>> Yes. I do not have permissions to do it. I was planning to take
>> Shwetha's
>>>>> help. I am fine with you doing it.
>>>>> Please let me know if there is any change in the plan.
>>>>> 
>>>>> Thanks,
>>>>> Abhishek
>>>>>> On Sep 21, 2016, at 2:11 AM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>>>>>> 
>>>>>> Abhishek,
>>>>>> 
>>>>>> You don't have git permissions to create the release branch.  I can
>> take
>>>>>> care of that on Thursday if you want.
>>>>>> 
>>>>>> 
>>>>>> - Robert
>>>>>> 
>>>>>> On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <bafna.iitr@gmail.com
>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> Hello All,
>>>>>>> 
>>>>>>> I think we can create the new branch by Thursday (IST) and update
>>>>> version
>>>>>>> for new branch and master. If we have any important pending Jira's by
>>>>> then
>>>>>>> we will commit them to both new branch and master.
>>>>>>> 
>>>>>>> I will start working on the next steps for release from release doc.
>>>>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
>>>>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
>>>>>>> 
>>>>>>> Please do provide if you have any feedback or suggestion.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Abhishek
>>>>>>>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> Sorry Abhishek for the long delay. Was on vacation and then busy
>>>>> catching
>>>>>>>> up with stuff. I did review a bunch of patches today which were
>> marked
>>>>>>>> 4.3.0 and Patch Available apart from those which Robert or others
>> have
>>>>>>>> already not commented on. Skipping those as they have better
>> context.
>>>>> Tag
>>>>>>>> my name if there is something else that needs to be looked at.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Rohini
>>>>>>>> 
>>>>>>>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hello Robert, Rohini, Purshotam, Jaydeep
>>>>>>>>> 
>>>>>>>>> As we discussed earlier, will be bumping the priority of some
>> Jira's
>>>>> to
>>>>>>>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try
>> to
>>>>>>>>> complete as much possible out of that. (It would to good complete
>>>>> all.)
>>>>>>>>> 
>>>>>>>>> @All, If I miss something, Please do priorities or send a message
>>>>> here.
>>>>>>>>> 
>>>>>>>>> Thanks a lot for all the efforts and time we have put in last few
>>>>> weeks.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Abhishek
>>>>>>>>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <bafna.iitr@gmail.com
>>> 
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi All,
>>>>>>>>>> 
>>>>>>>>>> It has been a long time since we started the scope discussion for
>> the
>>>>>>>>> Oozie-4.3.0 release.
>>>>>>>>>> Since then a lot of patches got reviewed and committed and I
>> would to
>>>>>>>>> thank each one you for contributing your efforts for that.
>>>>>>>>>> 
>>>>>>>>>> I think we should move to the next stage and go into blocker only
>>>>> mode
>>>>>>>>> from next week (one week). After that 7-10 days for preparing and
>>>>>>>>> completing release condidate.
>>>>>>>>>> 
>>>>>>>>>> We did plan to update some of the old tech and some of them are
>> still
>>>>>>>>> pending for review. If can spend time into reviewing them, it
>> would be
>>>>>>>>> good, else we might have to move without them.
>>>>>>>>>> 
>>>>>>>>>> There are also some good to have patches available for review and
>>>>>>>>> commit, I will bump up the priority for them. If I miss something,
>>>>>>> please
>>>>>>>>> feel free to do the same.
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Abhishek
>>>>>>>>>> 
>>>>>>>>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <
>> bafna.iitr@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi All,
>>>>>>>>>>> 
>>>>>>>>>>> Can we please look into these upgrade related bugs?
>>>>>>>>>>> 
>>>>>>>>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
>>>>>>>>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
>>>>>>>>>>> OpenJPA              https://issues.apache.org/
>>>>> jira/browse/OOZIE-2488
>>>>>>>>>>> Curator              https://issues.apache.org/
>>>>> jira/browse/OOZIE-2231
>>>>>>>>>>> Hive                 https://issues.apache.org/
>>>>> jira/browse/OOZIE-2613
>>>>>>>>>>> 
>>>>>>>>>>> Are the current patches good and feedback if there is any.
>>>>>>>>>>> 
>>>>>>>>>>> Also, If we can do some reviews collectively.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Abhishek
>>>>>>>>>>> 
>>>>>>>>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hello All,
>>>>>>>>>>>> 
>>>>>>>>>>>> How is it looking now?
>>>>>>>>>>>> 
>>>>>>>>>>>> I think we should move to blocker and critical patch mode. If
>>>>>>>>> required, we can take couple of days extra there.
>>>>>>>>>>>> 
>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <
>>>>> bafna.iitr@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Sure Robert. We can use some more time.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <
>> rkanter@cloudera.com
>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Do you think we could wait one more week?  I've been really
>>>>>>> hammered
>>>>>>>>> with
>>>>>>>>>>>>>> some other work and there's more that I'd like to review here
>>>>> that
>>>>>>> I
>>>>>>>>>>>>>> haven't been able to get to.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Sure. I guess we can one extra week and complete some more
>> work.
>>>>>>>>>>>>>>> Thanks everyone for responding.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
>>>>>>>>> purushah@yahoo-inc.com.INVALID>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Yes. I also have some pending patches.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>>>>>>>>>>>> rkanter@cloudera.com> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Ya, I could use some more time as well.  I have some
>> deadlines
>>>>>>>>> early next
>>>>>>>>>>>>>>>> week, which has kept me from doing more reviews.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>>>>>>>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Let see if we can wait for another week. So we can merge
>> few
>>>>>>> more
>>>>>>>>>>>>>>> important
>>>>>>>>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
>>>>>>> merged.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Jaydeep
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> As previously discussed, we will be moving with whatever
>>>>>>> patches
>>>>>>>>> can be
>>>>>>>>>>>>>>>>>> reviewed/merged till end of this week.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> But there are some which needs to be reviewed and merged,
>>>>>>> mainly
>>>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I
>>>>> think
>>>>>>> we
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".
>>>>> Patch
>>>>>>>>>>>>>>>>> Available
>>>>>>>>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%
>> 20%3D%204.3.0%20and%20status%
>>>>>>>>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%
>> 20DESC
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next
>> week
>>>>> to
>>>>>>>>> see if
>>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a
>> few
>>>>>>>>> which are
>>>>>>>>>>>>>>>>>>> unassigned.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> If there is any jira, someone would like to include in
>> 4.3.0
>>>>>>>>> please
>>>>>>>>>>>>>>>>>> update
>>>>>>>>>>>>>>>>>>> the jira.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Unset the open ones without sending emails. Will update
>> the
>>>>>>>>> resolved,
>>>>>>>>>>>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I
>> agree,
>>>>>>> we
>>>>>>>>> need
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review
>> Patch
>>>>>>>>> Available
>>>>>>>>>>>>>>>>>> JIRAs.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop
>> workflowgenerator.
>>>>>>>>> We can
>>>>>>>>>>>>>>>>>> discuss
>>>>>>>>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
>>>>>>>>> OOZIE-2625 about
>>>>>>>>>>>>>>>>>> why.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>>>>>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset
>>>>> fix
>>>>>>>>> version
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of
>> them
>>>>>>>>> being
>>>>>>>>>>>>>>>>> worked
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and
>> close
>>>>>>>>> many of
>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>>> renaming
>>>>>>>>> it to
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> release number.
>>>>>>>>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we
>> go
>>>>>>> with
>>>>>>>>>>>>>>>>> unsetting
>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
>>>>>>>>> create a
>>>>>>>>>>>>>>>>> 4.4.0
>>>>>>>>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras
>> from
>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I think it would be better to send out email for the
>>>>> bulk
>>>>>>>>> change.
>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
>>>>>>> respond.
>>>>>>>>> We do
>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks
>> come
>>>>>>>>> back with
>>>>>>>>>>>>>>>>>>>>>> feedback
>>>>>>>>>>>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please
>>>>> remember
>>>>>>>>> to tell
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
>>>>>>> we're
>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> "master"
>>>>>>>>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a
>>>>> little
>>>>>>>>>>>>>>>>> confusing.
>>>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>>>>>>> renaming it
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label
>> any
>>>>>>>>> current
>>>>>>>>>>>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine
>> that
>>>>>>> the
>>>>>>>>> next
>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we
>> can
>>>>>>>>> always
>>>>>>>>>>>>>>>>>>>>> rename it
>>>>>>>>>>>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
>>>>>>>>> suppose we
>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make
>>>>> things
>>>>>>>>> official
>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send
>> an
>>>>>>>>> email?
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I
>>>>> think
>>>>>>>>>>>>>>> Rohini's
>>>>>>>>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next
>>>>> step.
>>>>>>>>> You're
>>>>>>>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of
>>>>> doing
>>>>>>>>> each
>>>>>>>>>>>>>>> JIRA
>>>>>>>>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
>>>>>>>>> reviews this
>>>>>>>>>>>>>>>>>>>>> week,
>>>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>>>> actually already started.  I should have some time
>> to
>>>>>>>>> continue
>>>>>>>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty
>> slow
>>>>>>> about
>>>>>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy
>> <
>>>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
>>>>>>>>> Satish was
>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini
>> Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to
>> trunk as
>>>>>>>>> 4.3.0
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will
>>>>> help
>>>>>>>>> filter
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc.
>> I
>>>>> can
>>>>>>>>> pick up
>>>>>>>>>>>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try
>> to
>>>>>>> get
>>>>>>>>>>>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You
>> can
>>>>>>>>> then use
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
>>>>>>> release
>>>>>>>>>>>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the
>> trunk
>>>>>>>>> version
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>> release numbers when committing from the
>> beginning
>>>>>>> like
>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to
>> release
>>>>>>>>> version
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will
>> delete
>>>>>>>>> the trunk
>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to
>>>>> include
>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed
>> some
>>>>> of
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my
>> efforts to
>>>>>>>>> review
>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers
>>>>> and
>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>> committed.
>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of
>> the
>>>>>>>>> patches
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some
>>>>> fixed
>>>>>>>>>>>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly.
>>>>> Because,
>>>>>>>>> If we
>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then
>> it
>>>>>>>>> might take
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week
>> of
>>>>>>>>> August to
>>>>>>>>>>>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and
>> suggestions.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and
>> prepared
>>>>> the
>>>>>>>>> list. I
>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now,
>>>>> into
>>>>>>> a
>>>>>>>>> word
>>>>>>>>>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and
>>>>> comment).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
>>>>>>> Added
>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
>>>>>>>>> property).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below)
>>>>> and
>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
>>>>>>> process.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm
>> not
>>>>>>>>> sure of
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and
>> it
>>>>>>>>> might make
>>>>>>>>>>>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's
>>>>> quite
>>>>>>>>> ready
>>>>>>>>>>>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside
>> of
>>>>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support
>> for
>>>>>>>>> Java 1.6)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting
>> for
>>>>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
>>>>>>> security
>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available
>> JIRAs
>>>>> not
>>>>>>>>> on your
>>>>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't
>> need
>>>>>>> to
>>>>>>>>>>>>>>> include
>>>>>>>>>>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have
>>>>> the
>>>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the
>> list?
>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-
>>>>> patch-available
>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek
>> Bafna <
>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from
>>>>> currently
>>>>>>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If
>> there
>>>>> is
>>>>>>>>>>>>>>> anything,
>>>>>>>>>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on
>> the
>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review
>> and
>>>>>>>>> commit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
>>>>>>>>> ridiculously
>>>>>>>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for
>> Coord
>>>>>>>>> action
>>>>>>>>>>>>>>> rerun
>>>>>>>>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception
>> when
>>>>>>>>> reference
>>>>>>>>>>>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids
>>>>> for
>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for
>> startCreatedTime
>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns
>>>>> incorrect
>>>>>>>>> path for
>>>>>>>>>>>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown
>> when
>>>>>>>>> job is
>>>>>>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex
>> consume
>>>>>>>>> more than
>>>>>>>>>>>>>>>>>>>>> 90%
>>>>>>>>>>>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
>>>>>>>>> replication-enabled mysql
>>>>>>>>>>>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService
>> for
>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains
>>>>> spaces
>>>>>>>>> can not
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
>>>>>>>>> hdfs-site and
>>>>>>>>>>>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify
>>>>> group,it
>>>>>>>>> does`t
>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is
>> used
>>>>> for
>>>>>>>>> 'hive
>>>>>>>>>>>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does
>> not
>>>>>>>>> return
>>>>>>>>>>>>>>> port
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
>>>>>>> "Changing
>>>>>>>>>>>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration
>> for
>>>>>>>>>>>>>>> downstream
>>>>>>>>>>>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service
>> documentation
>>>>>>>>> for jobs
>>>>>>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option
>> mentioned
>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> minioozie
>>>>>>>>>>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not
>> be
>>>>>>>>> used for
>>>>>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and
>> Bundle
>>>>>>> job
>>>>>>>>>>>>>>>>>>>>> querying
>>>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files
>> and
>>>>>>>>>>>>>>>>>>>>> directories
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
>>>>>>> failed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
>>>>>>> connection
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
>>>>>>>>> workflow
>>>>>>>>>>>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib
>> create'
>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option
>>>>> removing
>>>>>>>>>>>>>>> completed
>>>>>>>>>>>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into
>> Failed
>>>>>>>>> state when
>>>>>>>>>>>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords
>> in
>>>>> the
>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna
>> <
>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
>>>>>>>>> looked good
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there
>>>>> are
>>>>>>>>> things,
>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which
>> are
>>>>> in
>>>>>>>>>>>>>>> progress
>>>>>>>>>>>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
>>>>>>>>> (discontinued)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
>>>>>>> upgrades.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release.
>> We
>>>>>>>>> will start
>>>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek
>> Bafna <
>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to
>> 1.7.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
>>>>>>>>> Curator,
>>>>>>>>>>>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we
>> have
>>>>>>>>> done
>>>>>>>>>>>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>> jira/browse/OOZIE-2231
>>>>>>>> ,
>>>>>>>>>>>>>>> curator
>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert
>> Kanter <
>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat
>> based
>>>>>>> on
>>>>>>>>> 1.1.0
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment
>>>>> out a
>>>>>>>>> few
>>>>>>>>>>>>>>>>>>>>> lines
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
>>>>>>> MiniHCatServer
>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>> resetDefaultDBCreation()
>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will
>> not
>>>>> be
>>>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is
>> no
>>>>>>>>> database
>>>>>>>>>>>>>>>>>>>>> named
>>>>>>>>>>>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(
>> true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need
>> this
>>>>> for
>>>>>>>>> Hive
>>>>>>>>>>>>>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(
>>>>> true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null,
>> false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>>>>>>> resetHiveConfStaticVariables()
>>>>>>>>>>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/
>> apache/oozie/test/hive/
>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2
>>>>> extends
>>>>>>>>>>>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_
>> SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>>>>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>> HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
>>>>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this
>> for
>>>>>>> Hive
>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
>>>>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
>>>>>>>>> "scratch");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
>>>>>>>>> Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
>>>>>>>>> version also
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines
>> defined
>>>>>>>>> anywhere
>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
>>>>>>> perhaps
>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said,
>> Hadoop
>>>>>>>>> did this
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine
>>>>> with
>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>>>> it in
>>>>>>>>>>>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure
>> to
>>>>>>> call
>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for
>>>>> anything
>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is
>> not
>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action
>>>>> JIRAs;
>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending
>> ones.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you
>> think
>>>>>>>>> needs more
>>>>>>>>>>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the
>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM,
>> Abhishek
>>>>>>> Bafna
>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:
>>>>> bafna.iitr@gmail.com
>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the
>> scope
>>>>> for
>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek
>>>>> Bafna
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process
>> for
>>>>>>>>> Oozie
>>>>>>>>>>>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
>>>>>>> release.
>>>>>>>>> Please
>>>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be
>> included
>>>>> (or
>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
>>>>>>>>> submitted to
>>>>>>>>>>>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more
>> if
>>>>>>>>> required.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>> jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>> jira/browse/OOZIE-2036
>>>>>>>>> )
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>> jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>> jira/browse/OOZIE-2488
>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>> jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>> jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>> jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>> jira/browse/OOZIE-2538
>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>> jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add
>> default JT
>>>>>>> and
>>>>>>>>> NN,
>>>>>>>>>>>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics,
>>>>> Tomcat
>>>>>>>>> upgrade
>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on
>>>>> inclusion/exclusion
>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>> above
>>>>>>>>>>>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can
>> also,
>>>>>>>>> take some
>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
>>>>>>> (depending
>>>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>> 
>> 
>> 
>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

- I took care of OOZIE-1814 yesterday.
- OOZIE-2658 should be good after it gets rebased (doesn't apply cleanly);
I'm guessing that will have to wait until Monday PDT due to timezones.
- I'm not sure what's the current status of OOZIE-2613.  test-patch looks
like it's ready, but from subsequent comments, it sounds like it's not?

- Robert

On Fri, Oct 7, 2016 at 5:18 AM, Abhishek Bafna <ba...@gmail.com> wrote:

> Hi All,
>
> Today EOD (UTC), we will freeze the scope and further commits into
> branch-4.3 expect any blocker issues.
>
> By early next week, we will have our first release candidate for review.
>
> Regards,
> Abhishek
> > On Sep 23, 2016, at 11:11 PM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > The new branch creation and update of the version in the current branch
> will be done on Monday (IST).
> >
> > Thanks,
> > Abhishek
> >> On Sep 21, 2016, at 10:56 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
> >>
> >> --94eb2c04982ae57b7b053d07d72b
> >> Content-Type: text/plain; charset=UTF-8
> >>
> >> Oh, right.  I forgot that Shwetha was going to help you with that.
> That's
> >> fine with me.
> >>
> >> On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna <ba...@gmail.com>
> >> wrote:
> >>
> >>> Hello Robert,
> >>>
> >>> Yes. I do not have permissions to do it. I was planning to take
> Shwetha's
> >>> help. I am fine with you doing it.
> >>> Please let me know if there is any change in the plan.
> >>>
> >>> Thanks,
> >>> Abhishek
> >>>> On Sep 21, 2016, at 2:11 AM, Robert Kanter <rk...@cloudera.com>
> wrote:
> >>>>
> >>>> Abhishek,
> >>>>
> >>>> You don't have git permissions to create the release branch.  I can
> take
> >>>> care of that on Thursday if you want.
> >>>>
> >>>>
> >>>> - Robert
> >>>>
> >>>> On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <bafna.iitr@gmail.com
> >
> >>>> wrote:
> >>>>
> >>>>> Hello All,
> >>>>>
> >>>>> I think we can create the new branch by Thursday (IST) and update
> >>> version
> >>>>> for new branch and master. If we have any important pending Jira's by
> >>> then
> >>>>> we will commit them to both new branch and master.
> >>>>>
> >>>>> I will start working on the next steps for release from release doc.
> >>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
> >>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
> >>>>>
> >>>>> Please do provide if you have any feedback or suggestion.
> >>>>>
> >>>>> Thanks,
> >>>>> Abhishek
> >>>>>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
> >>>>> rohini.aditya@gmail.com> wrote:
> >>>>>>
> >>>>>> Sorry Abhishek for the long delay. Was on vacation and then busy
> >>> catching
> >>>>>> up with stuff. I did review a bunch of patches today which were
> marked
> >>>>>> 4.3.0 and Patch Available apart from those which Robert or others
> have
> >>>>>> already not commented on. Skipping those as they have better
> context.
> >>> Tag
> >>>>>> my name if there is something else that needs to be looked at.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Rohini
> >>>>>>
> >>>>>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <
> bafna.iitr@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hello Robert, Rohini, Purshotam, Jaydeep
> >>>>>>>
> >>>>>>> As we discussed earlier, will be bumping the priority of some
> Jira's
> >>> to
> >>>>>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try
> to
> >>>>>>> complete as much possible out of that. (It would to good complete
> >>> all.)
> >>>>>>>
> >>>>>>> @All, If I miss something, Please do priorities or send a message
> >>> here.
> >>>>>>>
> >>>>>>> Thanks a lot for all the efforts and time we have put in last few
> >>> weeks.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Abhishek
> >>>>>>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <bafna.iitr@gmail.com
> >
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> It has been a long time since we started the scope discussion for
> the
> >>>>>>> Oozie-4.3.0 release.
> >>>>>>>> Since then a lot of patches got reviewed and committed and I
> would to
> >>>>>>> thank each one you for contributing your efforts for that.
> >>>>>>>>
> >>>>>>>> I think we should move to the next stage and go into blocker only
> >>> mode
> >>>>>>> from next week (one week). After that 7-10 days for preparing and
> >>>>>>> completing release condidate.
> >>>>>>>>
> >>>>>>>> We did plan to update some of the old tech and some of them are
> still
> >>>>>>> pending for review. If can spend time into reviewing them, it
> would be
> >>>>>>> good, else we might have to move without them.
> >>>>>>>>
> >>>>>>>> There are also some good to have patches available for review and
> >>>>>>> commit, I will bump up the priority for them. If I miss something,
> >>>>> please
> >>>>>>> feel free to do the same.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Abhishek
> >>>>>>>>
> >>>>>>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <
> bafna.iitr@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi All,
> >>>>>>>>>
> >>>>>>>>> Can we please look into these upgrade related bugs?
> >>>>>>>>>
> >>>>>>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
> >>>>>>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
> >>>>>>>>> OpenJPA              https://issues.apache.org/
> >>> jira/browse/OOZIE-2488
> >>>>>>>>> Curator              https://issues.apache.org/
> >>> jira/browse/OOZIE-2231
> >>>>>>>>> Hive                 https://issues.apache.org/
> >>> jira/browse/OOZIE-2613
> >>>>>>>>>
> >>>>>>>>> Are the current patches good and feedback if there is any.
> >>>>>>>>>
> >>>>>>>>> Also, If we can do some reviews collectively.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Abhishek
> >>>>>>>>>
> >>>>>>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <
> bafna.iitr@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hello All,
> >>>>>>>>>>
> >>>>>>>>>> How is it looking now?
> >>>>>>>>>>
> >>>>>>>>>> I think we should move to blocker and critical patch mode. If
> >>>>>>> required, we can take couple of days extra there.
> >>>>>>>>>>
> >>>>>>>>>> ~Abhishek
> >>>>>>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <
> >>> bafna.iitr@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Sure Robert. We can use some more time.
> >>>>>>>>>>>
> >>>>>>>>>>> ~Abhishek
> >>>>>>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <
> rkanter@cloudera.com
> >>>>
> >>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do you think we could wait one more week?  I've been really
> >>>>> hammered
> >>>>>>> with
> >>>>>>>>>>>> some other work and there's more that I'd like to review here
> >>> that
> >>>>> I
> >>>>>>>>>>>> haven't been able to get to.
> >>>>>>>>>>>>
> >>>>>>>>>>>> thanks
> >>>>>>>>>>>> - Robert
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
> >>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Sure. I guess we can one extra week and complete some more
> work.
> >>>>>>>>>>>>> Thanks everyone for responding.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> ~Abhishek
> >>>>>>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
> >>>>>>> purushah@yahoo-inc.com.INVALID>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yes. I also have some pending patches.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
> >>>>>>>>>>>>> rkanter@cloudera.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ya, I could use some more time as well.  I have some
> deadlines
> >>>>>>> early next
> >>>>>>>>>>>>>> week, which has kept me from doing more reviews.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
> >>>>>>>>>>>>>> jaydeepmailv@gmail.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Let see if we can wait for another week. So we can merge
> few
> >>>>> more
> >>>>>>>>>>>>> important
> >>>>>>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
> >>>>> merged.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>> Jaydeep
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
> >>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> As previously discussed, we will be moving with whatever
> >>>>> patches
> >>>>>>> can be
> >>>>>>>>>>>>>>>> reviewed/merged till end of this week.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> But there are some which needs to be reviewed and merged,
> >>>>> mainly
> >>>>>>>>>>>>>>> regarding
> >>>>>>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I
> >>> think
> >>>>> we
> >>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>> review OOZIE-2036 and merged it.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> ~Abhishek
> >>>>>>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".
> >>> Patch
> >>>>>>>>>>>>>>> Available
> >>>>>>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%
> 20%3D%204.3.0%20and%20status%
> >>>>>>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%
> 20DESC
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next
> week
> >>> to
> >>>>>>> see if
> >>>>>>>>>>>>>>> any
> >>>>>>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a
> few
> >>>>>>> which are
> >>>>>>>>>>>>>>>>> unassigned.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If there is any jira, someone would like to include in
> 4.3.0
> >>>>>>> please
> >>>>>>>>>>>>>>>> update
> >>>>>>>>>>>>>>>>> the jira.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>> rohini.aditya@gmail.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Unset the open ones without sending emails. Will update
> the
> >>>>>>> resolved,
> >>>>>>>>>>>>>>>>>> closed and Patch Available ones sending emails.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
> >>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I
> agree,
> >>>>> we
> >>>>>>> need
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review
> Patch
> >>>>>>> Available
> >>>>>>>>>>>>>>>> JIRAs.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop
> workflowgenerator.
> >>>>>>> We can
> >>>>>>>>>>>>>>>> discuss
> >>>>>>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
> >>>>>>> OOZIE-2625 about
> >>>>>>>>>>>>>>>> why.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
> >>>>>>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
> >>>>>>>>>>>>>>>> 20status%20DESC&startIndex=50
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset
> >>> fix
> >>>>>>> version
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of
> them
> >>>>>>> being
> >>>>>>>>>>>>>>> worked
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and
> close
> >>>>>>> many of
> >>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
> >>> renaming
> >>>>>>> it to
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> release number.
> >>>>>>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we
> go
> >>>>> with
> >>>>>>>>>>>>>>> unsetting
> >>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
> >>>>>>> create a
> >>>>>>>>>>>>>>> 4.4.0
> >>>>>>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras
> from
> >>>>>>> 4.3.0.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I think it would be better to send out email for the
> >>> bulk
> >>>>>>> change.
> >>>>>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
> >>>>> respond.
> >>>>>>> We do
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks
> come
> >>>>>>> back with
> >>>>>>>>>>>>>>>>>>>> feedback
> >>>>>>>>>>>>>>>>>>>>> on pulling back to release.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
> >>>>>>>>>>>>>>> rkanter@cloudera.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please
> >>> remember
> >>>>>>> to tell
> >>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> send out an email for every change :)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
> >>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
> >>>>> we're
> >>>>>>> on
> >>>>>>>>>>>>>>>>>>> "master"
> >>>>>>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a
> >>> little
> >>>>>>>>>>>>>>> confusing.
> >>>>>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
> >>>>>>> renaming it
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label
> any
> >>>>>>> current
> >>>>>>>>>>>>>>>>>>> "trunk"
> >>>>>>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine
> that
> >>>>> the
> >>>>>>> next
> >>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we
> can
> >>>>>>> always
> >>>>>>>>>>>>>>>>>>> rename it
> >>>>>>>>>>>>>>>>>>>>>>> later.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
> >>>>>>> suppose we
> >>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make
> >>> things
> >>>>>>> official
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send
> an
> >>>>>>> email?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I
> >>> think
> >>>>>>>>>>>>> Rohini's
> >>>>>>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next
> >>> step.
> >>>>>>> You're
> >>>>>>>>>>>>>>>>>>>>>> probably
> >>>>>>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of
> >>> doing
> >>>>>>> each
> >>>>>>>>>>>>> JIRA
> >>>>>>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
> >>>>>>> reviews this
> >>>>>>>>>>>>>>>>>>> week,
> >>>>>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>>>>> actually already started.  I should have some time
> to
> >>>>>>> continue
> >>>>>>>>>>>>>>>>>>> doing
> >>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty
> slow
> >>>>> about
> >>>>>>>>>>>>>>> getting
> >>>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>>>> done :(
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy
> <
> >>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
> >>>>>>> Satish was
> >>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini
> Palaniswamy <
> >>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Abhishek,
> >>>>>>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to
> trunk as
> >>>>>>> 4.3.0
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will
> >>> help
> >>>>>>> filter
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc.
> I
> >>> can
> >>>>>>> pick up
> >>>>>>>>>>>>>>>>>>>> 10-15
> >>>>>>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try
> to
> >>>>> get
> >>>>>>>>>>>>>>>>>>> majority of
> >>>>>>>>>>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You
> can
> >>>>>>> then use
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> 3rd
> >>>>>>>>>>>>>>>>>>>>>>>> week
> >>>>>>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
> >>>>> release
> >>>>>>>>>>>>>>>>>>> candidate.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Robert,
> >>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the
> trunk
> >>>>>>> version
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>> release numbers when committing from the
> beginning
> >>>>> like
> >>>>>>> all
> >>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>> projects
> >>>>>>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to
> release
> >>>>>>> version
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> every
> >>>>>>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will
> delete
> >>>>>>> the trunk
> >>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
> >>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to
> >>> include
> >>>>>>> in the
> >>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>> In the
> >>>>>>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed
> some
> >>> of
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my
> efforts to
> >>>>>>> review
> >>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>> patches.
> >>>>>>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers
> >>> and
> >>>>>>> get
> >>>>>>>>>>>>>>>>>>>> committed.
> >>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of
> the
> >>>>>>> patches
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> provide
> >>>>>>>>>>>>>>>>>>>>>>>>>> their feedback.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some
> >>> fixed
> >>>>>>>>>>>>>>>>>>> (approximate)
> >>>>>>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly.
> >>> Because,
> >>>>>>> If we
> >>>>>>>>>>>>>>> wait
> >>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> all the
> >>>>>>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then
> it
> >>>>>>> might take
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>> lot
> >>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week
> of
> >>>>>>> August to
> >>>>>>>>>>>>>>>>>>>>>> complete
> >>>>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and
> suggestions.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and
> prepared
> >>> the
> >>>>>>> list. I
> >>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>> put,
> >>>>>>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now,
> >>> into
> >>>>> a
> >>>>>>> word
> >>>>>>>>>>>>>>>>>>>>>> document.
> >>>>>>>>>>>>>>>>>>>>>>>> (For
> >>>>>>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and
> >>> comment).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
> >>>>> Added
> >>>>>>> two
> >>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>> Jira
> >>>>>>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
> >>>>>>> property).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below)
> >>> and
> >>>>>>> provide
> >>>>>>>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>>>>>>> comments.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
> >>>>> process.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Scope Document:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
> >>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> >>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm
> not
> >>>>>>> sure of
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and
> it
> >>>>>>> might make
> >>>>>>>>>>>>>>>>>>>>>> sense to
> >>>>>>>>>>>>>>>>>>>>>>>>>> let it
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's
> >>> quite
> >>>>>>> ready
> >>>>>>>>>>>>>>>>>>> yet.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside
> of
> >>>>>>> Oozie)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support
> for
> >>>>>>> Java 1.6)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting
> for
> >>>>>>> Oozie)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
> >>>>> security
> >>>>>>> and
> >>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>> fixes)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> fixes)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available
> JIRAs
> >>> not
> >>>>>>> on your
> >>>>>>>>>>>>>>>>>>>> list
> >>>>>>>>>>>>>>>>>>>>>>>>>> (including
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't
> need
> >>>>> to
> >>>>>>>>>>>>> include
> >>>>>>>>>>>>>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have
> >>> the
> >>>>>>> review
> >>>>>>>>>>>>>>>>>>>>>>>> bandwidth),
> >>>>>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the
> list?
> >>>>> I
> >>>>>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>>>>> see
> >>>>>>>>>>>>>>>>>>>>>>>> 97
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-
> >>> patch-available
> >>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
> >>>>>>>>>>>>>>>>>>> fixes/improvements,
> >>>>>>>>>>>>>>>>>>>>>> given
> >>>>>>>>>>>>>>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek
> Bafna <
> >>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from
> >>> currently
> >>>>>>>>>>>>>>>>>>> available
> >>>>>>>>>>>>>>>>>>>>>>>> patches.
> >>>>>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If
> there
> >>> is
> >>>>>>>>>>>>> anything,
> >>>>>>>>>>>>>>>>>>>>>> please
> >>>>>>>>>>>>>>>>>>>>>>>>>> reply
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on
> the
> >>>>>>> feedback.
> >>>>>>>>>>>>>>>>>>>>>> Important
> >>>>>>>>>>>>>>>>>>>>>>>>>> is,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review
> and
> >>>>>>> commit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
> >>>>>>> provide
> >>>>>>>>>>>>>>>>>>>> feedback.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> ------------------------------
> >>>>> ------------------------------
> >>>>>>>>>>>>>>>> ---------------
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
> >>>>>>> ridiculously
> >>>>>>>>>>>>>>>>>>> slow
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>>>>>>> cases
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for
> Coord
> >>>>>>> action
> >>>>>>>>>>>>> rerun
> >>>>>>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception
> when
> >>>>>>> reference
> >>>>>>>>>>>>>>>>>>>>>> variable
> >>>>>>>>>>>>>>>>>>>>>>>> name
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> containing dot
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids
> >>> for
> >>>>>>> action
> >>>>>>>>>>>>>>>>>>>>>> failures
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for
> startCreatedTime
> >>>>> and
> >>>>>>>>>>>>>>>>>>>>>> endCreatedTime
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns
> >>> incorrect
> >>>>>>> path for
> >>>>>>>>>>>>>>>>>>> jar
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown
> when
> >>>>>>> job is
> >>>>>>>>>>>>>>>>>>>> running
> >>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> than expected time
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex
> consume
> >>>>>>> more than
> >>>>>>>>>>>>>>>>>>> 90%
> >>>>>>>>>>>>>>>>>>>>>> cpu
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
> >>>>>>> replication-enabled mysql
> >>>>>>>>>>>>>>>>>>>> urls
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService
> for
> >>>>>>> logging
> >>>>>>>>>>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains
> >>> spaces
> >>>>>>> can not
> >>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>> handled
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> rightly
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
> >>>>>>> hdfs-site and
> >>>>>>>>>>>>>>>>>>>>>>>> mapred-site
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify
> >>> group,it
> >>>>>>> does`t
> >>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is
> used
> >>> for
> >>>>>>> 'hive
> >>>>>>>>>>>>>>>>>>> log4j
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does
> not
> >>>>>>> return
> >>>>>>>>>>>>> port
> >>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -servers
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
> >>>>> "Changing
> >>>>>>>>>>>>>>>>>>>>>>>> endtime/pausetime
> >>>>>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration
> for
> >>>>>>>>>>>>> downstream
> >>>>>>>>>>>>>>>>>>>>>>>> components
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service
> documentation
> >>>>>>> for jobs
> >>>>>>>>>>>>>>>>>>> API
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option
> mentioned
> >>>>> in
> >>>>>>>>>>>>>>>>>>> minioozie
> >>>>>>>>>>>>>>>>>>>>>> doc
> >>>>>>>>>>>>>>>>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not
> be
> >>>>>>> used for
> >>>>>>>>>>>>>>>>>>> error
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> audit
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> logs
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and
> Bundle
> >>>>> job
> >>>>>>>>>>>>>>>>>>> querying
> >>>>>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files
> and
> >>>>>>>>>>>>>>>>>>> directories
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> /tmp
> >>>>>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
> >>>>> failed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
> >>>>> connection
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
> >>>>>>> workflow
> >>>>>>>>>>>>>>>>>>> action
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib
> create'
> >>>>>>> should
> >>>>>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>>>>> uploaded
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option
> >>> removing
> >>>>>>>>>>>>> completed
> >>>>>>>>>>>>>>>>>>>>>> output
> >>>>>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into
> Failed
> >>>>>>> state when
> >>>>>>>>>>>>>>>>>>>> kill
> >>>>>>>>>>>>>>>>>>>>>>>>>> control
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords
> in
> >>> the
> >>>>>>> logs
> >>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>> logging
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> command arguments
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> ------------------------------
> >>>>> ------------------------------
> >>>>>>>>>>>>>>>> ---------------
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna
> <
> >>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
> >>>>>>> looked good
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> me
> >>>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there
> >>> are
> >>>>>>> things,
> >>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>> needs
> >>>>>>>>>>>>>>>>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which
> are
> >>> in
> >>>>>>>>>>>>> progress
> >>>>>>>>>>>>>>>>>>>>>> status.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
> >>>>>>> (discontinued)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
> >>>>> upgrades.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release.
> We
> >>>>>>> will start
> >>>>>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek
> Bafna <
> >>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to
> 1.7.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
> >>>>>>> Curator,
> >>>>>>>>>>>>>>>>>>>>>> HttpClient,
> >>>>>>>>>>>>>>>>>>>>>>>> Pig
> >>>>>>>>>>>>>>>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we
> have
> >>>>>>> done
> >>>>>>>>>>>>>>>>>>>>>> internally as
> >>>>>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>> jira/browse/OOZIE-2231
> >>>>>> ,
> >>>>>>>>>>>>> curator
> >>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert
> Kanter <
> >>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat
> based
> >>>>> on
> >>>>>>> 1.1.0
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> over
> >>>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> year
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment
> >>> out a
> >>>>>>> few
> >>>>>>>>>>>>>>>>>>> lines
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
> >>>>>>> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
> >>>>>>> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
> >>>>>>> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
> >>>>>>> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
> >>>>> MiniHCatServer
> >>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
> resetDefaultDBCreation()
> >>>>>>> throws
> >>>>>>>>>>>>>>>>>>>>>> Exception {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will
> not
> >>> be
> >>>>>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is
> no
> >>>>>>> database
> >>>>>>>>>>>>>>>>>>> named
> >>>>>>>>>>>>>>>>>>>>>>>> default)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
> >>>>>>> createDefaultDB");
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(
> true);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need
> this
> >>> for
> >>>>>>> Hive
> >>>>>>>>>>>>>>>>>>> 0.14
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
> >>>>>>> createDefaultDB");
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(
> >>> true);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null,
> false);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
> >>>>> resetHiveConfStaticVariables()
> >>>>>>>>>>>>>>>>>>> throws
> >>>>>>>>>>>>>>>>>>>>>>>>>> Exception
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/
> apache/oozie/test/hive/
> >>>>>>> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
> >>>>> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
> >>>>>>> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
> >>>>>>> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2
> >>> extends
> >>>>>>>>>>>>>>>>>>>>>>>> AbstractHiveService
> >>>>>>>>>>>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_
> SERVER2_THRIFT_BIND_HOST,
> >>>>>>>>>>>>>>>>>>>>>>>>>> getHost());
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
> >>>>>>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
> >>> HIVE_SERVER2_THRIFT_HTTP_PORT,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
> >>>>>>> resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this
> for
> >>>>> Hive
> >>>>>>> 0.14
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
> >>>>>>> resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
> >>>>>>> "scratch");
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
> >>>>>>> Ranganathan <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
> >>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
> >>>>>>> version also
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>> 1.x.
> >>>>>>>>>>>>>>>>>>>>>>>>>> 0.13
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> >>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> <mailto:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines
> defined
> >>>>>>> anywhere
> >>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
> >>>>> perhaps
> >>>>>>> we
> >>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> wait
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said,
> Hadoop
> >>>>>>> did this
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>> minor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine
> >>> with
> >>>>>>> doing
> >>>>>>>>>>>>>>>>>>> it in
> >>>>>>>>>>>>>>>>>>>>>> 4.3
> >>>>>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure
> to
> >>>>> call
> >>>>>>> that
> >>>>>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for
> >>> anything
> >>>>> we
> >>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>> leave
> >>>>>>>>>>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is
> not
> >>>>>>> currently
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> trunk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> anyway.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action
> >>> JIRAs;
> >>>>>>> we
> >>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>>>>> sure
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending
> ones.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you
> think
> >>>>>>> needs more
> >>>>>>>>>>>>>>>>>>>>>> time to
> >>>>>>>>>>>>>>>>>>>>>>>>>> bake
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the
> release?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM,
> Abhishek
> >>>>> Bafna
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:
> >>> bafna.iitr@gmail.com
> >>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the
> scope
> >>> for
> >>>>>>> the
> >>>>>>>>>>>>>>>>>>> release?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek
> >>> Bafna
> >>>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process
> for
> >>>>>>> Oozie
> >>>>>>>>>>>>>>>>>>> 4.3.0.
> >>>>>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>>>>> thread
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
> >>>>> release.
> >>>>>>> Please
> >>>>>>>>>>>>>>>>>>>>>> provide
> >>>>>>>>>>>>>>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be
> included
> >>> (or
> >>>>>>> any
> >>>>>>>>>>>>>>>>>>>>>> exclusions)
> >>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
> >>>>>>> submitted to
> >>>>>>>>>>>>>>>>>>>> trunk.
> >>>>>>>>>>>>>>>>>>>>>>>> Based
> >>>>>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more
> if
> >>>>>>> required.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>> jira/browse/OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>> jira/browse/OOZIE-2036
> >>>>>>> )
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>> jira/browse/OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>> jira/browse/OOZIE-2488
> >>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>> jira/browse/OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>> jira/browse/OOZIE-2231>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>> jira/browse/OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>> jira/browse/OOZIE-2538
> >>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2602 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>> jira/browse/OOZIE-2602>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add
> default JT
> >>>>> and
> >>>>>>> NN,
> >>>>>>>>>>>>>>>>>>>> Database
> >>>>>>>>>>>>>>>>>>>>>>>>>> schema
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics,
> >>> Tomcat
> >>>>>>> upgrade
> >>>>>>>>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on
> >>> inclusion/exclusion
> >>>>>>> of the
> >>>>>>>>>>>>>>>>>>> above
> >>>>>>>>>>>>>>>>>>>>>>>> points
> >>>>>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can
> also,
> >>>>>>> take some
> >>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
> >>>>> (depending
> >>>>>>> on the
> >>>>>>>>>>>>>>>>>>>>>>>>>> availability
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>
> >>>>>>>
> >
>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

Today EOD (UTC), we will freeze the scope and further commits into branch-4.3 expect any blocker issues.

By early next week, we will have our first release candidate for review.

Regards,
Abhishek
> On Sep 23, 2016, at 11:11 PM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hi All,
> 
> The new branch creation and update of the version in the current branch will be done on Monday (IST).
> 
> Thanks,
> Abhishek
>> On Sep 21, 2016, at 10:56 PM, Robert Kanter <rk...@cloudera.com> wrote:
>> 
>> --94eb2c04982ae57b7b053d07d72b
>> Content-Type: text/plain; charset=UTF-8
>> 
>> Oh, right.  I forgot that Shwetha was going to help you with that.  That's
>> fine with me.
>> 
>> On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>> 
>>> Hello Robert,
>>> 
>>> Yes. I do not have permissions to do it. I was planning to take Shwetha's
>>> help. I am fine with you doing it.
>>> Please let me know if there is any change in the plan.
>>> 
>>> Thanks,
>>> Abhishek
>>>> On Sep 21, 2016, at 2:11 AM, Robert Kanter <rk...@cloudera.com> wrote:
>>>> 
>>>> Abhishek,
>>>> 
>>>> You don't have git permissions to create the release branch.  I can take
>>>> care of that on Thursday if you want.
>>>> 
>>>> 
>>>> - Robert
>>>> 
>>>> On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>> 
>>>>> Hello All,
>>>>> 
>>>>> I think we can create the new branch by Thursday (IST) and update
>>> version
>>>>> for new branch and master. If we have any important pending Jira's by
>>> then
>>>>> we will commit them to both new branch and master.
>>>>> 
>>>>> I will start working on the next steps for release from release doc.
>>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
>>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
>>>>> 
>>>>> Please do provide if you have any feedback or suggestion.
>>>>> 
>>>>> Thanks,
>>>>> Abhishek
>>>>>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
>>>>> rohini.aditya@gmail.com> wrote:
>>>>>> 
>>>>>> Sorry Abhishek for the long delay. Was on vacation and then busy
>>> catching
>>>>>> up with stuff. I did review a bunch of patches today which were marked
>>>>>> 4.3.0 and Patch Available apart from those which Robert or others have
>>>>>> already not commented on. Skipping those as they have better context.
>>> Tag
>>>>>> my name if there is something else that needs to be looked at.
>>>>>> 
>>>>>> Regards,
>>>>>> Rohini
>>>>>> 
>>>>>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <ba...@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Hello Robert, Rohini, Purshotam, Jaydeep
>>>>>>> 
>>>>>>> As we discussed earlier, will be bumping the priority of some Jira's
>>> to
>>>>>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try to
>>>>>>> complete as much possible out of that. (It would to good complete
>>> all.)
>>>>>>> 
>>>>>>> @All, If I miss something, Please do priorities or send a message
>>> here.
>>>>>>> 
>>>>>>> Thanks a lot for all the efforts and time we have put in last few
>>> weeks.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Abhishek
>>>>>>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <ba...@gmail.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi All,
>>>>>>>> 
>>>>>>>> It has been a long time since we started the scope discussion for the
>>>>>>> Oozie-4.3.0 release.
>>>>>>>> Since then a lot of patches got reviewed and committed and I would to
>>>>>>> thank each one you for contributing your efforts for that.
>>>>>>>> 
>>>>>>>> I think we should move to the next stage and go into blocker only
>>> mode
>>>>>>> from next week (one week). After that 7-10 days for preparing and
>>>>>>> completing release condidate.
>>>>>>>> 
>>>>>>>> We did plan to update some of the old tech and some of them are still
>>>>>>> pending for review. If can spend time into reviewing them, it would be
>>>>>>> good, else we might have to move without them.
>>>>>>>> 
>>>>>>>> There are also some good to have patches available for review and
>>>>>>> commit, I will bump up the priority for them. If I miss something,
>>>>> please
>>>>>>> feel free to do the same.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Abhishek
>>>>>>>> 
>>>>>>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi All,
>>>>>>>>> 
>>>>>>>>> Can we please look into these upgrade related bugs?
>>>>>>>>> 
>>>>>>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
>>>>>>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
>>>>>>>>> OpenJPA              https://issues.apache.org/
>>> jira/browse/OOZIE-2488
>>>>>>>>> Curator              https://issues.apache.org/
>>> jira/browse/OOZIE-2231
>>>>>>>>> Hive                 https://issues.apache.org/
>>> jira/browse/OOZIE-2613
>>>>>>>>> 
>>>>>>>>> Are the current patches good and feedback if there is any.
>>>>>>>>> 
>>>>>>>>> Also, If we can do some reviews collectively.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Abhishek
>>>>>>>>> 
>>>>>>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hello All,
>>>>>>>>>> 
>>>>>>>>>> How is it looking now?
>>>>>>>>>> 
>>>>>>>>>> I think we should move to blocker and critical patch mode. If
>>>>>>> required, we can take couple of days extra there.
>>>>>>>>>> 
>>>>>>>>>> ~Abhishek
>>>>>>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <
>>> bafna.iitr@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Sure Robert. We can use some more time.
>>>>>>>>>>> 
>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rkanter@cloudera.com
>>>> 
>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Do you think we could wait one more week?  I've been really
>>>>> hammered
>>>>>>> with
>>>>>>>>>>>> some other work and there's more that I'd like to review here
>>> that
>>>>> I
>>>>>>>>>>>> haven't been able to get to.
>>>>>>>>>>>> 
>>>>>>>>>>>> thanks
>>>>>>>>>>>> - Robert
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Sure. I guess we can one extra week and complete some more work.
>>>>>>>>>>>>> Thanks everyone for responding.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
>>>>>>> purushah@yahoo-inc.com.INVALID>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Yes. I also have some pending patches.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>>>>>>>>>> rkanter@cloudera.com> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Ya, I could use some more time as well.  I have some deadlines
>>>>>>> early next
>>>>>>>>>>>>>> week, which has kept me from doing more reviews.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>>>>>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Let see if we can wait for another week. So we can merge few
>>>>> more
>>>>>>>>>>>>> important
>>>>>>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
>>>>> merged.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Jaydeep
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> As previously discussed, we will be moving with whatever
>>>>> patches
>>>>>>> can be
>>>>>>>>>>>>>>>> reviewed/merged till end of this week.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> But there are some which needs to be reviewed and merged,
>>>>> mainly
>>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I
>>> think
>>>>> we
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".
>>> Patch
>>>>>>>>>>>>>>> Available
>>>>>>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>>>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next week
>>> to
>>>>>>> see if
>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a few
>>>>>>> which are
>>>>>>>>>>>>>>>>> unassigned.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> If there is any jira, someone would like to include in 4.3.0
>>>>>>> please
>>>>>>>>>>>>>>>> update
>>>>>>>>>>>>>>>>> the jira.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Unset the open ones without sending emails. Will update the
>>>>>>> resolved,
>>>>>>>>>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree,
>>>>> we
>>>>>>> need
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch
>>>>>>> Available
>>>>>>>>>>>>>>>> JIRAs.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.
>>>>>>> We can
>>>>>>>>>>>>>>>> discuss
>>>>>>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
>>>>>>> OOZIE-2625 about
>>>>>>>>>>>>>>>> why.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>>>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset
>>> fix
>>>>>>> version
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of them
>>>>>>> being
>>>>>>>>>>>>>>> worked
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and close
>>>>>>> many of
>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>> renaming
>>>>>>> it to
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> release number.
>>>>>>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we go
>>>>> with
>>>>>>>>>>>>>>> unsetting
>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
>>>>>>> create a
>>>>>>>>>>>>>>> 4.4.0
>>>>>>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras from
>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> I think it would be better to send out email for the
>>> bulk
>>>>>>> change.
>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
>>>>> respond.
>>>>>>> We do
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come
>>>>>>> back with
>>>>>>>>>>>>>>>>>>>> feedback
>>>>>>>>>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>>>>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please
>>> remember
>>>>>>> to tell
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
>>>>> we're
>>>>>>> on
>>>>>>>>>>>>>>>>>>> "master"
>>>>>>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a
>>> little
>>>>>>>>>>>>>>> confusing.
>>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>>>>> renaming it
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any
>>>>>>> current
>>>>>>>>>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that
>>>>> the
>>>>>>> next
>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can
>>>>>>> always
>>>>>>>>>>>>>>>>>>> rename it
>>>>>>>>>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
>>>>>>> suppose we
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make
>>> things
>>>>>>> official
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send an
>>>>>>> email?
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I
>>> think
>>>>>>>>>>>>> Rohini's
>>>>>>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next
>>> step.
>>>>>>> You're
>>>>>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of
>>> doing
>>>>>>> each
>>>>>>>>>>>>> JIRA
>>>>>>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
>>>>>>> reviews this
>>>>>>>>>>>>>>>>>>> week,
>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>>> actually already started.  I should have some time to
>>>>>>> continue
>>>>>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow
>>>>> about
>>>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
>>>>>>> Satish was
>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as
>>>>>>> 4.3.0
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will
>>> help
>>>>>>> filter
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I
>>> can
>>>>>>> pick up
>>>>>>>>>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to
>>>>> get
>>>>>>>>>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can
>>>>>>> then use
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
>>>>> release
>>>>>>>>>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk
>>>>>>> version
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>> release numbers when committing from the beginning
>>>>> like
>>>>>>> all
>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release
>>>>>>> version
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete
>>>>>>> the trunk
>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to
>>> include
>>>>>>> in the
>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some
>>> of
>>>>>>> the
>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to
>>>>>>> review
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers
>>> and
>>>>>>> get
>>>>>>>>>>>>>>>>>>>> committed.
>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of the
>>>>>>> patches
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some
>>> fixed
>>>>>>>>>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly.
>>> Because,
>>>>>>> If we
>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it
>>>>>>> might take
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of
>>>>>>> August to
>>>>>>>>>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared
>>> the
>>>>>>> list. I
>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now,
>>> into
>>>>> a
>>>>>>> word
>>>>>>>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and
>>> comment).
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
>>>>> Added
>>>>>>> two
>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
>>>>>>> property).
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below)
>>> and
>>>>>>> provide
>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
>>>>> process.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not
>>>>>>> sure of
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it
>>>>>>> might make
>>>>>>>>>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's
>>> quite
>>>>>>> ready
>>>>>>>>>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of
>>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for
>>>>>>> Java 1.6)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for
>>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
>>>>> security
>>>>>>> and
>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs
>>> not
>>>>>>> on your
>>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need
>>>>> to
>>>>>>>>>>>>> include
>>>>>>>>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have
>>> the
>>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?
>>>>> I
>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-
>>> patch-available
>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from
>>> currently
>>>>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there
>>> is
>>>>>>>>>>>>> anything,
>>>>>>>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the
>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and
>>>>>>> commit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
>>>>>>> provide
>>>>>>>>>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>> ------------------------------
>>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
>>>>>>> ridiculously
>>>>>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord
>>>>>>> action
>>>>>>>>>>>>> rerun
>>>>>>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when
>>>>>>> reference
>>>>>>>>>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids
>>> for
>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime
>>>>> and
>>>>>>>>>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns
>>> incorrect
>>>>>>> path for
>>>>>>>>>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when
>>>>>>> job is
>>>>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume
>>>>>>> more than
>>>>>>>>>>>>>>>>>>> 90%
>>>>>>>>>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
>>>>>>> replication-enabled mysql
>>>>>>>>>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for
>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains
>>> spaces
>>>>>>> can not
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
>>>>>>> hdfs-site and
>>>>>>>>>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify
>>> group,it
>>>>>>> does`t
>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used
>>> for
>>>>>>> 'hive
>>>>>>>>>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not
>>>>>>> return
>>>>>>>>>>>>> port
>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
>>>>> "Changing
>>>>>>>>>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>>>>>>>>>>>>> downstream
>>>>>>>>>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation
>>>>>>> for jobs
>>>>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned
>>>>> in
>>>>>>>>>>>>>>>>>>> minioozie
>>>>>>>>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be
>>>>>>> used for
>>>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle
>>>>> job
>>>>>>>>>>>>>>>>>>> querying
>>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>>>>>>>>>>>>> directories
>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
>>>>> failed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
>>>>> connection
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
>>>>>>> workflow
>>>>>>>>>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create'
>>>>>>> should
>>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option
>>> removing
>>>>>>>>>>>>> completed
>>>>>>>>>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed
>>>>>>> state when
>>>>>>>>>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in
>>> the
>>>>>>> logs
>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>> ------------------------------
>>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
>>>>>>> looked good
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there
>>> are
>>>>>>> things,
>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are
>>> in
>>>>>>>>>>>>> progress
>>>>>>>>>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
>>>>>>> (discontinued)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
>>>>> upgrades.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We
>>>>>>> will start
>>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
>>>>>>> Curator,
>>>>>>>>>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have
>>>>>>> done
>>>>>>>>>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>> jira/browse/OOZIE-2231
>>>>>> ,
>>>>>>>>>>>>> curator
>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based
>>>>> on
>>>>>>> 1.1.0
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment
>>> out a
>>>>>>> few
>>>>>>>>>>>>>>>>>>> lines
>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
>>>>> MiniHCatServer
>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation()
>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not
>>> be
>>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no
>>>>>>> database
>>>>>>>>>>>>>>>>>>> named
>>>>>>>>>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this
>>> for
>>>>>>> Hive
>>>>>>>>>>>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(
>>> true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>>>>> resetHiveConfStaticVariables()
>>>>>>>>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2
>>> extends
>>>>>>>>>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>> HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
>>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for
>>>>> Hive
>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
>>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
>>>>>>> "scratch");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
>>>>>>> Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
>>>>>>> version also
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined
>>>>>>> anywhere
>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
>>>>> perhaps
>>>>>>> we
>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop
>>>>>>> did this
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine
>>> with
>>>>>>> doing
>>>>>>>>>>>>>>>>>>> it in
>>>>>>>>>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to
>>>>> call
>>>>>>> that
>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for
>>> anything
>>>>> we
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not
>>>>>>> currently
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action
>>> JIRAs;
>>>>>>> we
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think
>>>>>>> needs more
>>>>>>>>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek
>>>>> Bafna
>>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:
>>> bafna.iitr@gmail.com
>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope
>>> for
>>>>>>> the
>>>>>>>>>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek
>>> Bafna
>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for
>>>>>>> Oozie
>>>>>>>>>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
>>>>> release.
>>>>>>> Please
>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included
>>> (or
>>>>>>> any
>>>>>>>>>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
>>>>>>> submitted to
>>>>>>>>>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if
>>>>>>> required.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>> jira/browse/OOZIE-2036
>>>>>>> )
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>> jira/browse/OOZIE-2488
>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>> jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>> jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>> jira/browse/OOZIE-2538
>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>> jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT
>>>>> and
>>>>>>> NN,
>>>>>>>>>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics,
>>> Tomcat
>>>>>>> upgrade
>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on
>>> inclusion/exclusion
>>>>>>> of the
>>>>>>>>>>>>>>>>>>> above
>>>>>>>>>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also,
>>>>>>> take some
>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
>>>>> (depending
>>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>> 
>>>>>>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

The new branch creation and update of the version in the current branch will be done on Monday (IST).

Thanks,
Abhishek
> On Sep 21, 2016, at 10:56 PM, Robert Kanter <rk...@cloudera.com> wrote:
> 
> --94eb2c04982ae57b7b053d07d72b
> Content-Type: text/plain; charset=UTF-8
> 
> Oh, right.  I forgot that Shwetha was going to help you with that.  That's
> fine with me.
> 
> On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> 
>> Hello Robert,
>> 
>> Yes. I do not have permissions to do it. I was planning to take Shwetha's
>> help. I am fine with you doing it.
>> Please let me know if there is any change in the plan.
>> 
>> Thanks,
>> Abhishek
>>> On Sep 21, 2016, at 2:11 AM, Robert Kanter <rk...@cloudera.com> wrote:
>>> 
>>> Abhishek,
>>> 
>>> You don't have git permissions to create the release branch.  I can take
>>> care of that on Thursday if you want.
>>> 
>>> 
>>> - Robert
>>> 
>>> On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <ba...@gmail.com>
>>> wrote:
>>> 
>>>> Hello All,
>>>> 
>>>> I think we can create the new branch by Thursday (IST) and update
>> version
>>>> for new branch and master. If we have any important pending Jira's by
>> then
>>>> we will commit them to both new branch and master.
>>>> 
>>>> I will start working on the next steps for release from release doc.
>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
>>>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
>>>> 
>>>> Please do provide if you have any feedback or suggestion.
>>>> 
>>>> Thanks,
>>>> Abhishek
>>>>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
>>>> rohini.aditya@gmail.com> wrote:
>>>>> 
>>>>> Sorry Abhishek for the long delay. Was on vacation and then busy
>> catching
>>>>> up with stuff. I did review a bunch of patches today which were marked
>>>>> 4.3.0 and Patch Available apart from those which Robert or others have
>>>>> already not commented on. Skipping those as they have better context.
>> Tag
>>>>> my name if there is something else that needs to be looked at.
>>>>> 
>>>>> Regards,
>>>>> Rohini
>>>>> 
>>>>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <ba...@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> Hello Robert, Rohini, Purshotam, Jaydeep
>>>>>> 
>>>>>> As we discussed earlier, will be bumping the priority of some Jira's
>> to
>>>>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try to
>>>>>> complete as much possible out of that. (It would to good complete
>> all.)
>>>>>> 
>>>>>> @All, If I miss something, Please do priorities or send a message
>> here.
>>>>>> 
>>>>>> Thanks a lot for all the efforts and time we have put in last few
>> weeks.
>>>>>> 
>>>>>> Thanks,
>>>>>> Abhishek
>>>>>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <ba...@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> It has been a long time since we started the scope discussion for the
>>>>>> Oozie-4.3.0 release.
>>>>>>> Since then a lot of patches got reviewed and committed and I would to
>>>>>> thank each one you for contributing your efforts for that.
>>>>>>> 
>>>>>>> I think we should move to the next stage and go into blocker only
>> mode
>>>>>> from next week (one week). After that 7-10 days for preparing and
>>>>>> completing release condidate.
>>>>>>> 
>>>>>>> We did plan to update some of the old tech and some of them are still
>>>>>> pending for review. If can spend time into reviewing them, it would be
>>>>>> good, else we might have to move without them.
>>>>>>> 
>>>>>>> There are also some good to have patches available for review and
>>>>>> commit, I will bump up the priority for them. If I miss something,
>>>> please
>>>>>> feel free to do the same.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Abhishek
>>>>>>> 
>>>>>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com>
>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi All,
>>>>>>>> 
>>>>>>>> Can we please look into these upgrade related bugs?
>>>>>>>> 
>>>>>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
>>>>>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
>>>>>>>> OpenJPA              https://issues.apache.org/
>> jira/browse/OOZIE-2488
>>>>>>>> Curator              https://issues.apache.org/
>> jira/browse/OOZIE-2231
>>>>>>>> Hive                 https://issues.apache.org/
>> jira/browse/OOZIE-2613
>>>>>>>> 
>>>>>>>> Are the current patches good and feedback if there is any.
>>>>>>>> 
>>>>>>>> Also, If we can do some reviews collectively.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Abhishek
>>>>>>>> 
>>>>>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com>
>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Hello All,
>>>>>>>>> 
>>>>>>>>> How is it looking now?
>>>>>>>>> 
>>>>>>>>> I think we should move to blocker and critical patch mode. If
>>>>>> required, we can take couple of days extra there.
>>>>>>>>> 
>>>>>>>>> ~Abhishek
>>>>>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com>
>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Sure Robert. We can use some more time.
>>>>>>>>>> 
>>>>>>>>>> ~Abhishek
>>>>>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rkanter@cloudera.com
>>> 
>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Do you think we could wait one more week?  I've been really
>>>> hammered
>>>>>> with
>>>>>>>>>>> some other work and there's more that I'd like to review here
>> that
>>>> I
>>>>>>>>>>> haven't been able to get to.
>>>>>>>>>>> 
>>>>>>>>>>> thanks
>>>>>>>>>>> - Robert
>>>>>>>>>>> 
>>>>>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Sure. I guess we can one extra week and complete some more work.
>>>>>>>>>>>> Thanks everyone for responding.
>>>>>>>>>>>> 
>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
>>>>>> purushah@yahoo-inc.com.INVALID>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Yes. I also have some pending patches.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>>>>>>>>> rkanter@cloudera.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Ya, I could use some more time as well.  I have some deadlines
>>>>>> early next
>>>>>>>>>>>>> week, which has kept me from doing more reviews.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> thanks
>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>>>>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Let see if we can wait for another week. So we can merge few
>>>> more
>>>>>>>>>>>> important
>>>>>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
>>>> merged.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Jaydeep
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> As previously discussed, we will be moving with whatever
>>>> patches
>>>>>> can be
>>>>>>>>>>>>>>> reviewed/merged till end of this week.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> But there are some which needs to be reviewed and merged,
>>>> mainly
>>>>>>>>>>>>>> regarding
>>>>>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I
>> think
>>>> we
>>>>>>>>>>>> should
>>>>>>>>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".
>> Patch
>>>>>>>>>>>>>> Available
>>>>>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next week
>> to
>>>>>> see if
>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a few
>>>>>> which are
>>>>>>>>>>>>>>>> unassigned.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> If there is any jira, someone would like to include in 4.3.0
>>>>>> please
>>>>>>>>>>>>>>> update
>>>>>>>>>>>>>>>> the jira.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Unset the open ones without sending emails. Will update the
>>>>>> resolved,
>>>>>>>>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree,
>>>> we
>>>>>> need
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch
>>>>>> Available
>>>>>>>>>>>>>>> JIRAs.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.
>>>>>> We can
>>>>>>>>>>>>>>> discuss
>>>>>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
>>>>>> OOZIE-2625 about
>>>>>>>>>>>>>>> why.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset
>> fix
>>>>>> version
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of them
>>>>>> being
>>>>>>>>>>>>>> worked
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and close
>>>>>> many of
>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>> renaming
>>>>>> it to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> release number.
>>>>>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we go
>>>> with
>>>>>>>>>>>>>> unsetting
>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
>>>>>> create a
>>>>>>>>>>>>>> 4.4.0
>>>>>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras from
>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I think it would be better to send out email for the
>> bulk
>>>>>> change.
>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
>>>> respond.
>>>>>> We do
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come
>>>>>> back with
>>>>>>>>>>>>>>>>>>> feedback
>>>>>>>>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>>>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please
>> remember
>>>>>> to tell
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
>>>> we're
>>>>>> on
>>>>>>>>>>>>>>>>>> "master"
>>>>>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a
>> little
>>>>>>>>>>>>>> confusing.
>>>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>>>> renaming it
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any
>>>>>> current
>>>>>>>>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that
>>>> the
>>>>>> next
>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can
>>>>>> always
>>>>>>>>>>>>>>>>>> rename it
>>>>>>>>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
>>>>>> suppose we
>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make
>> things
>>>>>> official
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send an
>>>>>> email?
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I
>> think
>>>>>>>>>>>> Rohini's
>>>>>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next
>> step.
>>>>>> You're
>>>>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of
>> doing
>>>>>> each
>>>>>>>>>>>> JIRA
>>>>>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
>>>>>> reviews this
>>>>>>>>>>>>>>>>>> week,
>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>> actually already started.  I should have some time to
>>>>>> continue
>>>>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow
>>>> about
>>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
>>>>>> Satish was
>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as
>>>>>> 4.3.0
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will
>> help
>>>>>> filter
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I
>> can
>>>>>> pick up
>>>>>>>>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to
>>>> get
>>>>>>>>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can
>>>>>> then use
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
>>>> release
>>>>>>>>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk
>>>>>> version
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>> release numbers when committing from the beginning
>>>> like
>>>>>> all
>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release
>>>>>> version
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete
>>>>>> the trunk
>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to
>> include
>>>>>> in the
>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some
>> of
>>>>>> the
>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to
>>>>>> review
>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers
>> and
>>>>>> get
>>>>>>>>>>>>>>>>>>> committed.
>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of the
>>>>>> patches
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some
>> fixed
>>>>>>>>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly.
>> Because,
>>>>>> If we
>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it
>>>>>> might take
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of
>>>>>> August to
>>>>>>>>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared
>> the
>>>>>> list. I
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now,
>> into
>>>> a
>>>>>> word
>>>>>>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and
>> comment).
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
>>>> Added
>>>>>> two
>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
>>>>>> property).
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below)
>> and
>>>>>> provide
>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
>>>> process.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not
>>>>>> sure of
>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it
>>>>>> might make
>>>>>>>>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's
>> quite
>>>>>> ready
>>>>>>>>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of
>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for
>>>>>> Java 1.6)
>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for
>>>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
>>>> security
>>>>>> and
>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs
>> not
>>>>>> on your
>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need
>>>> to
>>>>>>>>>>>> include
>>>>>>>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have
>> the
>>>>>> review
>>>>>>>>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?
>>>> I
>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-
>> patch-available
>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from
>> currently
>>>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there
>> is
>>>>>>>>>>>> anything,
>>>>>>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the
>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and
>>>>>> commit.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
>>>>>> provide
>>>>>>>>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> ------------------------------
>>>> ------------------------------
>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
>>>>>> ridiculously
>>>>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord
>>>>>> action
>>>>>>>>>>>> rerun
>>>>>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when
>>>>>> reference
>>>>>>>>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids
>> for
>>>>>> action
>>>>>>>>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime
>>>> and
>>>>>>>>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns
>> incorrect
>>>>>> path for
>>>>>>>>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when
>>>>>> job is
>>>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume
>>>>>> more than
>>>>>>>>>>>>>>>>>> 90%
>>>>>>>>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
>>>>>> replication-enabled mysql
>>>>>>>>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for
>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains
>> spaces
>>>>>> can not
>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
>>>>>> hdfs-site and
>>>>>>>>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify
>> group,it
>>>>>> does`t
>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used
>> for
>>>>>> 'hive
>>>>>>>>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not
>>>>>> return
>>>>>>>>>>>> port
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
>>>> "Changing
>>>>>>>>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>>>>>>>>>>>> downstream
>>>>>>>>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation
>>>>>> for jobs
>>>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned
>>>> in
>>>>>>>>>>>>>>>>>> minioozie
>>>>>>>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be
>>>>>> used for
>>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle
>>>> job
>>>>>>>>>>>>>>>>>> querying
>>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>>>>>>>>>>>> directories
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
>>>> failed
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
>>>> connection
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
>>>>>> workflow
>>>>>>>>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create'
>>>>>> should
>>>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option
>> removing
>>>>>>>>>>>> completed
>>>>>>>>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed
>>>>>> state when
>>>>>>>>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in
>> the
>>>>>> logs
>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> ------------------------------
>>>> ------------------------------
>>>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
>>>>>> looked good
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there
>> are
>>>>>> things,
>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are
>> in
>>>>>>>>>>>> progress
>>>>>>>>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
>>>>>> (discontinued)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
>>>> upgrades.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We
>>>>>> will start
>>>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
>>>>>> Curator,
>>>>>>>>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have
>>>>>> done
>>>>>>>>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2231
>>>>> ,
>>>>>>>>>>>> curator
>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based
>>>> on
>>>>>> 1.1.0
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment
>> out a
>>>>>> few
>>>>>>>>>>>>>>>>>> lines
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
>>>> MiniHCatServer
>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation()
>>>>>> throws
>>>>>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not
>> be
>>>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no
>>>>>> database
>>>>>>>>>>>>>>>>>> named
>>>>>>>>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this
>> for
>>>>>> Hive
>>>>>>>>>>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(
>> true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>>>> resetHiveConfStaticVariables()
>>>>>>>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2
>> extends
>>>>>>>>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>> HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for
>>>> Hive
>>>>>> 0.14
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
>>>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
>>>>>> "scratch");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
>>>>>> Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
>>>>>> version also
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined
>>>>>> anywhere
>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
>>>> perhaps
>>>>>> we
>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop
>>>>>> did this
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine
>> with
>>>>>> doing
>>>>>>>>>>>>>>>>>> it in
>>>>>>>>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to
>>>> call
>>>>>> that
>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for
>> anything
>>>> we
>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not
>>>>>> currently
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action
>> JIRAs;
>>>>>> we
>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think
>>>>>> needs more
>>>>>>>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek
>>>> Bafna
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:
>> bafna.iitr@gmail.com
>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope
>> for
>>>>>> the
>>>>>>>>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek
>> Bafna
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for
>>>>>> Oozie
>>>>>>>>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
>>>> release.
>>>>>> Please
>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included
>> (or
>>>>>> any
>>>>>>>>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
>>>>>> submitted to
>>>>>>>>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if
>>>>>> required.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2036
>>>>>> )
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2488
>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>>>> jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2538
>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT
>>>> and
>>>>>> NN,
>>>>>>>>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics,
>> Tomcat
>>>>>> upgrade
>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on
>> inclusion/exclusion
>>>>>> of the
>>>>>>>>>>>>>>>>>> above
>>>>>>>>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also,
>>>>>> take some
>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
>>>> (depending
>>>>>> on the
>>>>>>>>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>> 
>>>>>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

Oh, right.  I forgot that Shwetha was going to help you with that.  That's
fine with me.

On Wed, Sep 21, 2016 at 3:22 AM, Abhishek Bafna <ba...@gmail.com>
wrote:

> Hello Robert,
>
> Yes. I do not have permissions to do it. I was planning to take Shwetha's
> help. I am fine with you doing it.
> Please let me know if there is any change in the plan.
>
> Thanks,
> Abhishek
> > On Sep 21, 2016, at 2:11 AM, Robert Kanter <rk...@cloudera.com> wrote:
> >
> > Abhishek,
> >
> > You don't have git permissions to create the release branch.  I can take
> > care of that on Thursday if you want.
> >
> >
> > - Robert
> >
> > On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <ba...@gmail.com>
> > wrote:
> >
> >> Hello All,
> >>
> >> I think we can create the new branch by Thursday (IST) and update
> version
> >> for new branch and master. If we have any important pending Jira's by
> then
> >> we will commit them to both new branch and master.
> >>
> >> I will start working on the next steps for release from release doc.
> >> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
> >> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
> >>
> >> Please do provide if you have any feedback or suggestion.
> >>
> >> Thanks,
> >> Abhishek
> >>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
> >> rohini.aditya@gmail.com> wrote:
> >>>
> >>> Sorry Abhishek for the long delay. Was on vacation and then busy
> catching
> >>> up with stuff. I did review a bunch of patches today which were marked
> >>> 4.3.0 and Patch Available apart from those which Robert or others have
> >>> already not commented on. Skipping those as they have better context.
> Tag
> >>> my name if there is something else that needs to be looked at.
> >>>
> >>> Regards,
> >>> Rohini
> >>>
> >>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <ba...@gmail.com>
> >>> wrote:
> >>>
> >>>> Hello Robert, Rohini, Purshotam, Jaydeep
> >>>>
> >>>> As we discussed earlier, will be bumping the priority of some Jira's
> to
> >>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try to
> >>>> complete as much possible out of that. (It would to good complete
> all.)
> >>>>
> >>>> @All, If I miss something, Please do priorities or send a message
> here.
> >>>>
> >>>> Thanks a lot for all the efforts and time we have put in last few
> weeks.
> >>>>
> >>>> Thanks,
> >>>> Abhishek
> >>>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <ba...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> It has been a long time since we started the scope discussion for the
> >>>> Oozie-4.3.0 release.
> >>>>> Since then a lot of patches got reviewed and committed and I would to
> >>>> thank each one you for contributing your efforts for that.
> >>>>>
> >>>>> I think we should move to the next stage and go into blocker only
> mode
> >>>> from next week (one week). After that 7-10 days for preparing and
> >>>> completing release condidate.
> >>>>>
> >>>>> We did plan to update some of the old tech and some of them are still
> >>>> pending for review. If can spend time into reviewing them, it would be
> >>>> good, else we might have to move without them.
> >>>>>
> >>>>> There are also some good to have patches available for review and
> >>>> commit, I will bump up the priority for them. If I miss something,
> >> please
> >>>> feel free to do the same.
> >>>>>
> >>>>> Regards,
> >>>>> Abhishek
> >>>>>
> >>>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>> Hi All,
> >>>>>>
> >>>>>> Can we please look into these upgrade related bugs?
> >>>>>>
> >>>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
> >>>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
> >>>>>> OpenJPA              https://issues.apache.org/
> jira/browse/OOZIE-2488
> >>>>>> Curator              https://issues.apache.org/
> jira/browse/OOZIE-2231
> >>>>>> Hive                 https://issues.apache.org/
> jira/browse/OOZIE-2613
> >>>>>>
> >>>>>> Are the current patches good and feedback if there is any.
> >>>>>>
> >>>>>> Also, If we can do some reviews collectively.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Abhishek
> >>>>>>
> >>>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>> Hello All,
> >>>>>>>
> >>>>>>> How is it looking now?
> >>>>>>>
> >>>>>>> I think we should move to blocker and critical patch mode. If
> >>>> required, we can take couple of days extra there.
> >>>>>>>
> >>>>>>> ~Abhishek
> >>>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <
> bafna.iitr@gmail.com>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>> Sure Robert. We can use some more time.
> >>>>>>>>
> >>>>>>>> ~Abhishek
> >>>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rkanter@cloudera.com
> >
> >>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Do you think we could wait one more week?  I've been really
> >> hammered
> >>>> with
> >>>>>>>>> some other work and there's more that I'd like to review here
> that
> >> I
> >>>>>>>>> haven't been able to get to.
> >>>>>>>>>
> >>>>>>>>> thanks
> >>>>>>>>> - Robert
> >>>>>>>>>
> >>>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
> >>>> bafna.iitr@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Sure. I guess we can one extra week and complete some more work.
> >>>>>>>>>> Thanks everyone for responding.
> >>>>>>>>>>
> >>>>>>>>>> ~Abhishek
> >>>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
> >>>> purushah@yahoo-inc.com.INVALID>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Yes. I also have some pending patches.
> >>>>>>>>>>>
> >>>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
> >>>>>>>>>> rkanter@cloudera.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Ya, I could use some more time as well.  I have some deadlines
> >>>> early next
> >>>>>>>>>>> week, which has kept me from doing more reviews.
> >>>>>>>>>>>
> >>>>>>>>>>> thanks
> >>>>>>>>>>> - Robert
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
> >>>>>>>>>>> jaydeepmailv@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Let see if we can wait for another week. So we can merge few
> >> more
> >>>>>>>>>> important
> >>>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
> >> merged.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regards,
> >>>>>>>>>>>> Jaydeep
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
> >>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> As previously discussed, we will be moving with whatever
> >> patches
> >>>> can be
> >>>>>>>>>>>>> reviewed/merged till end of this week.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> But there are some which needs to be reviewed and merged,
> >> mainly
> >>>>>>>>>>>> regarding
> >>>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I
> think
> >> we
> >>>>>>>>>> should
> >>>>>>>>>>>>> review OOZIE-2036 and merged it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> ~Abhishek
> >>>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
> >>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".
> Patch
> >>>>>>>>>>>> Available
> >>>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
> >>>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next week
> to
> >>>> see if
> >>>>>>>>>>>> any
> >>>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a few
> >>>> which are
> >>>>>>>>>>>>>> unassigned.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If there is any jira, someone would like to include in 4.3.0
> >>>> please
> >>>>>>>>>>>>> update
> >>>>>>>>>>>>>> the jira.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>> rohini.aditya@gmail.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Unset the open ones without sending emails. Will update the
> >>>> resolved,
> >>>>>>>>>>>>>>> closed and Patch Available ones sending emails.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
> >>>> rkanter@cloudera.com
> >>>>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree,
> >> we
> >>>> need
> >>>>>>>>>> to
> >>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch
> >>>> Available
> >>>>>>>>>>>>> JIRAs.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.
> >>>> We can
> >>>>>>>>>>>>> discuss
> >>>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
> >>>> OOZIE-2625 about
> >>>>>>>>>>>>> why.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
> >>>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
> >>>>>>>>>>>>> 20status%20DESC&startIndex=50
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset
> fix
> >>>> version
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of them
> >>>> being
> >>>>>>>>>>>> worked
> >>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and close
> >>>> many of
> >>>>>>>>>>>> these
> >>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
> renaming
> >>>> it to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> release number.
> >>>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we go
> >> with
> >>>>>>>>>>>> unsetting
> >>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
> >>>> create a
> >>>>>>>>>>>> 4.4.0
> >>>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras from
> >>>> 4.3.0.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I think it would be better to send out email for the
> bulk
> >>>> change.
> >>>>>>>>>>>> If
> >>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
> >> respond.
> >>>> We do
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come
> >>>> back with
> >>>>>>>>>>>>>>>>> feedback
> >>>>>>>>>>>>>>>>>> on pulling back to release.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
> >>>>>>>>>>>> rkanter@cloudera.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please
> remember
> >>>> to tell
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> send out an email for every change :)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
> >>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
> >> we're
> >>>> on
> >>>>>>>>>>>>>>>> "master"
> >>>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a
> little
> >>>>>>>>>>>> confusing.
> >>>>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
> >>>> renaming it
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any
> >>>> current
> >>>>>>>>>>>>>>>> "trunk"
> >>>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that
> >> the
> >>>> next
> >>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can
> >>>> always
> >>>>>>>>>>>>>>>> rename it
> >>>>>>>>>>>>>>>>>>>> later.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
> >>>> suppose we
> >>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make
> things
> >>>> official
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send an
> >>>> email?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I
> think
> >>>>>>>>>> Rohini's
> >>>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next
> step.
> >>>> You're
> >>>>>>>>>>>>>>>>>>> probably
> >>>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of
> doing
> >>>> each
> >>>>>>>>>> JIRA
> >>>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
> >>>> reviews this
> >>>>>>>>>>>>>>>> week,
> >>>>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>>>> actually already started.  I should have some time to
> >>>> continue
> >>>>>>>>>>>>>>>> doing
> >>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow
> >> about
> >>>>>>>>>>>> getting
> >>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>> done :(
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
> >>>> Satish was
> >>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Abhishek,
> >>>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as
> >>>> 4.3.0
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will
> help
> >>>> filter
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I
> can
> >>>> pick up
> >>>>>>>>>>>>>>>>> 10-15
> >>>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to
> >> get
> >>>>>>>>>>>>>>>> majority of
> >>>>>>>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can
> >>>> then use
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> 3rd
> >>>>>>>>>>>>>>>>>>>>> week
> >>>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
> >> release
> >>>>>>>>>>>>>>>> candidate.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Robert,
> >>>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk
> >>>> version
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>> release numbers when committing from the beginning
> >> like
> >>>> all
> >>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>> projects
> >>>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release
> >>>> version
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> every
> >>>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete
> >>>> the trunk
> >>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> https://docs.google.com/document/d/
> >>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to
> include
> >>>> in the
> >>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>> In the
> >>>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some
> of
> >>>> the
> >>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to
> >>>> review
> >>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>> patches.
> >>>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers
> and
> >>>> get
> >>>>>>>>>>>>>>>>> committed.
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of the
> >>>> patches
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> provide
> >>>>>>>>>>>>>>>>>>>>>>> their feedback.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some
> fixed
> >>>>>>>>>>>>>>>> (approximate)
> >>>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly.
> Because,
> >>>> If we
> >>>>>>>>>>>> wait
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> all the
> >>>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it
> >>>> might take
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> lot
> >>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of
> >>>> August to
> >>>>>>>>>>>>>>>>>>> complete
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared
> the
> >>>> list. I
> >>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>> put,
> >>>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now,
> into
> >> a
> >>>> word
> >>>>>>>>>>>>>>>>>>> document.
> >>>>>>>>>>>>>>>>>>>>> (For
> >>>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and
> comment).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
> >> Added
> >>>> two
> >>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>> Jira
> >>>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
> >>>> property).
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below)
> and
> >>>> provide
> >>>>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>>>> comments.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
> >> process.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Scope Document:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> https://docs.google.com/document/d/
> >>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> >>>>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not
> >>>> sure of
> >>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>> OOZIE-2306
> >>>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it
> >>>> might make
> >>>>>>>>>>>>>>>>>>> sense to
> >>>>>>>>>>>>>>>>>>>>>>> let it
> >>>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's
> quite
> >>>> ready
> >>>>>>>>>>>>>>>> yet.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
> >>>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of
> >>>> Oozie)
> >>>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for
> >>>> Java 1.6)
> >>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for
> >>>> Oozie)
> >>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
> >> security
> >>>> and
> >>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>> fixes)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs
> not
> >>>> on your
> >>>>>>>>>>>>>>>>> list
> >>>>>>>>>>>>>>>>>>>>>>> (including
> >>>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need
> >> to
> >>>>>>>>>> include
> >>>>>>>>>>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have
> the
> >>>> review
> >>>>>>>>>>>>>>>>>>>>> bandwidth),
> >>>>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?
> >> I
> >>>>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>>>> see
> >>>>>>>>>>>>>>>>>>>>> 97
> >>>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-
> patch-available
> >>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> >>>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
> >>>>>>>>>>>>>>>> fixes/improvements,
> >>>>>>>>>>>>>>>>>>> given
> >>>>>>>>>>>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from
> currently
> >>>>>>>>>>>>>>>> available
> >>>>>>>>>>>>>>>>>>>>> patches.
> >>>>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there
> is
> >>>>>>>>>> anything,
> >>>>>>>>>>>>>>>>>>> please
> >>>>>>>>>>>>>>>>>>>>>>> reply
> >>>>>>>>>>>>>>>>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the
> >>>> feedback.
> >>>>>>>>>>>>>>>>>>> Important
> >>>>>>>>>>>>>>>>>>>>>>> is,
> >>>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and
> >>>> commit.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
> >>>> provide
> >>>>>>>>>>>>>>>>> feedback.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> ------------------------------
> >> ------------------------------
> >>>>>>>>>>>>> ---------------
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
> >>>> ridiculously
> >>>>>>>>>>>>>>>> slow
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>>>> cases
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord
> >>>> action
> >>>>>>>>>> rerun
> >>>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when
> >>>> reference
> >>>>>>>>>>>>>>>>>>> variable
> >>>>>>>>>>>>>>>>>>>>> name
> >>>>>>>>>>>>>>>>>>>>>>>>>> containing dot
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids
> for
> >>>> action
> >>>>>>>>>>>>>>>>>>> failures
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime
> >> and
> >>>>>>>>>>>>>>>>>>> endCreatedTime
> >>>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns
> incorrect
> >>>> path for
> >>>>>>>>>>>>>>>> jar
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when
> >>>> job is
> >>>>>>>>>>>>>>>>> running
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>>>> than expected time
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume
> >>>> more than
> >>>>>>>>>>>>>>>> 90%
> >>>>>>>>>>>>>>>>>>> cpu
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
> >>>> replication-enabled mysql
> >>>>>>>>>>>>>>>>> urls
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for
> >>>> logging
> >>>>>>>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains
> spaces
> >>>> can not
> >>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>> handled
> >>>>>>>>>>>>>>>>>>>>>>>>>> rightly
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
> >>>> hdfs-site and
> >>>>>>>>>>>>>>>>>>>>> mapred-site
> >>>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify
> group,it
> >>>> does`t
> >>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used
> for
> >>>> 'hive
> >>>>>>>>>>>>>>>> log4j
> >>>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not
> >>>> return
> >>>>>>>>>> port
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>>>> -servers
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
> >> "Changing
> >>>>>>>>>>>>>>>>>>>>> endtime/pausetime
> >>>>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
> >>>>>>>>>> downstream
> >>>>>>>>>>>>>>>>>>>>> components
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation
> >>>> for jobs
> >>>>>>>>>>>>>>>> API
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned
> >> in
> >>>>>>>>>>>>>>>> minioozie
> >>>>>>>>>>>>>>>>>>> doc
> >>>>>>>>>>>>>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be
> >>>> used for
> >>>>>>>>>>>>>>>> error
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>> audit
> >>>>>>>>>>>>>>>>>>>>>>>>>> logs
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle
> >> job
> >>>>>>>>>>>>>>>> querying
> >>>>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
> >>>>>>>>>>>>>>>> directories
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> /tmp
> >>>>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
> >> failed
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
> >> connection
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
> >>>> workflow
> >>>>>>>>>>>>>>>> action
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create'
> >>>> should
> >>>>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>>>> uploaded
> >>>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option
> removing
> >>>>>>>>>> completed
> >>>>>>>>>>>>>>>>>>> output
> >>>>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed
> >>>> state when
> >>>>>>>>>>>>>>>>> kill
> >>>>>>>>>>>>>>>>>>>>>>> control
> >>>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
> >>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in
> the
> >>>> logs
> >>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>> logging
> >>>>>>>>>>>>>>>>>>>>>>>>>> command arguments
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> ------------------------------
> >> ------------------------------
> >>>>>>>>>>>>> ---------------
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
> >>>> looked good
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> me
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there
> are
> >>>> things,
> >>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>> needs
> >>>>>>>>>>>>>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are
> in
> >>>>>>>>>> progress
> >>>>>>>>>>>>>>>>>>> status.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
> >>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
> >>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
> >>>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
> >>>> (discontinued)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
> >> upgrades.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We
> >>>> will start
> >>>>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
> >>>> Curator,
> >>>>>>>>>>>>>>>>>>> HttpClient,
> >>>>>>>>>>>>>>>>>>>>> Pig
> >>>>>>>>>>>>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have
> >>>> done
> >>>>>>>>>>>>>>>>>>> internally as
> >>>>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2231
> >>> ,
> >>>>>>>>>> curator
> >>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> >>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based
> >> on
> >>>> 1.1.0
> >>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> over
> >>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>> year
> >>>>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment
> out a
> >>>> few
> >>>>>>>>>>>>>>>> lines
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
> >>>> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
> >>>> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
> >>>> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
> >>>> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
> >> MiniHCatServer
> >>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation()
> >>>> throws
> >>>>>>>>>>>>>>>>>>> Exception {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not
> be
> >>>>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>> local
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no
> >>>> database
> >>>>>>>>>>>>>>>> named
> >>>>>>>>>>>>>>>>>>>>> default)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
> >>>> createDefaultDB");
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this
> for
> >>>> Hive
> >>>>>>>>>>>>>>>> 0.14
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
> >>>> createDefaultDB");
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(
> true);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
> >> resetHiveConfStaticVariables()
> >>>>>>>>>>>>>>>> throws
> >>>>>>>>>>>>>>>>>>>>>>> Exception
> >>>>>>>>>>>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
> >>>> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
> >> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
> >>>> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
> >>>> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2
> extends
> >>>>>>>>>>>>>>>>>>>>> AbstractHiveService
> >>>>>>>>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> >>>>>>>>>>>>>>>>>>>>>>> getHost());
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
> >>>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
> >>>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
> HIVE_SERVER2_THRIFT_HTTP_PORT,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
> >>>> resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for
> >> Hive
> >>>> 0.14
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
> >>>> resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
> >>>> "scratch");
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
> >>>> Ranganathan <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
> >>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
> >>>> version also
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> 1.x.
> >>>>>>>>>>>>>>>>>>>>>>> 0.13
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> >>>>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>>>> <mailto:
> >>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined
> >>>> anywhere
> >>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
> >> perhaps
> >>>> we
> >>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> wait
> >>>>>>>>>>>>>>>>>>>>>>>>>> until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop
> >>>> did this
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>> minor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine
> with
> >>>> doing
> >>>>>>>>>>>>>>>> it in
> >>>>>>>>>>>>>>>>>>> 4.3
> >>>>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to
> >> call
> >>>> that
> >>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for
> anything
> >> we
> >>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>> leave
> >>>>>>>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not
> >>>> currently
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> trunk
> >>>>>>>>>>>>>>>>>>>>>>>>>> anyway.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action
> JIRAs;
> >>>> we
> >>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>>>> sure
> >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think
> >>>> needs more
> >>>>>>>>>>>>>>>>>>> time to
> >>>>>>>>>>>>>>>>>>>>>>> bake
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek
> >> Bafna
> >>>> <
> >>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:
> bafna.iitr@gmail.com
> >>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope
> for
> >>>> the
> >>>>>>>>>>>>>>>> release?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek
> Bafna
> >> <
> >>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for
> >>>> Oozie
> >>>>>>>>>>>>>>>> 4.3.0.
> >>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>>>> thread
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
> >> release.
> >>>> Please
> >>>>>>>>>>>>>>>>>>> provide
> >>>>>>>>>>>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included
> (or
> >>>> any
> >>>>>>>>>>>>>>>>>>> exclusions)
> >>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
> >>>> submitted to
> >>>>>>>>>>>>>>>>> trunk.
> >>>>>>>>>>>>>>>>>>>>> Based
> >>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if
> >>>> required.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>> jira/browse/OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2036
> >>>> )
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>> jira/browse/OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2488
> >>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>> jira/browse/OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2231>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >>>> jira/browse/OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2538
> >>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
> >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2602>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT
> >> and
> >>>> NN,
> >>>>>>>>>>>>>>>>> Database
> >>>>>>>>>>>>>>>>>>>>>>> schema
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics,
> Tomcat
> >>>> upgrade
> >>>>>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on
> inclusion/exclusion
> >>>> of the
> >>>>>>>>>>>>>>>> above
> >>>>>>>>>>>>>>>>>>>>> points
> >>>>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also,
> >>>> take some
> >>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
> >> (depending
> >>>> on the
> >>>>>>>>>>>>>>>>>>>>>>> availability
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>
> >>>>
> >>>>
> >>
> >>
>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hello Robert,

Yes. I do not have permissions to do it. I was planning to take Shwetha's help. I am fine with you doing it.
Please let me know if there is any change in the plan.

Thanks,
Abhishek
> On Sep 21, 2016, at 2:11 AM, Robert Kanter <rk...@cloudera.com> wrote:
> 
> Abhishek,
> 
> You don't have git permissions to create the release branch.  I can take
> care of that on Thursday if you want.
> 
> 
> - Robert
> 
> On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> 
>> Hello All,
>> 
>> I think we can create the new branch by Thursday (IST) and update version
>> for new branch and master. If we have any important pending Jira's by then
>> we will commit them to both new branch and master.
>> 
>> I will start working on the next steps for release from release doc.
>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
>> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
>> 
>> Please do provide if you have any feedback or suggestion.
>> 
>> Thanks,
>> Abhishek
>>> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
>> rohini.aditya@gmail.com> wrote:
>>> 
>>> Sorry Abhishek for the long delay. Was on vacation and then busy catching
>>> up with stuff. I did review a bunch of patches today which were marked
>>> 4.3.0 and Patch Available apart from those which Robert or others have
>>> already not commented on. Skipping those as they have better context. Tag
>>> my name if there is something else that needs to be looked at.
>>> 
>>> Regards,
>>> Rohini
>>> 
>>> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <ba...@gmail.com>
>>> wrote:
>>> 
>>>> Hello Robert, Rohini, Purshotam, Jaydeep
>>>> 
>>>> As we discussed earlier, will be bumping the priority of some Jira's to
>>>> 'blocker' for to be included into Oozie 4.3.0 release. We will try to
>>>> complete as much possible out of that. (It would to good complete all.)
>>>> 
>>>> @All, If I miss something, Please do priorities or send a message here.
>>>> 
>>>> Thanks a lot for all the efforts and time we have put in last few weeks.
>>>> 
>>>> Thanks,
>>>> Abhishek
>>>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> It has been a long time since we started the scope discussion for the
>>>> Oozie-4.3.0 release.
>>>>> Since then a lot of patches got reviewed and committed and I would to
>>>> thank each one you for contributing your efforts for that.
>>>>> 
>>>>> I think we should move to the next stage and go into blocker only mode
>>>> from next week (one week). After that 7-10 days for preparing and
>>>> completing release condidate.
>>>>> 
>>>>> We did plan to update some of the old tech and some of them are still
>>>> pending for review. If can spend time into reviewing them, it would be
>>>> good, else we might have to move without them.
>>>>> 
>>>>> There are also some good to have patches available for review and
>>>> commit, I will bump up the priority for them. If I miss something,
>> please
>>>> feel free to do the same.
>>>>> 
>>>>> Regards,
>>>>> Abhishek
>>>>> 
>>>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> Can we please look into these upgrade related bugs?
>>>>>> 
>>>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
>>>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
>>>>>> OpenJPA              https://issues.apache.org/jira/browse/OOZIE-2488
>>>>>> Curator              https://issues.apache.org/jira/browse/OOZIE-2231
>>>>>> Hive                 https://issues.apache.org/jira/browse/OOZIE-2613
>>>>>> 
>>>>>> Are the current patches good and feedback if there is any.
>>>>>> 
>>>>>> Also, If we can do some reviews collectively.
>>>>>> 
>>>>>> Thanks,
>>>>>> Abhishek
>>>>>> 
>>>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>>>>> 
>>>>>>> Hello All,
>>>>>>> 
>>>>>>> How is it looking now?
>>>>>>> 
>>>>>>> I think we should move to blocker and critical patch mode. If
>>>> required, we can take couple of days extra there.
>>>>>>> 
>>>>>>> ~Abhishek
>>>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>>>>>> 
>>>>>>>> Sure Robert. We can use some more time.
>>>>>>>> 
>>>>>>>> ~Abhishek
>>>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com>
>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Do you think we could wait one more week?  I've been really
>> hammered
>>>> with
>>>>>>>>> some other work and there's more that I'd like to review here that
>> I
>>>>>>>>> haven't been able to get to.
>>>>>>>>> 
>>>>>>>>> thanks
>>>>>>>>> - Robert
>>>>>>>>> 
>>>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
>>>> bafna.iitr@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Sure. I guess we can one extra week and complete some more work.
>>>>>>>>>> Thanks everyone for responding.
>>>>>>>>>> 
>>>>>>>>>> ~Abhishek
>>>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
>>>> purushah@yahoo-inc.com.INVALID>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Yes. I also have some pending patches.
>>>>>>>>>>> 
>>>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>>>>>>> rkanter@cloudera.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Ya, I could use some more time as well.  I have some deadlines
>>>> early next
>>>>>>>>>>> week, which has kept me from doing more reviews.
>>>>>>>>>>> 
>>>>>>>>>>> thanks
>>>>>>>>>>> - Robert
>>>>>>>>>>> 
>>>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Let see if we can wait for another week. So we can merge few
>> more
>>>>>>>>>> important
>>>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
>> merged.
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Jaydeep
>>>>>>>>>>>> 
>>>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> As previously discussed, we will be moving with whatever
>> patches
>>>> can be
>>>>>>>>>>>>> reviewed/merged till end of this week.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> But there are some which needs to be reviewed and merged,
>> mainly
>>>>>>>>>>>> regarding
>>>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I think
>> we
>>>>>>>>>> should
>>>>>>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
>>>>>>>>>>>> Available
>>>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next week to
>>>> see if
>>>>>>>>>>>> any
>>>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a few
>>>> which are
>>>>>>>>>>>>>> unassigned.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> If there is any jira, someone would like to include in 4.3.0
>>>> please
>>>>>>>>>>>>> update
>>>>>>>>>>>>>> the jira.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Unset the open ones without sending emails. Will update the
>>>> resolved,
>>>>>>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
>>>> rkanter@cloudera.com
>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree,
>> we
>>>> need
>>>>>>>>>> to
>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch
>>>> Available
>>>>>>>>>>>>> JIRAs.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.
>>>> We can
>>>>>>>>>>>>> discuss
>>>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
>>>> OOZIE-2625 about
>>>>>>>>>>>>> why.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset fix
>>>> version
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of them
>>>> being
>>>>>>>>>>>> worked
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and close
>>>> many of
>>>>>>>>>>>> these
>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming
>>>> it to
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> release number.
>>>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we go
>> with
>>>>>>>>>>>> unsetting
>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
>>>> create a
>>>>>>>>>>>> 4.4.0
>>>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras from
>>>> 4.3.0.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I think it would be better to send out email for the bulk
>>>> change.
>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
>> respond.
>>>> We do
>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come
>>>> back with
>>>>>>>>>>>>>>>>> feedback
>>>>>>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember
>>>> to tell
>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
>> we're
>>>> on
>>>>>>>>>>>>>>>> "master"
>>>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>>>>>>>>>>>> confusing.
>>>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>>>> renaming it
>>>>>>>>>> to
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any
>>>> current
>>>>>>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that
>> the
>>>> next
>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can
>>>> always
>>>>>>>>>>>>>>>> rename it
>>>>>>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
>>>> suppose we
>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make things
>>>> official
>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send an
>>>> email?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
>>>>>>>>>> Rohini's
>>>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.
>>>> You're
>>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of doing
>>>> each
>>>>>>>>>> JIRA
>>>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
>>>> reviews this
>>>>>>>>>>>>>>>> week,
>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>> actually already started.  I should have some time to
>>>> continue
>>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow
>> about
>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
>>>> Satish was
>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as
>>>> 4.3.0
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help
>>>> filter
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I can
>>>> pick up
>>>>>>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to
>> get
>>>>>>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can
>>>> then use
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
>> release
>>>>>>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk
>>>> version
>>>>>>>>>> and
>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>> release numbers when committing from the beginning
>> like
>>>> all
>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release
>>>> version
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete
>>>> the trunk
>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to include
>>>> in the
>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of
>>>> the
>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to
>>>> review
>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and
>>>> get
>>>>>>>>>>>>>>>>> committed.
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of the
>>>> patches
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly. Because,
>>>> If we
>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it
>>>> might take
>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of
>>>> August to
>>>>>>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared the
>>>> list. I
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now, into
>> a
>>>> word
>>>>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
>> Added
>>>> two
>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
>>>> property).
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below) and
>>>> provide
>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
>> process.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> https://docs.google.com/document/d/
>>>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not
>>>> sure of
>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it
>>>> might make
>>>>>>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite
>>>> ready
>>>>>>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of
>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for
>>>> Java 1.6)
>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for
>>>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
>> security
>>>> and
>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not
>>>> on your
>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need
>> to
>>>>>>>>>> include
>>>>>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the
>>>> review
>>>>>>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?
>> I
>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
>>>>>>>>>> anything,
>>>>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the
>>>> feedback.
>>>>>>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and
>>>> commit.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
>>>> provide
>>>>>>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> ------------------------------
>> ------------------------------
>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
>>>> ridiculously
>>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord
>>>> action
>>>>>>>>>> rerun
>>>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when
>>>> reference
>>>>>>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for
>>>> action
>>>>>>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime
>> and
>>>>>>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect
>>>> path for
>>>>>>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when
>>>> job is
>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume
>>>> more than
>>>>>>>>>>>>>>>> 90%
>>>>>>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
>>>> replication-enabled mysql
>>>>>>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for
>>>> logging
>>>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces
>>>> can not
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
>>>> hdfs-site and
>>>>>>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it
>>>> does`t
>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for
>>>> 'hive
>>>>>>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not
>>>> return
>>>>>>>>>> port
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
>> "Changing
>>>>>>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>>>>>>>>>> downstream
>>>>>>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation
>>>> for jobs
>>>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned
>> in
>>>>>>>>>>>>>>>> minioozie
>>>>>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be
>>>> used for
>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle
>> job
>>>>>>>>>>>>>>>> querying
>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>>>>>>>>>> directories
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
>> failed
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
>> connection
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
>>>> workflow
>>>>>>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create'
>>>> should
>>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
>>>>>>>>>> completed
>>>>>>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed
>>>> state when
>>>>>>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the
>>>> logs
>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> ------------------------------
>> ------------------------------
>>>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
>>>> looked good
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there are
>>>> things,
>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
>>>>>>>>>> progress
>>>>>>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
>>>> (discontinued)
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
>> upgrades.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We
>>>> will start
>>>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
>>>> Curator,
>>>>>>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have
>>>> done
>>>>>>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231
>>> ,
>>>>>>>>>> curator
>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based
>> on
>>>> 1.1.0
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a
>>>> few
>>>>>>>>>>>>>>>> lines
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>>>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
>> MiniHCatServer
>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation()
>>>> throws
>>>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not be
>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no
>>>> database
>>>>>>>>>>>>>>>> named
>>>>>>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for
>>>> Hive
>>>>>>>>>>>>>>>> 0.14
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>>>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
>> resetHiveConfStaticVariables()
>>>>>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>>>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for
>> Hive
>>>> 0.14
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
>>>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
>>>> "scratch");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
>>>> Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
>>>> version also
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined
>>>> anywhere
>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
>> perhaps
>>>> we
>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop
>>>> did this
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with
>>>> doing
>>>>>>>>>>>>>>>> it in
>>>>>>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to
>> call
>>>> that
>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything
>> we
>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not
>>>> currently
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs;
>>>> we
>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think
>>>> needs more
>>>>>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek
>> Bafna
>>>> <
>>>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:bafna.iitr@gmail.com
>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for
>>>> the
>>>>>>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna
>> <
>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for
>>>> Oozie
>>>>>>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
>> release.
>>>> Please
>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or
>>>> any
>>>>>>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
>>>> submitted to
>>>>>>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if
>>>> required.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>> jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036
>>>> )
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>> jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488
>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>> jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>>>> jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538
>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT
>> and
>>>> NN,
>>>>>>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat
>>>> upgrade
>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion
>>>> of the
>>>>>>>>>>>>>>>> above
>>>>>>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also,
>>>> take some
>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
>> (depending
>>>> on the
>>>>>>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>> 
>>>> 
>>>> 
>> 
>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

Abhishek,

You don't have git permissions to create the release branch.  I can take
care of that on Thursday if you want.


- Robert

On Tue, Sep 20, 2016 at 1:30 PM, Abhishek Bafna <ba...@gmail.com>
wrote:

> Hello All,
>
> I think we can create the new branch by Thursday (IST) and update version
> for new branch and master. If we have any important pending Jira's by then
> we will commit them to both new branch and master.
>
> I will start working on the next steps for release from release doc.
> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <
> https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>
>
> Please do provide if you have any feedback or suggestion.
>
> Thanks,
> Abhishek
> > On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <
> rohini.aditya@gmail.com> wrote:
> >
> > Sorry Abhishek for the long delay. Was on vacation and then busy catching
> > up with stuff. I did review a bunch of patches today which were marked
> > 4.3.0 and Patch Available apart from those which Robert or others have
> > already not commented on. Skipping those as they have better context. Tag
> > my name if there is something else that needs to be looked at.
> >
> > Regards,
> > Rohini
> >
> > On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <ba...@gmail.com>
> > wrote:
> >
> >> Hello Robert, Rohini, Purshotam, Jaydeep
> >>
> >> As we discussed earlier, will be bumping the priority of some Jira's to
> >> 'blocker' for to be included into Oozie 4.3.0 release. We will try to
> >> complete as much possible out of that. (It would to good complete all.)
> >>
> >> @All, If I miss something, Please do priorities or send a message here.
> >>
> >> Thanks a lot for all the efforts and time we have put in last few weeks.
> >>
> >> Thanks,
> >> Abhishek
> >>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <ba...@gmail.com>
> >> wrote:
> >>>
> >>> Hi All,
> >>>
> >>> It has been a long time since we started the scope discussion for the
> >> Oozie-4.3.0 release.
> >>> Since then a lot of patches got reviewed and committed and I would to
> >> thank each one you for contributing your efforts for that.
> >>>
> >>> I think we should move to the next stage and go into blocker only mode
> >> from next week (one week). After that 7-10 days for preparing and
> >> completing release condidate.
> >>>
> >>> We did plan to update some of the old tech and some of them are still
> >> pending for review. If can spend time into reviewing them, it would be
> >> good, else we might have to move without them.
> >>>
> >>> There are also some good to have patches available for review and
> >> commit, I will bump up the priority for them. If I miss something,
> please
> >> feel free to do the same.
> >>>
> >>> Regards,
> >>> Abhishek
> >>>
> >>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com>
> >> wrote:
> >>>>
> >>>> Hi All,
> >>>>
> >>>> Can we please look into these upgrade related bugs?
> >>>>
> >>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
> >>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
> >>>> OpenJPA              https://issues.apache.org/jira/browse/OOZIE-2488
> >>>> Curator              https://issues.apache.org/jira/browse/OOZIE-2231
> >>>> Hive                 https://issues.apache.org/jira/browse/OOZIE-2613
> >>>>
> >>>> Are the current patches good and feedback if there is any.
> >>>>
> >>>> Also, If we can do some reviews collectively.
> >>>>
> >>>> Thanks,
> >>>> Abhishek
> >>>>
> >>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com>
> >> wrote:
> >>>>>
> >>>>> Hello All,
> >>>>>
> >>>>> How is it looking now?
> >>>>>
> >>>>> I think we should move to blocker and critical patch mode. If
> >> required, we can take couple of days extra there.
> >>>>>
> >>>>> ~Abhishek
> >>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <ba...@gmail.com>
> >> wrote:
> >>>>>>
> >>>>>> Sure Robert. We can use some more time.
> >>>>>>
> >>>>>> ~Abhishek
> >>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com>
> >> wrote:
> >>>>>>>
> >>>>>>> Do you think we could wait one more week?  I've been really
> hammered
> >> with
> >>>>>>> some other work and there's more that I'd like to review here that
> I
> >>>>>>> haven't been able to get to.
> >>>>>>>
> >>>>>>> thanks
> >>>>>>> - Robert
> >>>>>>>
> >>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
> >> bafna.iitr@gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Sure. I guess we can one extra week and complete some more work.
> >>>>>>>> Thanks everyone for responding.
> >>>>>>>>
> >>>>>>>> ~Abhishek
> >>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
> >> purushah@yahoo-inc.com.INVALID>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Yes. I also have some pending patches.
> >>>>>>>>>
> >>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
> >>>>>>>> rkanter@cloudera.com> wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Ya, I could use some more time as well.  I have some deadlines
> >> early next
> >>>>>>>>> week, which has kept me from doing more reviews.
> >>>>>>>>>
> >>>>>>>>> thanks
> >>>>>>>>> - Robert
> >>>>>>>>>
> >>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
> >>>>>>>>> jaydeepmailv@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> Let see if we can wait for another week. So we can merge few
> more
> >>>>>>>> important
> >>>>>>>>>> jiras.  Few of them I personally want to get reviewed and
> merged.
> >>>>>>>>>>
> >>>>>>>>>> Regards,
> >>>>>>>>>> Jaydeep
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
> >> bafna.iitr@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> As previously discussed, we will be moving with whatever
> patches
> >> can be
> >>>>>>>>>>> reviewed/merged till end of this week.
> >>>>>>>>>>>
> >>>>>>>>>>> But there are some which needs to be reviewed and merged,
> mainly
> >>>>>>>>>> regarding
> >>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I think
> we
> >>>>>>>> should
> >>>>>>>>>>> review OOZIE-2036 and merged it.
> >>>>>>>>>>>
> >>>>>>>>>>> ~Abhishek
> >>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
> >>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
> >>>>>>>>>> Available
> >>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
> >>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
> >>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
> >>>>>>>>>>>>
> >>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next week to
> >> see if
> >>>>>>>>>> any
> >>>>>>>>>>>> pending ones needs to be pushed out. There are quite a few
> >> which are
> >>>>>>>>>>>> unassigned.
> >>>>>>>>>>>>
> >>>>>>>>>>>> If there is any jira, someone would like to include in 4.3.0
> >> please
> >>>>>>>>>>> update
> >>>>>>>>>>>> the jira.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regards,
> >>>>>>>>>>>> Rohini
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
> >>>>>>>>>>> rohini.aditya@gmail.com
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Unset the open ones without sending emails. Will update the
> >> resolved,
> >>>>>>>>>>>>> closed and Patch Available ones sending emails.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
> >> rkanter@cloudera.com
> >>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree,
> we
> >> need
> >>>>>>>> to
> >>>>>>>>>>> put
> >>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch
> >> Available
> >>>>>>>>>>> JIRAs.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.
> >> We can
> >>>>>>>>>>> discuss
> >>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
> >> OOZIE-2625 about
> >>>>>>>>>>> why.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
> >>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
> >>>>>>>>>>> 20status%20DESC&startIndex=50
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset fix
> >> version
> >>>>>>>>>> on
> >>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>> or mark them for next release? Don't see majority of them
> >> being
> >>>>>>>>>> worked
> >>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>> even for next release. We need to do bug scrub and close
> >> many of
> >>>>>>>>>> these
> >>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming
> >> it to
> >>>>>>>>>> the
> >>>>>>>>>>>>>>> release number.
> >>>>>>>>>>>>>>> There is some amount of relabeling either way. If we go
> with
> >>>>>>>>>> unsetting
> >>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
> >> create a
> >>>>>>>>>> 4.4.0
> >>>>>>>>>>>>>>> yesterday to move some of the patch available jiras from
> >> 4.3.0.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I think it would be better to send out email for the bulk
> >> change.
> >>>>>>>>>> If
> >>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can
> respond.
> >> We do
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come
> >> back with
> >>>>>>>>>>>>>>> feedback
> >>>>>>>>>>>>>>>> on pulling back to release.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
> >>>>>>>>>> rkanter@cloudera.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember
> >> to tell
> >>>>>>>>>> it
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> send out an email for every change :)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
> >>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because
> we're
> >> on
> >>>>>>>>>>>>>> "master"
> >>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
> >>>>>>>>>> confusing.
> >>>>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
> >> renaming it
> >>>>>>>> to
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any
> >> current
> >>>>>>>>>>>>>> "trunk"
> >>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that
> the
> >> next
> >>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can
> >> always
> >>>>>>>>>>>>>> rename it
> >>>>>>>>>>>>>>>>>> later.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
> >> suppose we
> >>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>> probably have a vote in the private list to make things
> >> official
> >>>>>>>>>> as
> >>>>>>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send an
> >> email?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
> >>>>>>>> Rohini's
> >>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.
> >> You're
> >>>>>>>>>>>>>>>>> probably
> >>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of doing
> >> each
> >>>>>>>> JIRA
> >>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
> >> reviews this
> >>>>>>>>>>>>>> week,
> >>>>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>>>> actually already started.  I should have some time to
> >> continue
> >>>>>>>>>>>>>> doing
> >>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow
> about
> >>>>>>>>>> getting
> >>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>> done :(
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
> >> Satish was
> >>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Abhishek,
> >>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as
> >> 4.3.0
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help
> >> filter
> >>>>>>>>>> on
> >>>>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I can
> >> pick up
> >>>>>>>>>>>>>>> 10-15
> >>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to
> get
> >>>>>>>>>>>>>> majority of
> >>>>>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can
> >> then use
> >>>>>>>>>> the
> >>>>>>>>>>>>>>> 3rd
> >>>>>>>>>>>>>>>>>>> week
> >>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the
> release
> >>>>>>>>>>>>>> candidate.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Robert,
> >>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk
> >> version
> >>>>>>>> and
> >>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>> release numbers when committing from the beginning
> like
> >> all
> >>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>> projects
> >>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release
> >> version
> >>>>>>>>>> for
> >>>>>>>>>>>>>>>>> every
> >>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete
> >> the trunk
> >>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://docs.google.com/document/d/
> >> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to include
> >> in the
> >>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>> In the
> >>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of
> >> the
> >>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to
> >> review
> >>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>> patches.
> >>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and
> >> get
> >>>>>>>>>>>>>>> committed.
> >>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of the
> >> patches
> >>>>>>>>>> and
> >>>>>>>>>>>>>>>>> provide
> >>>>>>>>>>>>>>>>>>>>> their feedback.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
> >>>>>>>>>>>>>> (approximate)
> >>>>>>>>>>>>>>>>>>>>> release date and start working accordingly. Because,
> >> If we
> >>>>>>>>>> wait
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> all the
> >>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it
> >> might take
> >>>>>>>>>> a
> >>>>>>>>>>>>>>> lot
> >>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of
> >> August to
> >>>>>>>>>>>>>>>>> complete
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> >>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared the
> >> list. I
> >>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>> put,
> >>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now, into
> a
> >> word
> >>>>>>>>>>>>>>>>> document.
> >>>>>>>>>>>>>>>>>>> (For
> >>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list.
> Added
> >> two
> >>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>> Jira
> >>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
> >> property).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below) and
> >> provide
> >>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>> comments.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release
> process.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Scope Document:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://docs.google.com/document/d/
> >> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>>>>>>>>>> mFsuHQZmrT63Ow/edit#
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> >>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not
> >> sure of
> >>>>>>>> is
> >>>>>>>>>>>>>>>>>>> OOZIE-2306
> >>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it
> >> might make
> >>>>>>>>>>>>>>>>> sense to
> >>>>>>>>>>>>>>>>>>>>> let it
> >>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite
> >> ready
> >>>>>>>>>>>>>> yet.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
> >>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of
> >> Oozie)
> >>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for
> >> Java 1.6)
> >>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for
> >> Oozie)
> >>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for
> security
> >> and
> >>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>> fixes)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not
> >> on your
> >>>>>>>>>>>>>>> list
> >>>>>>>>>>>>>>>>>>>>> (including
> >>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need
> to
> >>>>>>>> include
> >>>>>>>>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the
> >> review
> >>>>>>>>>>>>>>>>>>> bandwidth),
> >>>>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?
> I
> >>>>>>>>>>>>>> currently
> >>>>>>>>>>>>>>>>> see
> >>>>>>>>>>>>>>>>>>> 97
> >>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> >>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> >>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
> >>>>>>>>>>>>>> fixes/improvements,
> >>>>>>>>>>>>>>>>> given
> >>>>>>>>>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
> >>>>>>>>>>>>>> available
> >>>>>>>>>>>>>>>>>>> patches.
> >>>>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
> >>>>>>>> anything,
> >>>>>>>>>>>>>>>>> please
> >>>>>>>>>>>>>>>>>>>>> reply
> >>>>>>>>>>>>>>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the
> >> feedback.
> >>>>>>>>>>>>>>>>> Important
> >>>>>>>>>>>>>>>>>>>>> is,
> >>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and
> >> commit.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
> >> provide
> >>>>>>>>>>>>>>> feedback.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ------------------------------
> ------------------------------
> >>>>>>>>>>> ---------------
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
> >> ridiculously
> >>>>>>>>>>>>>> slow
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>>>> cases
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord
> >> action
> >>>>>>>> rerun
> >>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when
> >> reference
> >>>>>>>>>>>>>>>>> variable
> >>>>>>>>>>>>>>>>>>> name
> >>>>>>>>>>>>>>>>>>>>>>>> containing dot
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for
> >> action
> >>>>>>>>>>>>>>>>> failures
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime
> and
> >>>>>>>>>>>>>>>>> endCreatedTime
> >>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect
> >> path for
> >>>>>>>>>>>>>> jar
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when
> >> job is
> >>>>>>>>>>>>>>> running
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>>>> than expected time
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume
> >> more than
> >>>>>>>>>>>>>> 90%
> >>>>>>>>>>>>>>>>> cpu
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
> >> replication-enabled mysql
> >>>>>>>>>>>>>>> urls
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for
> >> logging
> >>>>>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>>>> queued commnads
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces
> >> can not
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>> handled
> >>>>>>>>>>>>>>>>>>>>>>>> rightly
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
> >> hdfs-site and
> >>>>>>>>>>>>>>>>>>> mapred-site
> >>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it
> >> does`t
> >>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for
> >> 'hive
> >>>>>>>>>>>>>> log4j
> >>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not
> >> return
> >>>>>>>> port
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>>>> -servers
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for
> "Changing
> >>>>>>>>>>>>>>>>>>> endtime/pausetime
> >>>>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
> >>>>>>>> downstream
> >>>>>>>>>>>>>>>>>>> components
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation
> >> for jobs
> >>>>>>>>>>>>>> API
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned
> in
> >>>>>>>>>>>>>> minioozie
> >>>>>>>>>>>>>>>>> doc
> >>>>>>>>>>>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be
> >> used for
> >>>>>>>>>>>>>> error
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> audit
> >>>>>>>>>>>>>>>>>>>>>>>> logs
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle
> job
> >>>>>>>>>>>>>> querying
> >>>>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>>>> last modified filter
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
> >>>>>>>>>>>>>> directories
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> /tmp
> >>>>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1
> failed
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP
> connection
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
> >> workflow
> >>>>>>>>>>>>>> action
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create'
> >> should
> >>>>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>>>> uploaded
> >>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
> >>>>>>>> completed
> >>>>>>>>>>>>>>>>> output
> >>>>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed
> >> state when
> >>>>>>>>>>>>>>> kill
> >>>>>>>>>>>>>>>>>>>>> control
> >>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
> >>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the
> >> logs
> >>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>> logging
> >>>>>>>>>>>>>>>>>>>>>>>> command arguments
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ------------------------------
> ------------------------------
> >>>>>>>>>>> ---------------
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
> >> looked good
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> me
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there are
> >> things,
> >>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>> needs
> >>>>>>>>>>>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
> >>>>>>>> progress
> >>>>>>>>>>>>>>>>> status.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
> >>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
> >>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
> >>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
> >>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
> >>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
> >>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
> >>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
> >> (discontinued)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above
> upgrades.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We
> >> will start
> >>>>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
> >> Curator,
> >>>>>>>>>>>>>>>>> HttpClient,
> >>>>>>>>>>>>>>>>>>> Pig
> >>>>>>>>>>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have
> >> done
> >>>>>>>>>>>>>>>>> internally as
> >>>>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231
> >,
> >>>>>>>> curator
> >>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> >>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based
> on
> >> 1.1.0
> >>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> over
> >>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>> year
> >>>>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a
> >> few
> >>>>>>>>>>>>>> lines
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
> >>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
> >> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
> >> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
> >> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
> >> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class
> MiniHCatServer
> >> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation()
> >> throws
> >>>>>>>>>>>>>>>>> Exception {
> >>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not be
> >>>>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> local
> >>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
> >>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no
> >> database
> >>>>>>>>>>>>>> named
> >>>>>>>>>>>>>>>>>>> default)
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
> >>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
> >> createDefaultDB");
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for
> >> Hive
> >>>>>>>>>>>>>> 0.14
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
> >>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
> >> createDefaultDB");
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> public static void
> resetHiveConfStaticVariables()
> >>>>>>>>>>>>>> throws
> >>>>>>>>>>>>>>>>>>>>> Exception
> >>>>>>>>>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
> >> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
> >> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
> >> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> >>>>>>>>>>>>>>>>>>> AbstractHiveService
> >>>>>>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> >>>>>>>>>>>>>>>>>>>>> getHost());
> >>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
> >>>>>>>> HIVE_SERVER2_THRIFT_PORT,
> >>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
> >>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
> >> resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for
> Hive
> >> 0.14
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
> >> resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
> >> "scratch");
> >>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
> >> Ranganathan <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
> >>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
> >> version also
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> 1.x.
> >>>>>>>>>>>>>>>>>>>>> 0.13
> >>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> old
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> >>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>> <mailto:
> >>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined
> >> anywhere
> >>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but
> perhaps
> >> we
> >>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> wait
> >>>>>>>>>>>>>>>>>>>>>>>> until
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop
> >> did this
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> minor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with
> >> doing
> >>>>>>>>>>>>>> it in
> >>>>>>>>>>>>>>>>> 4.3
> >>>>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to
> call
> >> that
> >>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything
> we
> >>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>> leave
> >>>>>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not
> >> currently
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> trunk
> >>>>>>>>>>>>>>>>>>>>>>>> anyway.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs;
> >> we
> >>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>>>> sure
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think
> >> needs more
> >>>>>>>>>>>>>>>>> time to
> >>>>>>>>>>>>>>>>>>>>> bake
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek
> Bafna
> >> <
> >>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <mailto:bafna.iitr@gmail.com
> >>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for
> >> the
> >>>>>>>>>>>>>> release?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna
> <
> >>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for
> >> Oozie
> >>>>>>>>>>>>>> 4.3.0.
> >>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>>> thread
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the
> release.
> >> Please
> >>>>>>>>>>>>>>>>> provide
> >>>>>>>>>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or
> >> any
> >>>>>>>>>>>>>>>>> exclusions)
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
> >> submitted to
> >>>>>>>>>>>>>>> trunk.
> >>>>>>>>>>>>>>>>>>> Based
> >>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if
> >> required.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >> jira/browse/OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036
> >>)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >> jira/browse/OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488
> >>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >> jira/browse/OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> >> jira/browse/OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538
> >>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
> >>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT
> and
> >> NN,
> >>>>>>>>>>>>>>> Database
> >>>>>>>>>>>>>>>>>>>>> schema
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat
> >> upgrade
> >>>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion
> >> of the
> >>>>>>>>>>>>>> above
> >>>>>>>>>>>>>>>>>>> points
> >>>>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also,
> >> take some
> >>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them
> (depending
> >> on the
> >>>>>>>>>>>>>>>>>>>>> availability
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>
> >>
> >>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hello All,

I think we can create the new branch by Thursday (IST) and update version for new branch and master. If we have any important pending Jira's by then we will commit them to both new branch and master.

I will start working on the next steps for release from release doc.
https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release <https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release>

Please do provide if you have any feedback or suggestion.

Thanks,
Abhishek
> On Sep 20, 2016, at 12:52 AM, Rohini Palaniswamy <ro...@gmail.com> wrote:
> 
> Sorry Abhishek for the long delay. Was on vacation and then busy catching
> up with stuff. I did review a bunch of patches today which were marked
> 4.3.0 and Patch Available apart from those which Robert or others have
> already not commented on. Skipping those as they have better context. Tag
> my name if there is something else that needs to be looked at.
> 
> Regards,
> Rohini
> 
> On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> 
>> Hello Robert, Rohini, Purshotam, Jaydeep
>> 
>> As we discussed earlier, will be bumping the priority of some Jira's to
>> 'blocker' for to be included into Oozie 4.3.0 release. We will try to
>> complete as much possible out of that. (It would to good complete all.)
>> 
>> @All, If I miss something, Please do priorities or send a message here.
>> 
>> Thanks a lot for all the efforts and time we have put in last few weeks.
>> 
>> Thanks,
>> Abhishek
>>> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>>> 
>>> Hi All,
>>> 
>>> It has been a long time since we started the scope discussion for the
>> Oozie-4.3.0 release.
>>> Since then a lot of patches got reviewed and committed and I would to
>> thank each one you for contributing your efforts for that.
>>> 
>>> I think we should move to the next stage and go into blocker only mode
>> from next week (one week). After that 7-10 days for preparing and
>> completing release condidate.
>>> 
>>> We did plan to update some of the old tech and some of them are still
>> pending for review. If can spend time into reviewing them, it would be
>> good, else we might have to move without them.
>>> 
>>> There are also some good to have patches available for review and
>> commit, I will bump up the priority for them. If I miss something, please
>> feel free to do the same.
>>> 
>>> Regards,
>>> Abhishek
>>> 
>>>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> Can we please look into these upgrade related bugs?
>>>> 
>>>> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
>>>> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
>>>> OpenJPA              https://issues.apache.org/jira/browse/OOZIE-2488
>>>> Curator              https://issues.apache.org/jira/browse/OOZIE-2231
>>>> Hive                 https://issues.apache.org/jira/browse/OOZIE-2613
>>>> 
>>>> Are the current patches good and feedback if there is any.
>>>> 
>>>> Also, If we can do some reviews collectively.
>>>> 
>>>> Thanks,
>>>> Abhishek
>>>> 
>>>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>>>>> 
>>>>> Hello All,
>>>>> 
>>>>> How is it looking now?
>>>>> 
>>>>> I think we should move to blocker and critical patch mode. If
>> required, we can take couple of days extra there.
>>>>> 
>>>>> ~Abhishek
>>>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>>>>>> 
>>>>>> Sure Robert. We can use some more time.
>>>>>> 
>>>>>> ~Abhishek
>>>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>>>>>>> 
>>>>>>> Do you think we could wait one more week?  I've been really hammered
>> with
>>>>>>> some other work and there's more that I'd like to review here that I
>>>>>>> haven't been able to get to.
>>>>>>> 
>>>>>>> thanks
>>>>>>> - Robert
>>>>>>> 
>>>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Sure. I guess we can one extra week and complete some more work.
>>>>>>>> Thanks everyone for responding.
>>>>>>>> 
>>>>>>>> ~Abhishek
>>>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
>> purushah@yahoo-inc.com.INVALID>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Yes. I also have some pending patches.
>>>>>>>>> 
>>>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>>>>> rkanter@cloudera.com> wrote:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Ya, I could use some more time as well.  I have some deadlines
>> early next
>>>>>>>>> week, which has kept me from doing more reviews.
>>>>>>>>> 
>>>>>>>>> thanks
>>>>>>>>> - Robert
>>>>>>>>> 
>>>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>>> Let see if we can wait for another week. So we can merge few more
>>>>>>>> important
>>>>>>>>>> jiras.  Few of them I personally want to get reviewed and merged.
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Jaydeep
>>>>>>>>>> 
>>>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
>> bafna.iitr@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> As previously discussed, we will be moving with whatever patches
>> can be
>>>>>>>>>>> reviewed/merged till end of this week.
>>>>>>>>>>> 
>>>>>>>>>>> But there are some which needs to be reviewed and merged, mainly
>>>>>>>>>> regarding
>>>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I think we
>>>>>>>> should
>>>>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>>>>> 
>>>>>>>>>>> ~Abhishek
>>>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
>>>>>>>>>> Available
>>>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>>>>> 
>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>>>>>>> 
>>>>>>>>>>>> There are 42 jiras to go. We can take a look mid next week to
>> see if
>>>>>>>>>> any
>>>>>>>>>>>> pending ones needs to be pushed out. There are quite a few
>> which are
>>>>>>>>>>>> unassigned.
>>>>>>>>>>>> 
>>>>>>>>>>>> If there is any jira, someone would like to include in 4.3.0
>> please
>>>>>>>>>>> update
>>>>>>>>>>>> the jira.
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Rohini
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Unset the open ones without sending emails. Will update the
>> resolved,
>>>>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
>> rkanter@cloudera.com
>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we
>> need
>>>>>>>> to
>>>>>>>>>>> put
>>>>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch
>> Available
>>>>>>>>>>> JIRAs.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.
>> We can
>>>>>>>>>>> discuss
>>>>>>>>>>>>>> the issue more there, but I listed a few reasons on
>> OOZIE-2625 about
>>>>>>>>>>> why.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset fix
>> version
>>>>>>>>>> on
>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>> or mark them for next release? Don't see majority of them
>> being
>>>>>>>>>> worked
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> even for next release. We need to do bug scrub and close
>> many of
>>>>>>>>>> these
>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming
>> it to
>>>>>>>>>> the
>>>>>>>>>>>>>>> release number.
>>>>>>>>>>>>>>> There is some amount of relabeling either way. If we go with
>>>>>>>>>> unsetting
>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
>> create a
>>>>>>>>>> 4.4.0
>>>>>>>>>>>>>>> yesterday to move some of the patch available jiras from
>> 4.3.0.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I think it would be better to send out email for the bulk
>> change.
>>>>>>>>>> If
>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can respond.
>> We do
>>>>>>>>>>> that
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come
>> back with
>>>>>>>>>>>>>>> feedback
>>>>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember
>> to tell
>>>>>>>>>> it
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because we're
>> on
>>>>>>>>>>>>>> "master"
>>>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>>>>>>>>>> confusing.
>>>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
>> renaming it
>>>>>>>> to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any
>> current
>>>>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the
>> next
>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can
>> always
>>>>>>>>>>>>>> rename it
>>>>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
>> suppose we
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> probably have a vote in the private list to make things
>> official
>>>>>>>>>> as
>>>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>>>> a release-management-related change.  Can you send an
>> email?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
>>>>>>>> Rohini's
>>>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.
>> You're
>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>> best off using the bulk change feature instead of doing
>> each
>>>>>>>> JIRA
>>>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
>> reviews this
>>>>>>>>>>>>>> week,
>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>> actually already started.  I should have some time to
>> continue
>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
>>>>>>>>>> getting
>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
>> Satish was
>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as
>> 4.3.0
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help
>> filter
>>>>>>>>>> on
>>>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I can
>> pick up
>>>>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to get
>>>>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can
>> then use
>>>>>>>>>> the
>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the release
>>>>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk
>> version
>>>>>>>> and
>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>> release numbers when committing from the beginning like
>> all
>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release
>> version
>>>>>>>>>> for
>>>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete
>> the trunk
>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> https://docs.google.com/document/d/
>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to include
>> in the
>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of
>> the
>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to
>> review
>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and
>> get
>>>>>>>>>>>>>>> committed.
>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> request other contributors also, review some of the
>> patches
>>>>>>>>>> and
>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>>>>> release date and start working accordingly. Because,
>> If we
>>>>>>>>>> wait
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it
>> might take
>>>>>>>>>> a
>>>>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of
>> August to
>>>>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared the
>> list. I
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now, into a
>> word
>>>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added
>> two
>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
>> property).
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below) and
>> provide
>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> https://docs.google.com/document/d/
>> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not
>> sure of
>>>>>>>> is
>>>>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it
>> might make
>>>>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite
>> ready
>>>>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of
>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for
>> Java 1.6)
>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for
>> Oozie)
>>>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security
>> and
>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not
>> on your
>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to
>>>>>>>> include
>>>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the
>> review
>>>>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?  I
>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
>>>>>>>> anything,
>>>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the
>> feedback.
>>>>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and
>> commit.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
>> provide
>>>>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
>> ridiculously
>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord
>> action
>>>>>>>> rerun
>>>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when
>> reference
>>>>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for
>> action
>>>>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
>>>>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect
>> path for
>>>>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when
>> job is
>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume
>> more than
>>>>>>>>>>>>>> 90%
>>>>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
>> replication-enabled mysql
>>>>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for
>> logging
>>>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces
>> can not
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
>> hdfs-site and
>>>>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it
>> does`t
>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for
>> 'hive
>>>>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not
>> return
>>>>>>>> port
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
>>>>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>>>>>>>> downstream
>>>>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation
>> for jobs
>>>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
>>>>>>>>>>>>>> minioozie
>>>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be
>> used for
>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
>>>>>>>>>>>>>> querying
>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>>>>>>>> directories
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
>> workflow
>>>>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create'
>> should
>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
>>>>>>>> completed
>>>>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed
>> state when
>>>>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the
>> logs
>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
>> looked good
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> me
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there are
>> things,
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
>>>>>>>> progress
>>>>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
>> (discontinued)
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We
>> will start
>>>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
>> Curator,
>>>>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have
>> done
>>>>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
>>>>>>>> curator
>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on
>> 1.1.0
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a
>> few
>>>>>>>>>>>>>> lines
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
>> MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer
>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation()
>> throws
>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not be
>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no
>> database
>>>>>>>>>>>>>> named
>>>>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for
>> Hive
>>>>>>>>>>>>>> 0.14
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
>> createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables()
>>>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
>> MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive
>> 0.14
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
>> resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
>> "scratch");
>>>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
>> Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
>> version also
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined
>> anywhere
>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps
>> we
>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop
>> did this
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with
>> doing
>>>>>>>>>>>>>> it in
>>>>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call
>> that
>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not
>> currently
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs;
>> we
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think
>> needs more
>>>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna
>> <
>>>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for
>> the
>>>>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for
>> Oozie
>>>>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release.
>> Please
>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or
>> any
>>>>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
>> submitted to
>>>>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if
>> required.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488
>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
>> jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538
>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and
>> NN,
>>>>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat
>> upgrade
>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion
>> of the
>>>>>>>>>>>>>> above
>>>>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also,
>> take some
>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending
>> on the
>>>>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>> 
>> 
>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy <ro...@gmail.com>]

Sorry Abhishek for the long delay. Was on vacation and then busy catching
up with stuff. I did review a bunch of patches today which were marked
4.3.0 and Patch Available apart from those which Robert or others have
already not commented on. Skipping those as they have better context. Tag
my name if there is something else that needs to be looked at.

Regards,
Rohini

On Mon, Sep 19, 2016 at 6:33 AM, Abhishek Bafna <ba...@gmail.com>
wrote:

> Hello Robert, Rohini, Purshotam, Jaydeep
>
> As we discussed earlier, will be bumping the priority of some Jira's to
> 'blocker' for to be included into Oozie 4.3.0 release. We will try to
> complete as much possible out of that. (It would to good complete all.)
>
> @All, If I miss something, Please do priorities or send a message here.
>
> Thanks a lot for all the efforts and time we have put in last few weeks.
>
> Thanks,
> Abhishek
> > On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > It has been a long time since we started the scope discussion for the
> Oozie-4.3.0 release.
> > Since then a lot of patches got reviewed and committed and I would to
> thank each one you for contributing your efforts for that.
> >
> > I think we should move to the next stage and go into blocker only mode
> from next week (one week). After that 7-10 days for preparing and
> completing release condidate.
> >
> > We did plan to update some of the old tech and some of them are still
> pending for review. If can spend time into reviewing them, it would be
> good, else we might have to move without them.
> >
> > There are also some good to have patches available for review and
> commit, I will bump up the priority for them. If I miss something, please
> feel free to do the same.
> >
> > Regards,
> > Abhishek
> >
> >> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> >>
> >> Hi All,
> >>
> >> Can we please look into these upgrade related bugs?
> >>
> >> ActiveMQ     https://issues.apache.org/jira/browse/OOZIE-2552
> >> HttpClient   https://issues.apache.org/jira/browse/OOZIE-2538
> >> OpenJPA              https://issues.apache.org/jira/browse/OOZIE-2488
> >> Curator              https://issues.apache.org/jira/browse/OOZIE-2231
> >> Hive                 https://issues.apache.org/jira/browse/OOZIE-2613
> >>
> >> Are the current patches good and feedback if there is any.
> >>
> >> Also, If we can do some reviews collectively.
> >>
> >> Thanks,
> >> Abhishek
> >>
> >>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> >>>
> >>> Hello All,
> >>>
> >>> How is it looking now?
> >>>
> >>> I think we should move to blocker and critical patch mode. If
> required, we can take couple of days extra there.
> >>>
> >>> ~Abhishek
> >>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> >>>>
> >>>> Sure Robert. We can use some more time.
> >>>>
> >>>> ~Abhishek
> >>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com>
> wrote:
> >>>>>
> >>>>> Do you think we could wait one more week?  I've been really hammered
> with
> >>>>> some other work and there's more that I'd like to review here that I
> >>>>> haven't been able to get to.
> >>>>>
> >>>>> thanks
> >>>>> - Robert
> >>>>>
> >>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <
> bafna.iitr@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Sure. I guess we can one extra week and complete some more work.
> >>>>>> Thanks everyone for responding.
> >>>>>>
> >>>>>> ~Abhishek
> >>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <
> purushah@yahoo-inc.com.INVALID>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Yes. I also have some pending patches.
> >>>>>>>
> >>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
> >>>>>> rkanter@cloudera.com> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Ya, I could use some more time as well.  I have some deadlines
> early next
> >>>>>>> week, which has kept me from doing more reviews.
> >>>>>>>
> >>>>>>> thanks
> >>>>>>> - Robert
> >>>>>>>
> >>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
> >>>>>>> jaydeepmailv@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> Let see if we can wait for another week. So we can merge few more
> >>>>>> important
> >>>>>>>> jiras.  Few of them I personally want to get reviewed and merged.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Jaydeep
> >>>>>>>>
> >>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <
> bafna.iitr@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> As previously discussed, we will be moving with whatever patches
> can be
> >>>>>>>>> reviewed/merged till end of this week.
> >>>>>>>>>
> >>>>>>>>> But there are some which needs to be reviewed and merged, mainly
> >>>>>>>> regarding
> >>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I think we
> >>>>>> should
> >>>>>>>>> review OOZIE-2036 and merged it.
> >>>>>>>>>
> >>>>>>>>> ~Abhishek
> >>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
> >>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
> >>>>>>>> Available
> >>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
> >>>>>>>>>>
> >>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
> >>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
> >>>>>>>>>>
> >>>>>>>>>> There are 42 jiras to go. We can take a look mid next week to
> see if
> >>>>>>>> any
> >>>>>>>>>> pending ones needs to be pushed out. There are quite a few
> which are
> >>>>>>>>>> unassigned.
> >>>>>>>>>>
> >>>>>>>>>> If there is any jira, someone would like to include in 4.3.0
> please
> >>>>>>>>> update
> >>>>>>>>>> the jira.
> >>>>>>>>>>
> >>>>>>>>>> Regards,
> >>>>>>>>>> Rohini
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
> >>>>>>>>> rohini.aditya@gmail.com
> >>>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Unset the open ones without sending emails. Will update the
> resolved,
> >>>>>>>>>>> closed and Patch Available ones sending emails.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>> Rohini
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <
> rkanter@cloudera.com
> >>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we
> need
> >>>>>> to
> >>>>>>>>> put
> >>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch
> Available
> >>>>>>>>> JIRAs.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.
> We can
> >>>>>>>>> discuss
> >>>>>>>>>>>> the issue more there, but I listed a few reasons on
> OOZIE-2625 about
> >>>>>>>>> why.
> >>>>>>>>>>>>
> >>>>>>>>>>>> - Robert
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> >>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
> >>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
> >>>>>>>>> 20status%20DESC&startIndex=50
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset fix
> version
> >>>>>>>> on
> >>>>>>>>>>>> these
> >>>>>>>>>>>>> or mark them for next release? Don't see majority of them
> being
> >>>>>>>> worked
> >>>>>>>>>>>> on
> >>>>>>>>>>>>> even for next release. We need to do bug scrub and close
> many of
> >>>>>>>> these
> >>>>>>>>>>>> some
> >>>>>>>>>>>>> time.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming
> it to
> >>>>>>>> the
> >>>>>>>>>>>>> release number.
> >>>>>>>>>>>>> There is some amount of relabeling either way. If we go with
> >>>>>>>> unsetting
> >>>>>>>>>>>> fix
> >>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did
> create a
> >>>>>>>> 4.4.0
> >>>>>>>>>>>>> yesterday to move some of the patch available jiras from
> 4.3.0.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> >>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I think it would be better to send out email for the bulk
> change.
> >>>>>>>> If
> >>>>>>>>>>>>> there
> >>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can respond.
> We do
> >>>>>>>>> that
> >>>>>>>>>>>>> with
> >>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come
> back with
> >>>>>>>>>>>>> feedback
> >>>>>>>>>>>>>> on pulling back to release.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
> >>>>>>>> rkanter@cloudera.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember
> to tell
> >>>>>>>> it
> >>>>>>>>>>>> not
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> send out an email for every change :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
> >>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because we're
> on
> >>>>>>>>>>>> "master"
> >>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
> >>>>>>>> confusing.
> >>>>>>>>>>>>>>> Instead
> >>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just
> renaming it
> >>>>>> to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any
> current
> >>>>>>>>>>>> "trunk"
> >>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the
> next
> >>>>>>>>>>>> release
> >>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can
> always
> >>>>>>>>>>>> rename it
> >>>>>>>>>>>>>>>> later.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I
> suppose we
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>>>> probably have a vote in the private list to make things
> official
> >>>>>>>> as
> >>>>>>>>>>>>>>> this is
> >>>>>>>>>>>>>>>> a release-management-related change.  Can you send an
> email?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
> >>>>>> Rohini's
> >>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.
> You're
> >>>>>>>>>>>>>>> probably
> >>>>>>>>>>>>>>>> best off using the bulk change feature instead of doing
> each
> >>>>>> JIRA
> >>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of
> reviews this
> >>>>>>>>>>>> week,
> >>>>>>>>>>>>>>> I've
> >>>>>>>>>>>>>>>> actually already started.  I should have some time to
> continue
> >>>>>>>>>>>> doing
> >>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
> >>>>>>>> getting
> >>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>> done :(
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that
> Satish was
> >>>>>>>>>>>> working
> >>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> >>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Abhishek,
> >>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as
> 4.3.0
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help
> filter
> >>>>>>>> on
> >>>>>>>>>>>>>>> those
> >>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I can
> pick up
> >>>>>>>>>>>>> 10-15
> >>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to get
> >>>>>>>>>>>> majority of
> >>>>>>>>>>>>>>>>> patches
> >>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can
> then use
> >>>>>>>> the
> >>>>>>>>>>>>> 3rd
> >>>>>>>>>>>>>>>>> week
> >>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the release
> >>>>>>>>>>>> candidate.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Robert,
> >>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk
> version
> >>>>>> and
> >>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>> release numbers when committing from the beginning like
> all
> >>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>> projects
> >>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release
> version
> >>>>>>>> for
> >>>>>>>>>>>>>>> every
> >>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete
> the trunk
> >>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>> Rohini
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> https://docs.google.com/document/d/
> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to include
> in the
> >>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>> In the
> >>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of
> the
> >>>>>>>>>>>> patches
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to
> review
> >>>>>>>> more
> >>>>>>>>>>>>>>>>> patches.
> >>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and
> get
> >>>>>>>>>>>>> committed.
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>> request other contributors also, review some of the
> patches
> >>>>>>>> and
> >>>>>>>>>>>>>>> provide
> >>>>>>>>>>>>>>>>>>> their feedback.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
> >>>>>>>>>>>> (approximate)
> >>>>>>>>>>>>>>>>>>> release date and start working accordingly. Because,
> If we
> >>>>>>>> wait
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> all the
> >>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it
> might take
> >>>>>>>> a
> >>>>>>>>>>>>> lot
> >>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of
> August to
> >>>>>>>>>>>>>>> complete
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> >>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared the
> list. I
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>> put,
> >>>>>>>>>>>>>>>>>>> scope content which we have discussed till now, into a
> word
> >>>>>>>>>>>>>>> document.
> >>>>>>>>>>>>>>>>> (For
> >>>>>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added
> two
> >>>>>>>>>>>> more
> >>>>>>>>>>>>>>> Jira
> >>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0
> property).
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below) and
> provide
> >>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>> comments.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release process.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Scope Document:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> https://docs.google.com/document/d/
> 1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>>>>>>>> mFsuHQZmrT63Ow/edit#
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> >>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not
> sure of
> >>>>>> is
> >>>>>>>>>>>>>>>>> OOZIE-2306
> >>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it
> might make
> >>>>>>>>>>>>>>> sense to
> >>>>>>>>>>>>>>>>>>> let it
> >>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite
> ready
> >>>>>>>>>>>> yet.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
> >>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of
> Oozie)
> >>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for
> Java 1.6)
> >>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for
> Oozie)
> >>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security
> and
> >>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>> fixes)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not
> on your
> >>>>>>>>>>>>> list
> >>>>>>>>>>>>>>>>>>> (including
> >>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to
> >>>>>> include
> >>>>>>>>>>>>>>>>> everything
> >>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the
> review
> >>>>>>>>>>>>>>>>> bandwidth),
> >>>>>>>>>>>>>>>>>>> did
> >>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?  I
> >>>>>>>>>>>> currently
> >>>>>>>>>>>>>>> see
> >>>>>>>>>>>>>>>>> 97
> >>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> >>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> >>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
> >>>>>>>>>>>> fixes/improvements,
> >>>>>>>>>>>>>>> given
> >>>>>>>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>>>>>>> popular it seems to be.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
> >>>>>>>>>>>> available
> >>>>>>>>>>>>>>>>> patches.
> >>>>>>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
> >>>>>> anything,
> >>>>>>>>>>>>>>> please
> >>>>>>>>>>>>>>>>>>> reply
> >>>>>>>>>>>>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the
> feedback.
> >>>>>>>>>>>>>>> Important
> >>>>>>>>>>>>>>>>>>> is,
> >>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and
> commit.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and
> provide
> >>>>>>>>>>>>> feedback.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> ------------------------------------------------------------
> >>>>>>>>> ---------------
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is
> ridiculously
> >>>>>>>>>>>> slow
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>> cases
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord
> action
> >>>>>> rerun
> >>>>>>>>>>>>>>>>>>> [OOZIE-1735]
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when
> reference
> >>>>>>>>>>>>>>> variable
> >>>>>>>>>>>>>>>>> name
> >>>>>>>>>>>>>>>>>>>>>> containing dot
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for
> action
> >>>>>>>>>>>>>>> failures
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
> >>>>>>>>>>>>>>> endCreatedTime
> >>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect
> path for
> >>>>>>>>>>>> jar
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when
> job is
> >>>>>>>>>>>>> running
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>>>>>>>> than expected time
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume
> more than
> >>>>>>>>>>>> 90%
> >>>>>>>>>>>>>>> cpu
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support
> replication-enabled mysql
> >>>>>>>>>>>>> urls
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for
> logging
> >>>>>>>>>>>>>>>>> information
> >>>>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>>>> queued commnads
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces
> can not
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>> handled
> >>>>>>>>>>>>>>>>>>>>>> rightly
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site,
> hdfs-site and
> >>>>>>>>>>>>>>>>> mapred-site
> >>>>>>>>>>>>>>>>>>>>>> into spark launcher
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it
> does`t
> >>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for
> 'hive
> >>>>>>>>>>>> log4j
> >>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not
> return
> >>>>>> port
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> admin
> >>>>>>>>>>>>>>>>>>>>>> -servers
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
> >>>>>>>>>>>>>>>>> endtime/pausetime
> >>>>>>>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
> >>>>>> downstream
> >>>>>>>>>>>>>>>>> components
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation
> for jobs
> >>>>>>>>>>>> API
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
> >>>>>>>>>>>> minioozie
> >>>>>>>>>>>>>>> doc
> >>>>>>>>>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be
> used for
> >>>>>>>>>>>> error
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> audit
> >>>>>>>>>>>>>>>>>>>>>> logs
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
> >>>>>>>>>>>> querying
> >>>>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>>>>>>>> last modified filter
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
> >>>>>>>>>>>> directories
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> /tmp
> >>>>>>>>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for
> workflow
> >>>>>>>>>>>> action
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create'
> should
> >>>>>>>>>>>> ensure
> >>>>>>>>>>>>>>>>>>> uploaded
> >>>>>>>>>>>>>>>>>>>>>> jars are world readable
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
> >>>>>> completed
> >>>>>>>>>>>>>>> output
> >>>>>>>>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed
> state when
> >>>>>>>>>>>>> kill
> >>>>>>>>>>>>>>>>>>> control
> >>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
> >>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the
> logs
> >>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>> logging
> >>>>>>>>>>>>>>>>>>>>>> command arguments
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> ------------------------------------------------------------
> >>>>>>>>> ---------------
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> >>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all
> looked good
> >>>>>>>>>>>> to
> >>>>>>>>>>>>> me
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there are
> things,
> >>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>> needs
> >>>>>>>>>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
> >>>>>> progress
> >>>>>>>>>>>>>>> status.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
> >>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
> >>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
> >>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
> >>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
> >>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
> >>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
> >>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6
> (discontinued)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We
> will start
> >>>>>>>>>>>>>>> working
> >>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA,
> Curator,
> >>>>>>>>>>>>>>> HttpClient,
> >>>>>>>>>>>>>>>>> Pig
> >>>>>>>>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have
> done
> >>>>>>>>>>>>>>> internally as
> >>>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
> >>>>>> curator
> >>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> >>>>>>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on
> 1.1.0
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> over
> >>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>> year
> >>>>>>>>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a
> few
> >>>>>>>>>>>> lines
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
> >>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
> >>>>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/
> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/
> MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer
> {
> >>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation()
> throws
> >>>>>>>>>>>>>>> Exception {
> >>>>>>>>>>>>>>>>>>>>>>>>>  // Need to do this, else default db will not be
> >>>>>>>>>>>>> created
> >>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>> local
> >>>>>>>>>>>>>>>>>>>>>>>>> metastores.
> >>>>>>>>>>>>>>>>>>>>>>>>>  // TestHiveMain will fail with
> >>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no
> database
> >>>>>>>>>>>> named
> >>>>>>>>>>>>>>>>> default)
> >>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
> >>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
> createDefaultDB");
> >>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for
> Hive
> >>>>>>>>>>>> 0.14
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
> >>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("
> createDefaultDB");
> >>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables()
> >>>>>>>>>>>> throws
> >>>>>>>>>>>>>>>>>>> Exception
> >>>>>>>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
> >>>>>>>>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/
> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/
> MiniHS2.java
> >>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> >>>>>>>>>>>>>>>>> AbstractHiveService
> >>>>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> >>>>>>>>>>>>>>>>>>> getHost());
> >>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
> >>>>>> HIVE_SERVER2_THRIFT_PORT,
> >>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
> >>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.
> resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive
> 0.14
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.
> resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir,
> "scratch");
> >>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat
> Ranganathan <
> >>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
> >>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive
> version also
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> 1.x.
> >>>>>>>>>>>>>>>>>>> 0.13
> >>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> old
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> >>>>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>>>>> <mailto:
> >>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined
> anywhere
> >>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>>>>>>>> okay
> >>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps
> we
> >>>>>>>>>>>> need
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> wait
> >>>>>>>>>>>>>>>>>>>>>> until
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop
> did this
> >>>>>>>>>>>> in
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>> minor
> >>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with
> doing
> >>>>>>>>>>>> it in
> >>>>>>>>>>>>>>> 4.3
> >>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
> >>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call
> that
> >>>>>>>>>>>> out
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>> notes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>>> leave
> >>>>>>>>>>>>>>>>> out
> >>>>>>>>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not
> currently
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>> trunk
> >>>>>>>>>>>>>>>>>>>>>> anyway.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs;
> we
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>>> make
> >>>>>>>>>>>>>>>>>>> sure
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
> >>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think
> needs more
> >>>>>>>>>>>>>>> time to
> >>>>>>>>>>>>>>>>>>> bake
> >>>>>>>>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna
> <
> >>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for
> the
> >>>>>>>>>>>> release?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for
> Oozie
> >>>>>>>>>>>> 4.3.0.
> >>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>> thread
> >>>>>>>>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release.
> Please
> >>>>>>>>>>>>>>> provide
> >>>>>>>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or
> any
> >>>>>>>>>>>>>>> exclusions)
> >>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches
> submitted to
> >>>>>>>>>>>>> trunk.
> >>>>>>>>>>>>>>>>> Based
> >>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if
> required.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488
> >>),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/
> jira/browse/OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538
> >>),
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
> >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> ).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and
> NN,
> >>>>>>>>>>>>> Database
> >>>>>>>>>>>>>>>>>>> schema
> >>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat
> upgrade
> >>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion
> of the
> >>>>>>>>>>>> above
> >>>>>>>>>>>>>>>>> points
> >>>>>>>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also,
> take some
> >>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending
> on the
> >>>>>>>>>>>>>>>>>>> availability
> >>>>>>>>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hello Robert, Rohini, Purshotam, Jaydeep

As we discussed earlier, will be bumping the priority of some Jira's to 'blocker' for to be included into Oozie 4.3.0 release. We will try to complete as much possible out of that. (It would to good complete all.)

@All, If I miss something, Please do priorities or send a message here.

Thanks a lot for all the efforts and time we have put in last few weeks.

Thanks,
Abhishek
> On Sep 8, 2016, at 10:14 AM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hi All,
> 
> It has been a long time since we started the scope discussion for the Oozie-4.3.0 release.
> Since then a lot of patches got reviewed and committed and I would to thank each one you for contributing your efforts for that.
> 
> I think we should move to the next stage and go into blocker only mode from next week (one week). After that 7-10 days for preparing and completing release condidate.
> 
> We did plan to update some of the old tech and some of them are still pending for review. If can spend time into reviewing them, it would be good, else we might have to move without them.
> 
> There are also some good to have patches available for review and commit, I will bump up the priority for them. If I miss something, please feel free to do the same.
> 
> Regards,
> Abhishek
> 
>> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com> wrote:
>> 
>> Hi All,
>> 
>> Can we please look into these upgrade related bugs?
>> 
>> ActiveMQ	https://issues.apache.org/jira/browse/OOZIE-2552
>> HttpClient	https://issues.apache.org/jira/browse/OOZIE-2538
>> OpenJPA		https://issues.apache.org/jira/browse/OOZIE-2488
>> Curator		https://issues.apache.org/jira/browse/OOZIE-2231
>> Hive			https://issues.apache.org/jira/browse/OOZIE-2613
>> 
>> Are the current patches good and feedback if there is any.
>> 
>> Also, If we can do some reviews collectively.
>> 
>> Thanks,
>> Abhishek
>> 
>>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>>> 
>>> Hello All,
>>> 
>>> How is it looking now?
>>> 
>>> I think we should move to blocker and critical patch mode. If required, we can take couple of days extra there.
>>> 
>>> ~Abhishek
>>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>>>> 
>>>> Sure Robert. We can use some more time.
>>>> 
>>>> ~Abhishek
>>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com> wrote:
>>>>> 
>>>>> Do you think we could wait one more week?  I've been really hammered with
>>>>> some other work and there's more that I'd like to review here that I
>>>>> haven't been able to get to.
>>>>> 
>>>>> thanks
>>>>> - Robert
>>>>> 
>>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <ba...@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> Sure. I guess we can one extra week and complete some more work.
>>>>>> Thanks everyone for responding.
>>>>>> 
>>>>>> ~Abhishek
>>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <pu...@yahoo-inc.com.INVALID>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Yes. I also have some pending patches.
>>>>>>> 
>>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>>> rkanter@cloudera.com> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> Ya, I could use some more time as well.  I have some deadlines early next
>>>>>>> week, which has kept me from doing more reviews.
>>>>>>> 
>>>>>>> thanks
>>>>>>> - Robert
>>>>>>> 
>>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>>> 
>>>>>>>> Let see if we can wait for another week. So we can merge few more
>>>>>> important
>>>>>>>> jiras.  Few of them I personally want to get reviewed and merged.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Jaydeep
>>>>>>>> 
>>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> As previously discussed, we will be moving with whatever patches can be
>>>>>>>>> reviewed/merged till end of this week.
>>>>>>>>> 
>>>>>>>>> But there are some which needs to be reviewed and merged, mainly
>>>>>>>> regarding
>>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I think we
>>>>>> should
>>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>>> 
>>>>>>>>> ~Abhishek
>>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
>>>>>>>> Available
>>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>>> 
>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>>>>> 
>>>>>>>>>> There are 42 jiras to go. We can take a look mid next week to see if
>>>>>>>> any
>>>>>>>>>> pending ones needs to be pushed out. There are quite a few which are
>>>>>>>>>> unassigned.
>>>>>>>>>> 
>>>>>>>>>> If there is any jira, someone would like to include in 4.3.0 please
>>>>>>>>> update
>>>>>>>>>> the jira.
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Rohini
>>>>>>>>>> 
>>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Unset the open ones without sending emails. Will update the resolved,
>>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>>> 
>>>>>>>>>>> Regards,
>>>>>>>>>>> Rohini
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rkanter@cloudera.com
>>>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we need
>>>>>> to
>>>>>>>>> put
>>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch Available
>>>>>>>>> JIRAs.
>>>>>>>>>>>> 
>>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
>>>>>>>>> discuss
>>>>>>>>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about
>>>>>>>>> why.
>>>>>>>>>>>> 
>>>>>>>>>>>> - Robert
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>>> 
>>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset fix version
>>>>>>>> on
>>>>>>>>>>>> these
>>>>>>>>>>>>> or mark them for next release? Don't see majority of them being
>>>>>>>> worked
>>>>>>>>>>>> on
>>>>>>>>>>>>> even for next release. We need to do bug scrub and close many of
>>>>>>>> these
>>>>>>>>>>>> some
>>>>>>>>>>>>> time.
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>>>>>>>> the
>>>>>>>>>>>>> release number.
>>>>>>>>>>>>> There is some amount of relabeling either way. If we go with
>>>>>>>> unsetting
>>>>>>>>>>>> fix
>>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did create a
>>>>>>>> 4.4.0
>>>>>>>>>>>>> yesterday to move some of the patch available jiras from 4.3.0.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I think it would be better to send out email for the bulk change.
>>>>>>>> If
>>>>>>>>>>>>> there
>>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can respond. We do
>>>>>>>>> that
>>>>>>>>>>>>> with
>>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come back with
>>>>>>>>>>>>> feedback
>>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell
>>>>>>>> it
>>>>>>>>>>>> not
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>>> rkanter@cloudera.com
>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because we're on
>>>>>>>>>>>> "master"
>>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>>>>>>>> confusing.
>>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it
>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any current
>>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
>>>>>>>>>>>> release
>>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
>>>>>>>>>>>> rename it
>>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> probably have a vote in the private list to make things official
>>>>>>>> as
>>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>>> a release-management-related change.  Can you send an email?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
>>>>>> Rohini's
>>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.  You're
>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>> best off using the bulk change feature instead of doing each
>>>>>> JIRA
>>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of reviews this
>>>>>>>>>>>> week,
>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>> actually already started.  I should have some time to continue
>>>>>>>>>>>> doing
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
>>>>>>>> getting
>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was
>>>>>>>>>>>> working
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as 4.3.0
>>>>>>>>>>>> and
>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter
>>>>>>>> on
>>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I can pick up
>>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to get
>>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
>>>>>>>> the
>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the release
>>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk version
>>>>>> and
>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>> release numbers when committing from the beginning like all
>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release version
>>>>>>>> for
>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk
>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to include in the
>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of the
>>>>>>>>>>>> patches
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to review
>>>>>>>> more
>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and get
>>>>>>>>>>>>> committed.
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> request other contributors also, review some of the patches
>>>>>>>> and
>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>>> release date and start working accordingly. Because, If we
>>>>>>>> wait
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it might take
>>>>>>>> a
>>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of August to
>>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared the list. I
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>>> scope content which we have discussed till now, into a word
>>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two
>>>>>>>>>>>> more
>>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below) and provide
>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of
>>>>>> is
>>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
>>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
>>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to
>>>>>> include
>>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
>>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?  I
>>>>>>>>>>>> currently
>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>>>>>>>> available
>>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
>>>>>> anything,
>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
>>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and commit.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and provide
>>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
>>>>>>>>>>>> slow
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action
>>>>>> rerun
>>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
>>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
>>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
>>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
>>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
>>>>>>>>>>>>> running
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
>>>>>>>>>>>> 90%
>>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
>>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
>>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
>>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return
>>>>>> port
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
>>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>>>>>> downstream
>>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
>>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
>>>>>>>>>>>> minioozie
>>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
>>>>>>>>>>>> error
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
>>>>>>>>>>>> querying
>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>>>>>> directories
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
>>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
>>>>>> completed
>>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
>>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
>>>>>>>>>>>> to
>>>>>>>>>>>>> me
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there are things,
>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
>>>>>> progress
>>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
>>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
>>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
>>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
>>>>>> curator
>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
>>>>>>>>>>>> for
>>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
>>>>>>>>>>>> lines
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation() throws
>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>  // Need to do this, else default db will not be
>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>>  // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
>>>>>>>>>>>> named
>>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
>>>>>>>>>>>> 0.14
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables()
>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
>>>>>>>>>>>> and
>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
>>>>>>>>>>>> need
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
>>>>>>>>>>>> in
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
>>>>>>>>>>>> it in
>>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
>>>>>>>>>>>> out
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>>>>>>>>>>> should
>>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
>>>>>>>>>>>> in
>>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
>>>>>>>>>>>> should
>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
>>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
>>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
>>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
>>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
>>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
>>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
>>>>>>>>>>>> above
>>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
>>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

It has been a long time since we started the scope discussion for the Oozie-4.3.0 release.
Since then a lot of patches got reviewed and committed and I would to thank each one you for contributing your efforts for that.

I think we should move to the next stage and go into blocker only mode from next week (one week). After that 7-10 days for preparing and completing release condidate.

We did plan to update some of the old tech and some of them are still pending for review. If can spend time into reviewing them, it would be good, else we might have to move without them.

There are also some good to have patches available for review and commit, I will bump up the priority for them. If I miss something, please feel free to do the same.

Regards,
Abhishek

> On Aug 31, 2016, at 2:30 PM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hi All,
> 
> Can we please look into these upgrade related bugs?
> 
> ActiveMQ	https://issues.apache.org/jira/browse/OOZIE-2552
> HttpClient	https://issues.apache.org/jira/browse/OOZIE-2538
> OpenJPA		https://issues.apache.org/jira/browse/OOZIE-2488
> Curator		https://issues.apache.org/jira/browse/OOZIE-2231
> Hive			https://issues.apache.org/jira/browse/OOZIE-2613
> 
> Are the current patches good and feedback if there is any.
> 
> Also, If we can do some reviews collectively.
> 
> Thanks,
> Abhishek
> 
>> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>> 
>> Hello All,
>> 
>> How is it looking now?
>> 
>> I think we should move to blocker and critical patch mode. If required, we can take couple of days extra there.
>> 
>> ~Abhishek
>>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>>> 
>>> Sure Robert. We can use some more time.
>>> 
>>> ~Abhishek
>>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com> wrote:
>>>> 
>>>> Do you think we could wait one more week?  I've been really hammered with
>>>> some other work and there's more that I'd like to review here that I
>>>> haven't been able to get to.
>>>> 
>>>> thanks
>>>> - Robert
>>>> 
>>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>> 
>>>>> Sure. I guess we can one extra week and complete some more work.
>>>>> Thanks everyone for responding.
>>>>> 
>>>>> ~Abhishek
>>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <pu...@yahoo-inc.com.INVALID>
>>>>> wrote:
>>>>>> 
>>>>>> Yes. I also have some pending patches.
>>>>>> 
>>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>>> rkanter@cloudera.com> wrote:
>>>>>> 
>>>>>> 
>>>>>> Ya, I could use some more time as well.  I have some deadlines early next
>>>>>> week, which has kept me from doing more reviews.
>>>>>> 
>>>>>> thanks
>>>>>> - Robert
>>>>>> 
>>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>>> jaydeepmailv@gmail.com> wrote:
>>>>>> 
>>>>>>> Let see if we can wait for another week. So we can merge few more
>>>>> important
>>>>>>> jiras.  Few of them I personally want to get reviewed and merged.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Jaydeep
>>>>>>> 
>>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> As previously discussed, we will be moving with whatever patches can be
>>>>>>>> reviewed/merged till end of this week.
>>>>>>>> 
>>>>>>>> But there are some which needs to be reviewed and merged, mainly
>>>>>>> regarding
>>>>>>>> the upgrades. Some of the patches require Java 1.7, so I think we
>>>>> should
>>>>>>>> review OOZIE-2036 and merged it.
>>>>>>>> 
>>>>>>>> ~Abhishek
>>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>>> rohini.aditya@gmail.com>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
>>>>>>> Available
>>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>>> 
>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>>>> 
>>>>>>>>> There are 42 jiras to go. We can take a look mid next week to see if
>>>>>>> any
>>>>>>>>> pending ones needs to be pushed out. There are quite a few which are
>>>>>>>>> unassigned.
>>>>>>>>> 
>>>>>>>>> If there is any jira, someone would like to include in 4.3.0 please
>>>>>>>> update
>>>>>>>>> the jira.
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Rohini
>>>>>>>>> 
>>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>>> rohini.aditya@gmail.com
>>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Unset the open ones without sending emails. Will update the resolved,
>>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>>> 
>>>>>>>>>> Regards,
>>>>>>>>>> Rohini
>>>>>>>>>> 
>>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rkanter@cloudera.com
>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we need
>>>>> to
>>>>>>>> put
>>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch Available
>>>>>>>> JIRAs.
>>>>>>>>>>> 
>>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
>>>>>>>> discuss
>>>>>>>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about
>>>>>>>> why.
>>>>>>>>>>> 
>>>>>>>>>>> - Robert
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>>> 
>>>>>>>>>>>> These are jiras open and marked trunk. Should we unset fix version
>>>>>>> on
>>>>>>>>>>> these
>>>>>>>>>>>> or mark them for next release? Don't see majority of them being
>>>>>>> worked
>>>>>>>>>>> on
>>>>>>>>>>>> even for next release. We need to do bug scrub and close many of
>>>>>>> these
>>>>>>>>>>> some
>>>>>>>>>>>> time.
>>>>>>>>>>>> 
>>>>>>>>>>>>> Instead
>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>>>>>>> the
>>>>>>>>>>>> release number.
>>>>>>>>>>>> There is some amount of relabeling either way. If we go with
>>>>>>> unsetting
>>>>>>>>>>> fix
>>>>>>>>>>>> version, then we can as well delete trunk. Also, I did create a
>>>>>>> 4.4.0
>>>>>>>>>>>> yesterday to move some of the patch available jiras from 4.3.0.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> I think it would be better to send out email for the bulk change.
>>>>>>> If
>>>>>>>>>>>> there
>>>>>>>>>>>>> are any issues or jiras wrongly marked someone can respond. We do
>>>>>>>> that
>>>>>>>>>>>> with
>>>>>>>>>>>>> Pig and there are always couple of jiras that folks come back with
>>>>>>>>>>>> feedback
>>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell
>>>>>>> it
>>>>>>>>>>> not
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>>> rkanter@cloudera.com
>>>>>>>>> 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because we're on
>>>>>>>>>>> "master"
>>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>>>>>>> confusing.
>>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it
>>>>> to
>>>>>>>>>>> the
>>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any current
>>>>>>>>>>> "trunk"
>>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
>>>>>>>>>>> release
>>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
>>>>>>>>>>> rename it
>>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
>>>>>>>>>>> should
>>>>>>>>>>>>>>> probably have a vote in the private list to make things official
>>>>>>> as
>>>>>>>>>>>>>> this is
>>>>>>>>>>>>>>> a release-management-related change.  Can you send an email?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
>>>>> Rohini's
>>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.  You're
>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>> best off using the bulk change feature instead of doing each
>>>>> JIRA
>>>>>>>>>>>>>>> individually.  I should have time to do a bunch of reviews this
>>>>>>>>>>> week,
>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>> actually already started.  I should have some time to continue
>>>>>>>>>>> doing
>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
>>>>>>> getting
>>>>>>>>>>>> these
>>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was
>>>>>>>>>>> working
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as 4.3.0
>>>>>>>>>>> and
>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter
>>>>>>> on
>>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I can pick up
>>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to get
>>>>>>>>>>> majority of
>>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
>>>>>>> the
>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>>> for any pending critical issues and rolling the release
>>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk version
>>>>> and
>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>> release numbers when committing from the beginning like all
>>>>>>>>>>> other
>>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release version
>>>>>>> for
>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk
>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> and listed all the things which we wanted to include in the
>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of the
>>>>>>>>>>> patches
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to review
>>>>>>> more
>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and get
>>>>>>>>>>>> committed.
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> request other contributors also, review some of the patches
>>>>>>> and
>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>>> release date and start working accordingly. Because, If we
>>>>>>> wait
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it might take
>>>>>>> a
>>>>>>>>>>>> lot
>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of August to
>>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared the list. I
>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>>> scope content which we have discussed till now, into a word
>>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two
>>>>>>>>>>> more
>>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Please take a look at the document (link below) and provide
>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of
>>>>> is
>>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
>>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
>>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
>>>>>>>>>>> other
>>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to
>>>>> include
>>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
>>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?  I
>>>>>>>>>>> currently
>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>>>>>>> available
>>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
>>>>> anything,
>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
>>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and commit.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and provide
>>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
>>>>>>>>>>> slow
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action
>>>>> rerun
>>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
>>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
>>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
>>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
>>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
>>>>>>>>>>>> running
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
>>>>>>>>>>> 90%
>>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
>>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
>>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
>>>>>>>>>>> be
>>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
>>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return
>>>>> port
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
>>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>>>>> downstream
>>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
>>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
>>>>>>>>>>> minioozie
>>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
>>>>>>>>>>> error
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
>>>>>>>>>>> querying
>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>>>>> directories
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
>>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
>>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
>>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
>>>>> completed
>>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
>>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
>>>>>>>>>>> when
>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
>>>>>>>>>>> to
>>>>>>>>>>>> me
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> should be included into the release. If there are things,
>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
>>>>> progress
>>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
>>>>>>>>>>>>>> working
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
>>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
>>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
>>>>> curator
>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
>>>>>>>>>>> for
>>>>>>>>>>>>>> over
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
>>>>>>>>>>> lines
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation() throws
>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>   // Need to do this, else default db will not be
>>>>>>>>>>>> created
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>>   // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
>>>>>>>>>>> named
>>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
>>>>>>>>>>> 0.14
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables()
>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
>>>>>>>>>>> and
>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
>>>>>>>>>>> to
>>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
>>>>>>>>>>> need
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
>>>>>>>>>>> in
>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
>>>>>>>>>>> it in
>>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
>>>>>>>>>>> out
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>>>>>>>>>> should
>>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
>>>>>>>>>>> in
>>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
>>>>>>>>>>> should
>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
>>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
>>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
>>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
>>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
>>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
>>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
>>>>>>>>>>> above
>>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
>>>>>>>>>>> work
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
>>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>> 
>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

Can we please look into these upgrade related bugs?

ActiveMQ	https://issues.apache.org/jira/browse/OOZIE-2552
HttpClient	https://issues.apache.org/jira/browse/OOZIE-2538
OpenJPA		https://issues.apache.org/jira/browse/OOZIE-2488
Curator		https://issues.apache.org/jira/browse/OOZIE-2231
Hive			https://issues.apache.org/jira/browse/OOZIE-2613

Are the current patches good and feedback if there is any.

Also, If we can do some reviews collectively.

Thanks,
Abhishek

> On Aug 27, 2016, at 1:14 AM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hello All,
> 
> How is it looking now?
> 
> I think we should move to blocker and critical patch mode. If required, we can take couple of days extra there.
> 
> ~Abhishek
>> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>> 
>> Sure Robert. We can use some more time.
>> 
>> ~Abhishek
>>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com> wrote:
>>> 
>>> Do you think we could wait one more week?  I've been really hammered with
>>> some other work and there's more that I'd like to review here that I
>>> haven't been able to get to.
>>> 
>>> thanks
>>> - Robert
>>> 
>>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <ba...@gmail.com>
>>> wrote:
>>> 
>>>> Sure. I guess we can one extra week and complete some more work.
>>>> Thanks everyone for responding.
>>>> 
>>>> ~Abhishek
>>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <pu...@yahoo-inc.com.INVALID>
>>>> wrote:
>>>>> 
>>>>> Yes. I also have some pending patches.
>>>>> 
>>>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>>> rkanter@cloudera.com> wrote:
>>>>> 
>>>>> 
>>>>> Ya, I could use some more time as well.  I have some deadlines early next
>>>>> week, which has kept me from doing more reviews.
>>>>> 
>>>>> thanks
>>>>> - Robert
>>>>> 
>>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>>> jaydeepmailv@gmail.com> wrote:
>>>>> 
>>>>>> Let see if we can wait for another week. So we can merge few more
>>>> important
>>>>>> jiras.  Few of them I personally want to get reviewed and merged.
>>>>>> 
>>>>>> Regards,
>>>>>> Jaydeep
>>>>>> 
>>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> As previously discussed, we will be moving with whatever patches can be
>>>>>>> reviewed/merged till end of this week.
>>>>>>> 
>>>>>>> But there are some which needs to be reviewed and merged, mainly
>>>>>> regarding
>>>>>>> the upgrades. Some of the patches require Java 1.7, so I think we
>>>> should
>>>>>>> review OOZIE-2036 and merged it.
>>>>>>> 
>>>>>>> ~Abhishek
>>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>>> rohini.aditya@gmail.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
>>>>>> Available
>>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>>> 
>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>>> 
>>>>>>>> There are 42 jiras to go. We can take a look mid next week to see if
>>>>>> any
>>>>>>>> pending ones needs to be pushed out. There are quite a few which are
>>>>>>>> unassigned.
>>>>>>>> 
>>>>>>>> If there is any jira, someone would like to include in 4.3.0 please
>>>>>>> update
>>>>>>>> the jira.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Rohini
>>>>>>>> 
>>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>>> rohini.aditya@gmail.com
>>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Unset the open ones without sending emails. Will update the resolved,
>>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Rohini
>>>>>>>>> 
>>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rkanter@cloudera.com
>>>>> 
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we need
>>>> to
>>>>>>> put
>>>>>>>>>> in some effort to clean out old JIRAs and to review Patch Available
>>>>>>> JIRAs.
>>>>>>>>>> 
>>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
>>>>>>> discuss
>>>>>>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about
>>>>>>> why.
>>>>>>>>>> 
>>>>>>>>>> - Robert
>>>>>>>>>> 
>>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>>> 
>>>>>>>>>>> These are jiras open and marked trunk. Should we unset fix version
>>>>>> on
>>>>>>>>>> these
>>>>>>>>>>> or mark them for next release? Don't see majority of them being
>>>>>> worked
>>>>>>>>>> on
>>>>>>>>>>> even for next release. We need to do bug scrub and close many of
>>>>>> these
>>>>>>>>>> some
>>>>>>>>>>> time.
>>>>>>>>>>> 
>>>>>>>>>>>> Instead
>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>>>>>> the
>>>>>>>>>>> release number.
>>>>>>>>>>> There is some amount of relabeling either way. If we go with
>>>>>> unsetting
>>>>>>>>>> fix
>>>>>>>>>>> version, then we can as well delete trunk. Also, I did create a
>>>>>> 4.4.0
>>>>>>>>>>> yesterday to move some of the patch available jiras from 4.3.0.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> I think it would be better to send out email for the bulk change.
>>>>>> If
>>>>>>>>>>> there
>>>>>>>>>>>> are any issues or jiras wrongly marked someone can respond. We do
>>>>>>> that
>>>>>>>>>>> with
>>>>>>>>>>>> Pig and there are always couple of jiras that folks come back with
>>>>>>>>>>> feedback
>>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>>> rkanter@cloudera.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell
>>>>>> it
>>>>>>>>>> not
>>>>>>>>>>>>> to
>>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>>> rkanter@cloudera.com
>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because we're on
>>>>>>>>>> "master"
>>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>>>>>> confusing.
>>>>>>>>>>>>> Instead
>>>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it
>>>> to
>>>>>>>>>> the
>>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any current
>>>>>>>>>> "trunk"
>>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
>>>>>>>>>> release
>>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
>>>>>>>>>> rename it
>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
>>>>>>>>>> should
>>>>>>>>>>>>>> probably have a vote in the private list to make things official
>>>>>> as
>>>>>>>>>>>>> this is
>>>>>>>>>>>>>> a release-management-related change.  Can you send an email?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
>>>> Rohini's
>>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.  You're
>>>>>>>>>>>>> probably
>>>>>>>>>>>>>> best off using the bulk change feature instead of doing each
>>>> JIRA
>>>>>>>>>>>>>> individually.  I should have time to do a bunch of reviews this
>>>>>>>>>> week,
>>>>>>>>>>>>> I've
>>>>>>>>>>>>>> actually already started.  I should have some time to continue
>>>>>>>>>> doing
>>>>>>>>>>>>> more
>>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
>>>>>> getting
>>>>>>>>>>> these
>>>>>>>>>>>>>> done :(
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was
>>>>>>>>>> working
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>>  Can you mark all jiras already committed to trunk as 4.3.0
>>>>>>>>>> and
>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter
>>>>>> on
>>>>>>>>>>>>> those
>>>>>>>>>>>>>>>> and review instead of referring to a google doc. I can pick up
>>>>>>>>>>> 10-15
>>>>>>>>>>>>>>>> jiras for review end of this week. We should try to get
>>>>>>>>>> majority of
>>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
>>>>>> the
>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>>> for any pending critical issues and rolling the release
>>>>>>>>>> candidate.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk version
>>>> and
>>>>>>>>>>> use
>>>>>>>>>>>>>>>> release numbers when committing from the beginning like all
>>>>>>>>>> other
>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release version
>>>>>> for
>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk
>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> and listed all the things which we wanted to include in the
>>>>>>>>>>> release.
>>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of the
>>>>>>>>>> patches
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to review
>>>>>> more
>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and get
>>>>>>>>>>> committed.
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> request other contributors also, review some of the patches
>>>>>> and
>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>>> release date and start working accordingly. Because, If we
>>>>>> wait
>>>>>>>>>>> for
>>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it might take
>>>>>> a
>>>>>>>>>>> lot
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>> time. I am thinking we should target third week of August to
>>>>>>>>>>>>> complete
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I looked into the available patches and prepared the list. I
>>>>>>>>>>> have
>>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>>> scope content which we have discussed till now, into a word
>>>>>>>>>>>>> document.
>>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two
>>>>>>>>>> more
>>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Please take a look at the document (link below) and provide
>>>>>>>>>> your
>>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of
>>>> is
>>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
>>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
>>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
>>>>>>>>>> other
>>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
>>>>>>>>>>> list
>>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to
>>>> include
>>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
>>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?  I
>>>>>>>>>> currently
>>>>>>>>>>>>> see
>>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>>> fixes/improvements,
>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>>>>>> available
>>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
>>>> anything,
>>>>>>>>>>>>> please
>>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
>>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and commit.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and provide
>>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> ------------------------------------------------------------
>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
>>>>>>>>>> slow
>>>>>>>>>>> in
>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action
>>>> rerun
>>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
>>>>>>>>>>>>> variable
>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
>>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
>>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
>>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
>>>>>>>>>>> running
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
>>>>>>>>>> 90%
>>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
>>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
>>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
>>>>>>>>>> be
>>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
>>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return
>>>> port
>>>>>>>>>>> for
>>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
>>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>>>> downstream
>>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
>>>>>>>>>> API
>>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
>>>>>>>>>> minioozie
>>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
>>>>>>>>>> error
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
>>>>>>>>>> querying
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>>>> directories
>>>>>>>>>>> in
>>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
>>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
>>>>>>>>>> action
>>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
>>>>>>>>>> ensure
>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
>>>> completed
>>>>>>>>>>>>> output
>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
>>>>>>>>>>> kill
>>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
>>>>>>>>>> when
>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> ------------------------------------------------------------
>>>>>>> ---------------
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
>>>>>>>>>> to
>>>>>>>>>>> me
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> should be included into the release. If there are things,
>>>>>>>>>>> which
>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
>>>> progress
>>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
>>>>>>>>>>>>> working
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
>>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
>>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
>>>> curator
>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
>>>>>>>>>> for
>>>>>>>>>>>>> over
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
>>>>>>>>>> lines
>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation() throws
>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>    // Need to do this, else default db will not be
>>>>>>>>>>> created
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>>    // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
>>>>>>>>>> named
>>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
>>>>>>>>>> 0.14
>>>>>>>>>>> and
>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables()
>>>>>>>>>> throws
>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
>>>>>>>>>> and
>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
>>>>>>>>>> to
>>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
>>>>>>>>>>> about
>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
>>>>>>>>>> need
>>>>>>>>>>> to
>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
>>>>>>>>>> in
>>>>>>>>>>> a
>>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
>>>>>>>>>> it in
>>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
>>>>>>>>>> out
>>>>>>>>>>> in
>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>>>>>>>>> should
>>>>>>>>>>>>> leave
>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
>>>>>>>>>> in
>>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
>>>>>>>>>> should
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
>>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
>>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
>>>>>>>>>> 4.3.0.
>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
>>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
>>>>>>>>>>> trunk.
>>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
>>>>>>>>>>> Database
>>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
>>>>>>>>>> above
>>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
>>>>>>>>>> work
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
>>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hello All,

How is it looking now?

I think we should move to blocker and critical patch mode. If required, we can take couple of days extra there.

~Abhishek
> On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Sure Robert. We can use some more time.
> 
> ~Abhishek
>> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com> wrote:
>> 
>> Do you think we could wait one more week?  I've been really hammered with
>> some other work and there's more that I'd like to review here that I
>> haven't been able to get to.
>> 
>> thanks
>> - Robert
>> 
>> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>> 
>>> Sure. I guess we can one extra week and complete some more work.
>>> Thanks everyone for responding.
>>> 
>>> ~Abhishek
>>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <pu...@yahoo-inc.com.INVALID>
>>> wrote:
>>>> 
>>>> Yes. I also have some pending patches.
>>>> 
>>>>  On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>>> rkanter@cloudera.com> wrote:
>>>> 
>>>> 
>>>> Ya, I could use some more time as well.  I have some deadlines early next
>>>> week, which has kept me from doing more reviews.
>>>> 
>>>> thanks
>>>> - Robert
>>>> 
>>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>>> jaydeepmailv@gmail.com> wrote:
>>>> 
>>>>> Let see if we can wait for another week. So we can merge few more
>>> important
>>>>> jiras.  Few of them I personally want to get reviewed and merged.
>>>>> 
>>>>> Regards,
>>>>> Jaydeep
>>>>> 
>>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
>>>>> wrote:
>>>>> 
>>>>>> As previously discussed, we will be moving with whatever patches can be
>>>>>> reviewed/merged till end of this week.
>>>>>> 
>>>>>> But there are some which needs to be reviewed and merged, mainly
>>>>> regarding
>>>>>> the upgrades. Some of the patches require Java 1.7, so I think we
>>> should
>>>>>> review OOZIE-2036 and merged it.
>>>>>> 
>>>>>> ~Abhishek
>>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>>> rohini.aditya@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
>>>>> Available
>>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>>> 
>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>>> 
>>>>>>> There are 42 jiras to go. We can take a look mid next week to see if
>>>>> any
>>>>>>> pending ones needs to be pushed out. There are quite a few which are
>>>>>>> unassigned.
>>>>>>> 
>>>>>>> If there is any jira, someone would like to include in 4.3.0 please
>>>>>> update
>>>>>>> the jira.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Rohini
>>>>>>> 
>>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>>> rohini.aditya@gmail.com
>>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Unset the open ones without sending emails. Will update the resolved,
>>>>>>>> closed and Patch Available ones sending emails.
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Rohini
>>>>>>>> 
>>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rkanter@cloudera.com
>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we need
>>> to
>>>>>> put
>>>>>>>>> in some effort to clean out old JIRAs and to review Patch Available
>>>>>> JIRAs.
>>>>>>>>> 
>>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
>>>>>> discuss
>>>>>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about
>>>>>> why.
>>>>>>>>> 
>>>>>>>>> - Robert
>>>>>>>>> 
>>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>>> 20status%20DESC&startIndex=50
>>>>>>>>>> 
>>>>>>>>>> These are jiras open and marked trunk. Should we unset fix version
>>>>> on
>>>>>>>>> these
>>>>>>>>>> or mark them for next release? Don't see majority of them being
>>>>> worked
>>>>>>>>> on
>>>>>>>>>> even for next release. We need to do bug scrub and close many of
>>>>> these
>>>>>>>>> some
>>>>>>>>>> time.
>>>>>>>>>> 
>>>>>>>>>>> Instead
>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>>>>> the
>>>>>>>>>> release number.
>>>>>>>>>> There is some amount of relabeling either way. If we go with
>>>>> unsetting
>>>>>>>>> fix
>>>>>>>>>> version, then we can as well delete trunk. Also, I did create a
>>>>> 4.4.0
>>>>>>>>>> yesterday to move some of the patch available jiras from 4.3.0.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I think it would be better to send out email for the bulk change.
>>>>> If
>>>>>>>>>> there
>>>>>>>>>>> are any issues or jiras wrongly marked someone can respond. We do
>>>>>> that
>>>>>>>>>> with
>>>>>>>>>>> Pig and there are always couple of jiras that folks come back with
>>>>>>>>>> feedback
>>>>>>>>>>> on pulling back to release.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>>> rkanter@cloudera.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell
>>>>> it
>>>>>>>>> not
>>>>>>>>>>>> to
>>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>>> rkanter@cloudera.com
>>>>>>> 
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because we're on
>>>>>>>>> "master"
>>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>>>>> confusing.
>>>>>>>>>>>> Instead
>>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it
>>> to
>>>>>>>>> the
>>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any current
>>>>>>>>> "trunk"
>>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
>>>>>>>>> release
>>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
>>>>>>>>> rename it
>>>>>>>>>>>>> later.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
>>>>>>>>> should
>>>>>>>>>>>>> probably have a vote in the private list to make things official
>>>>> as
>>>>>>>>>>>> this is
>>>>>>>>>>>>> a release-management-related change.  Can you send an email?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
>>> Rohini's
>>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.  You're
>>>>>>>>>>>> probably
>>>>>>>>>>>>> best off using the bulk change feature instead of doing each
>>> JIRA
>>>>>>>>>>>>> individually.  I should have time to do a bunch of reviews this
>>>>>>>>> week,
>>>>>>>>>>>> I've
>>>>>>>>>>>>> actually already started.  I should have some time to continue
>>>>>>>>> doing
>>>>>>>>>>>> more
>>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
>>>>> getting
>>>>>>>>>> these
>>>>>>>>>>>>> done :(
>>>>>>>>>>>>> 
>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was
>>>>>>>>> working
>>>>>>>>>>>> on
>>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>>   Can you mark all jiras already committed to trunk as 4.3.0
>>>>>>>>> and
>>>>>>>>>>>> those
>>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter
>>>>> on
>>>>>>>>>>>> those
>>>>>>>>>>>>>>> and review instead of referring to a google doc. I can pick up
>>>>>>>>>> 10-15
>>>>>>>>>>>>>>> jiras for review end of this week. We should try to get
>>>>>>>>> majority of
>>>>>>>>>>>>>> patches
>>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
>>>>> the
>>>>>>>>>> 3rd
>>>>>>>>>>>>>> week
>>>>>>>>>>>>>>> for any pending critical issues and rolling the release
>>>>>>>>> candidate.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk version
>>> and
>>>>>>>>>> use
>>>>>>>>>>>>>>> release numbers when committing from the beginning like all
>>>>>>>>> other
>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release version
>>>>> for
>>>>>>>>>>>> every
>>>>>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk
>>>>>>>>>>>> version
>>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> and listed all the things which we wanted to include in the
>>>>>>>>>> release.
>>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of the
>>>>>>>>> patches
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to review
>>>>> more
>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and get
>>>>>>>>>> committed.
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> request other contributors also, review some of the patches
>>>>> and
>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>>>>> (approximate)
>>>>>>>>>>>>>>>> release date and start working accordingly. Because, If we
>>>>> wait
>>>>>>>>>> for
>>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it might take
>>>>> a
>>>>>>>>>> lot
>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>> time. I am thinking we should target third week of August to
>>>>>>>>>>>> complete
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I looked into the available patches and prepared the list. I
>>>>>>>>>> have
>>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>>> scope content which we have discussed till now, into a word
>>>>>>>>>>>> document.
>>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two
>>>>>>>>> more
>>>>>>>>>>>> Jira
>>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Please take a look at the document (link below) and provide
>>>>>>>>> your
>>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of
>>> is
>>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
>>>>>>>>>>>> sense to
>>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
>>>>>>>>> yet.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
>>>>>>>>> other
>>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
>>>>>>>>>> list
>>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to
>>> include
>>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
>>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>> you consider each of them when creating the list?  I
>>>>>>>>> currently
>>>>>>>>>>>> see
>>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>>> fixes/improvements,
>>>>>>>>>>>> given
>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>>>>> available
>>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
>>> anything,
>>>>>>>>>>>> please
>>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
>>>>>>>>>>>> Important
>>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>>> how many of these we will be able to review and commit.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I will starting looking into these issues and provide
>>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> ------------------------------------------------------------
>>>>>> ---------------
>>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
>>>>>>>>> slow
>>>>>>>>>> in
>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action
>>> rerun
>>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
>>>>>>>>>>>> variable
>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
>>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
>>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
>>>>>>>>> jar
>>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
>>>>>>>>>> running
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
>>>>>>>>> 90%
>>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
>>>>>>>>>> urls
>>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
>>>>>>>>>>>>>> information
>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
>>>>>>>>> be
>>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
>>>>>>>>> work
>>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
>>>>>>>>> log4j
>>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return
>>> port
>>>>>>>>>> for
>>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
>>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>>> downstream
>>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
>>>>>>>>> API
>>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
>>>>>>>>> minioozie
>>>>>>>>>>>> doc
>>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
>>>>>>>>> error
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
>>>>>>>>> querying
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>>> directories
>>>>>>>>>> in
>>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
>>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
>>>>>>>>> action
>>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
>>>>>>>>> ensure
>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
>>> completed
>>>>>>>>>>>> output
>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
>>>>>>>>>> kill
>>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
>>>>>>>>> when
>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> ------------------------------------------------------------
>>>>>> ---------------
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
>>>>>>>>> to
>>>>>>>>>> me
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> should be included into the release. If there are things,
>>>>>>>>>> which
>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
>>> progress
>>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
>>>>>>>>>>>> working
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
>>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
>>>>>>>>>>>> internally as
>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
>>> curator
>>>>>>>>>>>> version
>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
>>>>>>>>> for
>>>>>>>>>>>> over
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
>>>>>>>>> lines
>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation() throws
>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>     // Need to do this, else default db will not be
>>>>>>>>>> created
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>>     // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
>>>>>>>>> named
>>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
>>>>>>>>> 0.14
>>>>>>>>>> and
>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables()
>>>>>>>>> throws
>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars.
>>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
>>>>>>>>> and
>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
>>>>>>>>> to
>>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
>>>>>>>>>> about
>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
>>>>>>>>> need
>>>>>>>>>> to
>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
>>>>>>>>> in
>>>>>>>>>> a
>>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
>>>>>>>>> it in
>>>>>>>>>>>> 4.3
>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
>>>>>>>>> out
>>>>>>>>>> in
>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>>>>>>>> should
>>>>>>>>>>>> leave
>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
>>>>>>>>> in
>>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
>>>>>>>>> should
>>>>>>>>>>>> make
>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
>>>>>>>>>>>> time to
>>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
>>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
>>>>>>>>> 4.3.0.
>>>>>>>>>>>> This
>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
>>>>>>>>>>>> exclusions)
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
>>>>>>>>>> trunk.
>>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
>>>>>>>>>> Database
>>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
>>>>>>>>> above
>>>>>>>>>>>>>> points
>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
>>>>>>>>> work
>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
>>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Sure Robert. We can use some more time.

~Abhishek
> On Aug 19, 2016, at 7:13 AM, Robert Kanter <rk...@cloudera.com> wrote:
> 
> Do you think we could wait one more week?  I've been really hammered with
> some other work and there's more that I'd like to review here that I
> haven't been able to get to.
> 
> thanks
> - Robert
> 
> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> 
>> Sure. I guess we can one extra week and complete some more work.
>> Thanks everyone for responding.
>> 
>> ~Abhishek
>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <pu...@yahoo-inc.com.INVALID>
>> wrote:
>>> 
>>> Yes. I also have some pending patches.
>>> 
>>>   On Friday, August 12, 2016 2:11 AM, Robert Kanter <
>> rkanter@cloudera.com> wrote:
>>> 
>>> 
>>> Ya, I could use some more time as well.  I have some deadlines early next
>>> week, which has kept me from doing more reviews.
>>> 
>>> thanks
>>> - Robert
>>> 
>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
>>> jaydeepmailv@gmail.com> wrote:
>>> 
>>>> Let see if we can wait for another week. So we can merge few more
>> important
>>>> jiras.  Few of them I personally want to get reviewed and merged.
>>>> 
>>>> Regards,
>>>> Jaydeep
>>>> 
>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
>>>> wrote:
>>>> 
>>>>> As previously discussed, we will be moving with whatever patches can be
>>>>> reviewed/merged till end of this week.
>>>>> 
>>>>> But there are some which needs to be reviewed and merged, mainly
>>>> regarding
>>>>> the upgrades. Some of the patches require Java 1.7, so I think we
>> should
>>>>> review OOZIE-2036 and merged it.
>>>>> 
>>>>> ~Abhishek
>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>>>> rohini.aditya@gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
>>>> Available
>>>>>> jiras targeted for trunk now point to 4.4.0.
>>>>>> 
>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>>>> 
>>>>>> There are 42 jiras to go. We can take a look mid next week to see if
>>>> any
>>>>>> pending ones needs to be pushed out. There are quite a few which are
>>>>>> unassigned.
>>>>>> 
>>>>>> If there is any jira, someone would like to include in 4.3.0 please
>>>>> update
>>>>>> the jira.
>>>>>> 
>>>>>> Regards,
>>>>>> Rohini
>>>>>> 
>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>>>> rohini.aditya@gmail.com
>>>>>>> wrote:
>>>>>> 
>>>>>>> Unset the open ones without sending emails. Will update the resolved,
>>>>>>> closed and Patch Available ones sending emails.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Rohini
>>>>>>> 
>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rkanter@cloudera.com
>>> 
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we need
>> to
>>>>> put
>>>>>>>> in some effort to clean out old JIRAs and to review Patch Available
>>>>> JIRAs.
>>>>>>>> 
>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
>>>>> discuss
>>>>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about
>>>>> why.
>>>>>>>> 
>>>>>>>> - Robert
>>>>>>>> 
>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>>>> 20status%20DESC&startIndex=50
>>>>>>>>> 
>>>>>>>>> These are jiras open and marked trunk. Should we unset fix version
>>>> on
>>>>>>>> these
>>>>>>>>> or mark them for next release? Don't see majority of them being
>>>> worked
>>>>>>>> on
>>>>>>>>> even for next release. We need to do bug scrub and close many of
>>>> these
>>>>>>>> some
>>>>>>>>> time.
>>>>>>>>> 
>>>>>>>>>> Instead
>>>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>>>> the
>>>>>>>>> release number.
>>>>>>>>> There is some amount of relabeling either way. If we go with
>>>> unsetting
>>>>>>>> fix
>>>>>>>>> version, then we can as well delete trunk. Also, I did create a
>>>> 4.4.0
>>>>>>>>> yesterday to move some of the patch available jiras from 4.3.0.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> I think it would be better to send out email for the bulk change.
>>>> If
>>>>>>>>> there
>>>>>>>>>> are any issues or jiras wrongly marked someone can respond. We do
>>>>> that
>>>>>>>>> with
>>>>>>>>>> Pig and there are always couple of jiras that folks come back with
>>>>>>>>> feedback
>>>>>>>>>> on pulling back to release.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>>>> rkanter@cloudera.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell
>>>> it
>>>>>>>> not
>>>>>>>>>>> to
>>>>>>>>>>> send out an email for every change :)
>>>>>>>>>>> 
>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>>>> rkanter@cloudera.com
>>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Rohini, that sounds good to me.  Especially because we're on
>>>>>>>> "master"
>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>>>> confusing.
>>>>>>>>>>> Instead
>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it
>> to
>>>>>>>> the
>>>>>>>>>>>> release number.  Otherwise, we'll have to re-label any current
>>>>>>>> "trunk"
>>>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
>>>>>>>> release
>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
>>>>>>>> rename it
>>>>>>>>>>>> later.
>>>>>>>>>>>> 
>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
>>>>>>>> should
>>>>>>>>>>>> probably have a vote in the private list to make things official
>>>> as
>>>>>>>>>>> this is
>>>>>>>>>>>> a release-management-related change.  Can you send an email?
>>>>>>>>>>>> 
>>>>>>>>>>>> Abhishek, thanks for putting this all together.  I think
>> Rohini's
>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step.  You're
>>>>>>>>>>> probably
>>>>>>>>>>>> best off using the bulk change feature instead of doing each
>> JIRA
>>>>>>>>>>>> individually.  I should have time to do a bunch of reviews this
>>>>>>>> week,
>>>>>>>>>>> I've
>>>>>>>>>>>> actually already started.  I should have some time to continue
>>>>>>>> doing
>>>>>>>>>>> more
>>>>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
>>>> getting
>>>>>>>>> these
>>>>>>>>>>>> done :(
>>>>>>>>>>>> 
>>>>>>>>>>>> - Robert
>>>>>>>>>>>> 
>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was
>>>>>>>> working
>>>>>>>>>>> on
>>>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>>>    Can you mark all jiras already committed to trunk as 4.3.0
>>>>>>>> and
>>>>>>>>>>> those
>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter
>>>> on
>>>>>>>>>>> those
>>>>>>>>>>>>>> and review instead of referring to a google doc. I can pick up
>>>>>>>>> 10-15
>>>>>>>>>>>>>> jiras for review end of this week. We should try to get
>>>>>>>> majority of
>>>>>>>>>>>>> patches
>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
>>>> the
>>>>>>>>> 3rd
>>>>>>>>>>>>> week
>>>>>>>>>>>>>> for any pending critical issues and rolling the release
>>>>>>>> candidate.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Robert,
>>>>>>>>>>>>>>  I am thinking, that we should get rid of the trunk version
>> and
>>>>>>>>> use
>>>>>>>>>>>>>> release numbers when committing from the beginning like all
>>>>>>>> other
>>>>>>>>>>>>> projects
>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release version
>>>> for
>>>>>>>>>>> every
>>>>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk
>>>>>>>>>>> version
>>>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Rohini
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> and listed all the things which we wanted to include in the
>>>>>>>>> release.
>>>>>>>>>>>>> In the
>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of the
>>>>>>>> patches
>>>>>>>>>>> and
>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to review
>>>> more
>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and get
>>>>>>>>> committed.
>>>>>>>>>>> I
>>>>>>>>>>>>>>> request other contributors also, review some of the patches
>>>> and
>>>>>>>>>>> provide
>>>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>>>> (approximate)
>>>>>>>>>>>>>>> release date and start working accordingly. Because, If we
>>>> wait
>>>>>>>>> for
>>>>>>>>>>>>> all the
>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it might take
>>>> a
>>>>>>>>> lot
>>>>>>>>>>>>> more
>>>>>>>>>>>>>>> time. I am thinking we should target third week of August to
>>>>>>>>>>> complete
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I looked into the available patches and prepared the list. I
>>>>>>>>> have
>>>>>>>>>>>>> put,
>>>>>>>>>>>>>>> scope content which we have discussed till now, into a word
>>>>>>>>>>> document.
>>>>>>>>>>>>> (For
>>>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two
>>>>>>>> more
>>>>>>>>>>> Jira
>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Please take a look at the document (link below) and provide
>>>>>>>> your
>>>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of
>> is
>>>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
>>>>>>>>>>> sense to
>>>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
>>>>>>>> yet.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
>>>>>>>> other
>>>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
>>>>>>>>> list
>>>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to
>> include
>>>>>>>>>>>>> everything
>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
>>>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>> you consider each of them when creating the list?  I
>>>>>>>> currently
>>>>>>>>>>> see
>>>>>>>>>>>>> 97
>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>>>> fixes/improvements,
>>>>>>>>>>> given
>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>>>> available
>>>>>>>>>>>>> patches.
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
>> anything,
>>>>>>>>>>> please
>>>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
>>>>>>>>>>> Important
>>>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>>>> how many of these we will be able to review and commit.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I will starting looking into these issues and provide
>>>>>>>>> feedback.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> ------------------------------------------------------------
>>>>> ---------------
>>>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
>>>>>>>> slow
>>>>>>>>> in
>>>>>>>>>>>>> some
>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action
>> rerun
>>>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
>>>>>>>>>>> variable
>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
>>>>>>>>>>> failures
>>>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
>>>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
>>>>>>>> jar
>>>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
>>>>>>>>> running
>>>>>>>>>>> for
>>>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
>>>>>>>> 90%
>>>>>>>>>>> cpu
>>>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
>>>>>>>>> urls
>>>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
>>>>>>>>>>>>> information
>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
>>>>>>>> be
>>>>>>>>>>>>> handled
>>>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>>>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
>>>>>>>> work
>>>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
>>>>>>>> log4j
>>>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return
>> port
>>>>>>>>> for
>>>>>>>>>>>>> admin
>>>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
>>>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
>> downstream
>>>>>>>>>>>>> components
>>>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
>>>>>>>> API
>>>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
>>>>>>>> minioozie
>>>>>>>>>>> doc
>>>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
>>>>>>>> error
>>>>>>>>>>> and
>>>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
>>>>>>>> querying
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>>>> directories
>>>>>>>>> in
>>>>>>>>>>>>> /tmp
>>>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
>>>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
>>>>>>>> action
>>>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
>>>>>>>> ensure
>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
>> completed
>>>>>>>>>>> output
>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
>>>>>>>>> kill
>>>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
>>>>>>>> when
>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> ------------------------------------------------------------
>>>>> ---------------
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
>>>>>>>> to
>>>>>>>>> me
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> should be included into the release. If there are things,
>>>>>>>>> which
>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
>> progress
>>>>>>>>>>> status.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
>>>>>>>>>>> working
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
>>>>>>>>>>> HttpClient,
>>>>>>>>>>>>> Pig
>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
>>>>>>>>>>> internally as
>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
>> curator
>>>>>>>>>>> version
>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
>>>>>>>> for
>>>>>>>>>>> over
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
>>>>>>>> lines
>>>>>>>>> in
>>>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>>>> 
>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>>>>>>>>>>>>>>>>  public static void resetDefaultDBCreation() throws
>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>      // Need to do this, else default db will not be
>>>>>>>>> created
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>>>      // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
>>>>>>>> named
>>>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
>>>>>>>> 0.14
>>>>>>>>> and
>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>  public static void resetHiveConfStaticVariables()
>>>>>>>> throws
>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>>>  hiveConf.setIntVar(ConfVars.
>> HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
>>>>>>>> and
>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>  Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>>>>>>>>>>>>>>>>  fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
>>>>>>>> to
>>>>>>>>>>> 1.x.
>>>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
>>>>>>>>> about
>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
>>>>>>>> need
>>>>>>>>> to
>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
>>>>>>>> in
>>>>>>>>> a
>>>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
>>>>>>>> it in
>>>>>>>>>>> 4.3
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
>>>>>>>> out
>>>>>>>>> in
>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>>>>>>> should
>>>>>>>>>>> leave
>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
>>>>>>>> in
>>>>>>>>>>> trunk
>>>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
>>>>>>>> should
>>>>>>>>>>> make
>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
>>>>>>>>>>> time to
>>>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
>>>>>>>> release?
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
>>>>>>>> 4.3.0.
>>>>>>>>>>> This
>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
>>>>>>>>>>> provide
>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
>>>>>>>>>>> exclusions)
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
>>>>>>>>> trunk.
>>>>>>>>>>>>> Based
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
>>>>>>>>> Database
>>>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
>>>>>>>> above
>>>>>>>>>>>>> points
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
>>>>>>>> work
>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
>>>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

Do you think we could wait one more week?  I've been really hammered with
some other work and there's more that I'd like to review here that I
haven't been able to get to.

thanks
- Robert

On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <ba...@gmail.com>
wrote:

> Sure. I guess we can one extra week and complete some more work.
> Thanks everyone for responding.
>
> ~Abhishek
> > On Aug 12, 2016, at 2:46 PM, Purshotam Shah <pu...@yahoo-inc.com.INVALID>
> wrote:
> >
> > Yes. I also have some pending patches.
> >
> >    On Friday, August 12, 2016 2:11 AM, Robert Kanter <
> rkanter@cloudera.com> wrote:
> >
> >
> > Ya, I could use some more time as well.  I have some deadlines early next
> > week, which has kept me from doing more reviews.
> >
> > thanks
> > - Robert
> >
> > On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
> > jaydeepmailv@gmail.com> wrote:
> >
> >> Let see if we can wait for another week. So we can merge few more
> important
> >> jiras.  Few of them I personally want to get reviewed and merged.
> >>
> >> Regards,
> >> Jaydeep
> >>
> >> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
> >> wrote:
> >>
> >>> As previously discussed, we will be moving with whatever patches can be
> >>> reviewed/merged till end of this week.
> >>>
> >>> But there are some which needs to be reviewed and merged, mainly
> >> regarding
> >>> the upgrades. Some of the patches require Java 1.7, so I think we
> should
> >>> review OOZIE-2036 and merged it.
> >>>
> >>> ~Abhishek
> >>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
> >> rohini.aditya@gmail.com>
> >>> wrote:
> >>>>
> >>>> Cleaned up that list and now no jira refers to "trunk".  Patch
> >> Available
> >>>> jiras targeted for trunk now point to 4.4.0.
> >>>>
> >>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
> >>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
> >>>>
> >>>> There are 42 jiras to go. We can take a look mid next week to see if
> >> any
> >>>> pending ones needs to be pushed out. There are quite a few which are
> >>>> unassigned.
> >>>>
> >>>> If there is any jira, someone would like to include in 4.3.0 please
> >>> update
> >>>> the jira.
> >>>>
> >>>> Regards,
> >>>> Rohini
> >>>>
> >>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
> >>> rohini.aditya@gmail.com
> >>>>> wrote:
> >>>>
> >>>>> Unset the open ones without sending emails. Will update the resolved,
> >>>>> closed and Patch Available ones sending emails.
> >>>>>
> >>>>> Regards,
> >>>>> Rohini
> >>>>>
> >>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rkanter@cloudera.com
> >
> >>>>> wrote:
> >>>>>
> >>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we need
> to
> >>> put
> >>>>>> in some effort to clean out old JIRAs and to review Patch Available
> >>> JIRAs.
> >>>>>>
> >>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
> >>> discuss
> >>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about
> >>> why.
> >>>>>>
> >>>>>> - Robert
> >>>>>>
> >>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> >>>>>> rohini.aditya@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
> >>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
> >>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
> >>> 20status%20DESC&startIndex=50
> >>>>>>>
> >>>>>>> These are jiras open and marked trunk. Should we unset fix version
> >> on
> >>>>>> these
> >>>>>>> or mark them for next release? Don't see majority of them being
> >> worked
> >>>>>> on
> >>>>>>> even for next release. We need to do bug scrub and close many of
> >> these
> >>>>>> some
> >>>>>>> time.
> >>>>>>>
> >>>>>>>> Instead
> >>>>>>> of deleting "trunk", I think we're better off just renaming it to
> >> the
> >>>>>>> release number.
> >>>>>>> There is some amount of relabeling either way. If we go with
> >> unsetting
> >>>>>> fix
> >>>>>>> version, then we can as well delete trunk. Also, I did create a
> >> 4.4.0
> >>>>>>> yesterday to move some of the patch available jiras from 4.3.0.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> >>>>>>> rohini.aditya@gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> I think it would be better to send out email for the bulk change.
> >> If
> >>>>>>> there
> >>>>>>>> are any issues or jiras wrongly marked someone can respond. We do
> >>> that
> >>>>>>> with
> >>>>>>>> Pig and there are always couple of jiras that folks come back with
> >>>>>>> feedback
> >>>>>>>> on pulling back to release.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
> >> rkanter@cloudera.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell
> >> it
> >>>>>> not
> >>>>>>>>> to
> >>>>>>>>> send out an email for every change :)
> >>>>>>>>>
> >>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
> >> rkanter@cloudera.com
> >>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Rohini, that sounds good to me.  Especially because we're on
> >>>>>> "master"
> >>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
> >> confusing.
> >>>>>>>>> Instead
> >>>>>>>>>> of deleting "trunk", I think we're better off just renaming it
> to
> >>>>>> the
> >>>>>>>>>> release number.  Otherwise, we'll have to re-label any current
> >>>>>> "trunk"
> >>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
> >>>>>> release
> >>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
> >>>>>> rename it
> >>>>>>>>>> later.
> >>>>>>>>>>
> >>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
> >>>>>> should
> >>>>>>>>>> probably have a vote in the private list to make things official
> >> as
> >>>>>>>>> this is
> >>>>>>>>>> a release-management-related change.  Can you send an email?
> >>>>>>>>>>
> >>>>>>>>>> Abhishek, thanks for putting this all together.  I think
> Rohini's
> >>>>>>>>>> suggestion of updating the JIRAs is the right next step.  You're
> >>>>>>>>> probably
> >>>>>>>>>> best off using the bulk change feature instead of doing each
> JIRA
> >>>>>>>>>> individually.  I should have time to do a bunch of reviews this
> >>>>>> week,
> >>>>>>>>> I've
> >>>>>>>>>> actually already started.  I should have some time to continue
> >>>>>> doing
> >>>>>>>>> more
> >>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
> >> getting
> >>>>>>> these
> >>>>>>>>>> done :(
> >>>>>>>>>>
> >>>>>>>>>> - Robert
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> >>>>>>>>>> rohini.aditya@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was
> >>>>>> working
> >>>>>>>>> on
> >>>>>>>>>>> and which might take time to review to 4.4.0
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>> Rohini
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> >>>>>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Abhishek,
> >>>>>>>>>>>>     Can you mark all jiras already committed to trunk as 4.3.0
> >>>>>> and
> >>>>>>>>> those
> >>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter
> >> on
> >>>>>>>>> those
> >>>>>>>>>>>> and review instead of referring to a google doc. I can pick up
> >>>>>>> 10-15
> >>>>>>>>>>>> jiras for review end of this week. We should try to get
> >>>>>> majority of
> >>>>>>>>>>> patches
> >>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
> >> the
> >>>>>>> 3rd
> >>>>>>>>>>> week
> >>>>>>>>>>>> for any pending critical issues and rolling the release
> >>>>>> candidate.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Robert,
> >>>>>>>>>>>>   I am thinking, that we should get rid of the trunk version
> and
> >>>>>>> use
> >>>>>>>>>>>> release numbers when committing from the beginning like all
> >>>>>> other
> >>>>>>>>>>> projects
> >>>>>>>>>>>> do. Unnecessary step to change from trunk to release version
> >> for
> >>>>>>>>> every
> >>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk
> >>>>>>>>> version
> >>>>>>>>>>>> altogether. Does that require a mail to private?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regards,
> >>>>>>>>>>>> Rohini
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> >>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >>>>>>>>>>>>
> >>>>>>>>>>>>> and listed all the things which we wanted to include in the
> >>>>>>> release.
> >>>>>>>>>>> In the
> >>>>>>>>>>>>> last few days, I spent some time and reviewed some of the
> >>>>>> patches
> >>>>>>>>> and
> >>>>>>>>>>>>> provided my feedback. I will continue my efforts to review
> >> more
> >>>>>>>>>>> patches.
> >>>>>>>>>>>>> These patches needs to be reviewed by committers and get
> >>>>>>> committed.
> >>>>>>>>> I
> >>>>>>>>>>>>> request other contributors also, review some of the patches
> >> and
> >>>>>>>>> provide
> >>>>>>>>>>>>> their feedback.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
> >>>>>> (approximate)
> >>>>>>>>>>>>> release date and start working accordingly. Because, If we
> >> wait
> >>>>>>> for
> >>>>>>>>>>> all the
> >>>>>>>>>>>>> listed items to be reviewed and committed, then it might take
> >> a
> >>>>>>> lot
> >>>>>>>>>>> more
> >>>>>>>>>>>>> time. I am thinking we should target third week of August to
> >>>>>>>>> complete
> >>>>>>>>>>> and
> >>>>>>>>>>>>> release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> >>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I looked into the available patches and prepared the list. I
> >>>>>>> have
> >>>>>>>>>>> put,
> >>>>>>>>>>>>> scope content which we have discussed till now, into a word
> >>>>>>>>> document.
> >>>>>>>>>>> (For
> >>>>>>>>>>>>> the better formatting and easy to read and comment).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two
> >>>>>> more
> >>>>>>>>> Jira
> >>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Please take a look at the document (link below) and provide
> >>>>>> your
> >>>>>>>>>>>>> comments.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If this looks good, we can start the release process.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Scope Document:
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> >>> mFsuHQZmrT63Ow/edit#
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> >>>>>>> rkanter@cloudera.com
> >>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of
> is
> >>>>>>>>>>> OOZIE-2306
> >>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
> >>>>>>>>> sense to
> >>>>>>>>>>>>> let it
> >>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
> >>>>>> yet.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'd also like to add a few others to the list:
> >>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> >>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> >>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> >>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
> >>>>>> other
> >>>>>>>>>>> fixes)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
> >>>>>>> list
> >>>>>>>>>>>>> (including
> >>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to
> include
> >>>>>>>>>>> everything
> >>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
> >>>>>>>>>>> bandwidth),
> >>>>>>>>>>>>> did
> >>>>>>>>>>>>>>> you consider each of them when creating the list?  I
> >>>>>> currently
> >>>>>>>>> see
> >>>>>>>>>>> 97
> >>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> >>>>>>>>>>>>>>> I think we should include all Spark Action
> >>>>>> fixes/improvements,
> >>>>>>>>> given
> >>>>>>>>>>>>> how
> >>>>>>>>>>>>>>> popular it seems to be.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> >>>>>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
> >>>>>> available
> >>>>>>>>>>> patches.
> >>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is
> anything,
> >>>>>>>>> please
> >>>>>>>>>>>>> reply
> >>>>>>>>>>>>>>>> here.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
> >>>>>>>>> Important
> >>>>>>>>>>>>> is,
> >>>>>>>>>>>>>>>> how many of these we will be able to review and commit.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I will starting looking into these issues and provide
> >>>>>>> feedback.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>> ------------------------------------------------------------
> >>> ---------------
> >>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
> >>>>>> slow
> >>>>>>> in
> >>>>>>>>>>> some
> >>>>>>>>>>>>> cases
> >>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action
> rerun
> >>>>>>>>>>>>> [OOZIE-1735]
> >>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
> >>>>>>>>> variable
> >>>>>>>>>>> name
> >>>>>>>>>>>>>>>> containing dot
> >>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
> >>>>>>>>> failures
> >>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
> >>>>>>>>> endCreatedTime
> >>>>>>>>>>>>>>>> filters for coord and bundles
> >>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
> >>>>>> jar
> >>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
> >>>>>>> running
> >>>>>>>>> for
> >>>>>>>>>>>>> longer
> >>>>>>>>>>>>>>>> than expected time
> >>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
> >>>>>> 90%
> >>>>>>>>> cpu
> >>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
> >>>>>>> urls
> >>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
> >>>>>>>>>>> information
> >>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>> queued commnads
> >>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
> >>>>>> be
> >>>>>>>>>>> handled
> >>>>>>>>>>>>>>>> rightly
> >>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
> >>>>>>>>>>> mapred-site
> >>>>>>>>>>>>>>>> into spark launcher
> >>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
> >>>>>> work
> >>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
> >>>>>> log4j
> >>>>>>>>>>>>>>>> configuration file for execution mode'
> >>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return
> port
> >>>>>>> for
> >>>>>>>>>>> admin
> >>>>>>>>>>>>>>>> -servers
> >>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
> >>>>>>>>>>> endtime/pausetime
> >>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
> >>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for
> downstream
> >>>>>>>>>>> components
> >>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
> >>>>>> API
> >>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
> >>>>>> minioozie
> >>>>>>>>> doc
> >>>>>>>>>>>>> does not
> >>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
> >>>>>> error
> >>>>>>>>> and
> >>>>>>>>>>>>> audit
> >>>>>>>>>>>>>>>> logs
> >>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
> >>>>>> querying
> >>>>>>>>>>> should
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>>>> last modified filter
> >>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
> >>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
> >>>>>> directories
> >>>>>>> in
> >>>>>>>>>>> /tmp
> >>>>>>>>>>>>> per
> >>>>>>>>>>>>>>>> day
> >>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> >>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
> >>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
> >>>>>> action
> >>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
> >>>>>> ensure
> >>>>>>>>>>>>> uploaded
> >>>>>>>>>>>>>>>> jars are world readable
> >>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing
> completed
> >>>>>>>>> output
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
> >>>>>>> kill
> >>>>>>>>>>>>> control
> >>>>>>>>>>>>>>>> node cannot resolve variable in message
> >>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
> >>>>>> when
> >>>>>>>>>>> logging
> >>>>>>>>>>>>>>>> command arguments
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>> ------------------------------------------------------------
> >>> ---------------
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> >>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
> >>>>>> to
> >>>>>>> me
> >>>>>>>>> and
> >>>>>>>>>>>>>>>> should be included into the release. If there are things,
> >>>>>>> which
> >>>>>>>>>>> needs
> >>>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>> excluded, provide your comments.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We still need to decide on the patch which are in
> progress
> >>>>>>>>> status.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On the upgrade side:
> >>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
> >>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
> >>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
> >>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
> >>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
> >>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
> >>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
> >>>>>>>>> working
> >>>>>>>>>>> on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> completing/testing the release.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> >>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
> >>>>>>>>> HttpClient,
> >>>>>>>>>>> Pig
> >>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
> >>>>>>>>> internally as
> >>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>,
> curator
> >>>>>>>>> version
> >>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>> can be upgraded to 2.11.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> >>>>>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
> >>>>>> for
> >>>>>>>>> over
> >>>>>>>>>>> a
> >>>>>>>>>>>>> year
> >>>>>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
> >>>>>> lines
> >>>>>>> in
> >>>>>>>>>>>>>>>> MiniHCatServer
> >>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>>>>
> >>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
> >>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> >>>>>>>>>>>>>>>>>>>   public static void resetDefaultDBCreation() throws
> >>>>>>>>> Exception {
> >>>>>>>>>>>>>>>>>>>       // Need to do this, else default db will not be
> >>>>>>> created
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>>> local
> >>>>>>>>>>>>>>>>>>> metastores.
> >>>>>>>>>>>>>>>>>>>       // TestHiveMain will fail with
> >>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
> >>>>>> named
> >>>>>>>>>>> default)
> >>>>>>>>>>>>>>>>>>> -        Field declaredField =
> >>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
> >>>>>> 0.14
> >>>>>>> and
> >>>>>>>>>>> later
> >>>>>>>>>>>>>>>>>>> +        //Field declaredField =
> >>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
> >>>>>>>>>>>>>>>>>>>   }
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>   public static void resetHiveConfStaticVariables()
> >>>>>> throws
> >>>>>>>>>>>>> Exception
> >>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>>>>>
> >>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
> >>>>>>>>>>>>>>>>>>> ---
> >>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>>>>> +++
> >>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> >>>>>>>>>>> AbstractHiveService
> >>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>>>>
> >>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> >>>>>>>>>>>>> getHost());
> >>>>>>>>>>>>>>>>>>>   hiveConf.setIntVar(ConfVars.
> HIVE_SERVER2_THRIFT_PORT,
> >>>>>>>>>>>>>>>> getBinaryPort());
> >>>>>>>>>>>>>>>>>>>
> >>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >>>>>>>>>>>>>>>>>>> getHttpPort());
> >>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
> >>>>>> and
> >>>>>>>>> later
> >>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
> >>>>>>>>>>>>>>>>>>>   fs.mkdirs(scratchDir);
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> >>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
> >>>>>>>>>>> vranganathan@hortonworks.com
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
> >>>>>> to
> >>>>>>>>> 1.x.
> >>>>>>>>>>>>> 0.13
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> old
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Venkat
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> >>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>> <mailto:
> >>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
> >>>>>>> about
> >>>>>>>>>>> when
> >>>>>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>>>> okay
> >>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
> >>>>>> need
> >>>>>>> to
> >>>>>>>>>>> wait
> >>>>>>>>>>>>>>>> until
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
> >>>>>> in
> >>>>>>> a
> >>>>>>>>>>> minor
> >>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
> >>>>>> it in
> >>>>>>>>> 4.3
> >>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>>> nobody has
> >>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
> >>>>>> out
> >>>>>>> in
> >>>>>>>>>>> our
> >>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>> notes.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
> >>>>>> should
> >>>>>>>>> leave
> >>>>>>>>>>> out
> >>>>>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
> >>>>>> in
> >>>>>>>>> trunk
> >>>>>>>>>>>>>>>> anyway.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
> >>>>>> should
> >>>>>>>>> make
> >>>>>>>>>>>>> sure
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
> >>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
> >>>>>>>>> time to
> >>>>>>>>>>>>> bake
> >>>>>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> >>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
> >>>>>> release?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> >>>>>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>>>>> <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
> >>>>>> 4.3.0.
> >>>>>>>>> This
> >>>>>>>>>>>>> thread
> >>>>>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
> >>>>>>>>> provide
> >>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
> >>>>>>>>> exclusions)
> >>>>>>>>>>> in
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
> >>>>>>> trunk.
> >>>>>>>>>>> Based
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
> >>>>>>>>>>>>>>>>>>>>>> Version Upgrade
> >>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> >>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> >>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
> >>>>>>>>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
> >>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> >>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
> >>>>>>>>>>>>>>>>>>>>> ).
> >>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
> >>>>>>> Database
> >>>>>>>>>>>>> schema
> >>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
> >>>>>> etc.
> >>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
> >>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
> >>>>>> above
> >>>>>>>>>>> points
> >>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
> >>>>>> work
> >>>>>>>>>>> which
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
> >>>>>>>>>>>>> availability
> >>>>>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>>>>> reviewers).
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
> >
> >
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Sure. I guess we can one extra week and complete some more work.
Thanks everyone for responding.

~Abhishek
> On Aug 12, 2016, at 2:46 PM, Purshotam Shah <pu...@yahoo-inc.com.INVALID> wrote:
> 
> Yes. I also have some pending patches. 
> 
>    On Friday, August 12, 2016 2:11 AM, Robert Kanter <rk...@cloudera.com> wrote:
> 
> 
> Ya, I could use some more time as well.  I have some deadlines early next
> week, which has kept me from doing more reviews.
> 
> thanks
> - Robert
> 
> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
> jaydeepmailv@gmail.com> wrote:
> 
>> Let see if we can wait for another week. So we can merge few more important
>> jiras.  Few of them I personally want to get reviewed and merged.
>> 
>> Regards,
>> Jaydeep
>> 
>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>> 
>>> As previously discussed, we will be moving with whatever patches can be
>>> reviewed/merged till end of this week.
>>> 
>>> But there are some which needs to be reviewed and merged, mainly
>> regarding
>>> the upgrades. Some of the patches require Java 1.7, so I think we should
>>> review OOZIE-2036 and merged it.
>>> 
>>> ~Abhishek
>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
>> rohini.aditya@gmail.com>
>>> wrote:
>>>> 
>>>> Cleaned up that list and now no jira refers to "trunk".  Patch
>> Available
>>>> jiras targeted for trunk now point to 4.4.0.
>>>> 
>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
>>>> 
>>>> There are 42 jiras to go. We can take a look mid next week to see if
>> any
>>>> pending ones needs to be pushed out. There are quite a few which are
>>>> unassigned.
>>>> 
>>>> If there is any jira, someone would like to include in 4.3.0 please
>>> update
>>>> the jira.
>>>> 
>>>> Regards,
>>>> Rohini
>>>> 
>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
>>> rohini.aditya@gmail.com
>>>>> wrote:
>>>> 
>>>>> Unset the open ones without sending emails. Will update the resolved,
>>>>> closed and Patch Available ones sending emails.
>>>>> 
>>>>> Regards,
>>>>> Rohini
>>>>> 
>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rk...@cloudera.com>
>>>>> wrote:
>>>>> 
>>>>>> Ya, let's unset those remaining open trunk ones.  I agree, we need to
>>> put
>>>>>> in some effort to clean out old JIRAs and to review Patch Available
>>> JIRAs.
>>>>>> 
>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
>>> discuss
>>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about
>>> why.
>>>>>> 
>>>>>> - Robert
>>>>>> 
>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>>>>> rohini.aditya@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> https://issues.apache.org/jira/issues/?jql=project%20%
>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
>>> 20status%20DESC&startIndex=50
>>>>>>> 
>>>>>>> These are jiras open and marked trunk. Should we unset fix version
>> on
>>>>>> these
>>>>>>> or mark them for next release? Don't see majority of them being
>> worked
>>>>>> on
>>>>>>> even for next release. We need to do bug scrub and close many of
>> these
>>>>>> some
>>>>>>> time.
>>>>>>> 
>>>>>>>> Instead
>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>> the
>>>>>>> release number.
>>>>>>> There is some amount of relabeling either way. If we go with
>> unsetting
>>>>>> fix
>>>>>>> version, then we can as well delete trunk. Also, I did create a
>> 4.4.0
>>>>>>> yesterday to move some of the patch available jiras from 4.3.0.
>>>>>>> 
>>>>>>> 
>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>>>>> rohini.aditya@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> I think it would be better to send out email for the bulk change.
>> If
>>>>>>> there
>>>>>>>> are any issues or jiras wrongly marked someone can respond. We do
>>> that
>>>>>>> with
>>>>>>>> Pig and there are always couple of jiras that folks come back with
>>>>>>> feedback
>>>>>>>> on pulling back to release.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
>> rkanter@cloudera.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell
>> it
>>>>>> not
>>>>>>>>> to
>>>>>>>>> send out an email for every change :)
>>>>>>>>> 
>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
>> rkanter@cloudera.com
>>>> 
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Rohini, that sounds good to me.  Especially because we're on
>>>>>> "master"
>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little
>> confusing.
>>>>>>>>> Instead
>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>>>>>> the
>>>>>>>>>> release number.  Otherwise, we'll have to re-label any current
>>>>>> "trunk"
>>>>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
>>>>>> release
>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
>>>>>> rename it
>>>>>>>>>> later.
>>>>>>>>>> 
>>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
>>>>>> should
>>>>>>>>>> probably have a vote in the private list to make things official
>> as
>>>>>>>>> this is
>>>>>>>>>> a release-management-related change.  Can you send an email?
>>>>>>>>>> 
>>>>>>>>>> Abhishek, thanks for putting this all together.  I think Rohini's
>>>>>>>>>> suggestion of updating the JIRAs is the right next step.  You're
>>>>>>>>> probably
>>>>>>>>>> best off using the bulk change feature instead of doing each JIRA
>>>>>>>>>> individually.  I should have time to do a bunch of reviews this
>>>>>> week,
>>>>>>>>> I've
>>>>>>>>>> actually already started.  I should have some time to continue
>>>>>> doing
>>>>>>>>> more
>>>>>>>>>> reviews.  As a general thing, we've been pretty slow about
>> getting
>>>>>>> these
>>>>>>>>>> done :(
>>>>>>>>>> 
>>>>>>>>>> - Robert
>>>>>>>>>> 
>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was
>>>>>> working
>>>>>>>>> on
>>>>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>>>>> 
>>>>>>>>>>> Regards,
>>>>>>>>>>> Rohini
>>>>>>>>>>> 
>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Abhishek,
>>>>>>>>>>>>     Can you mark all jiras already committed to trunk as 4.3.0
>>>>>> and
>>>>>>>>> those
>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter
>> on
>>>>>>>>> those
>>>>>>>>>>>> and review instead of referring to a google doc. I can pick up
>>>>>>> 10-15
>>>>>>>>>>>> jiras for review end of this week. We should try to get
>>>>>> majority of
>>>>>>>>>>> patches
>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
>> the
>>>>>>> 3rd
>>>>>>>>>>> week
>>>>>>>>>>>> for any pending critical issues and rolling the release
>>>>>> candidate.
>>>>>>>>>>>> 
>>>>>>>>>>>> Robert,
>>>>>>>>>>>>   I am thinking, that we should get rid of the trunk version and
>>>>>>> use
>>>>>>>>>>>> release numbers when committing from the beginning like all
>>>>>> other
>>>>>>>>>>> projects
>>>>>>>>>>>> do. Unnecessary step to change from trunk to release version
>> for
>>>>>>>>> every
>>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk
>>>>>>>>> version
>>>>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Rohini
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>>>>> 
>>>>>>>>>>>>> and listed all the things which we wanted to include in the
>>>>>>> release.
>>>>>>>>>>> In the
>>>>>>>>>>>>> last few days, I spent some time and reviewed some of the
>>>>>> patches
>>>>>>>>> and
>>>>>>>>>>>>> provided my feedback. I will continue my efforts to review
>> more
>>>>>>>>>>> patches.
>>>>>>>>>>>>> These patches needs to be reviewed by committers and get
>>>>>>> committed.
>>>>>>>>> I
>>>>>>>>>>>>> request other contributors also, review some of the patches
>> and
>>>>>>>>> provide
>>>>>>>>>>>>> their feedback.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>>>>> (approximate)
>>>>>>>>>>>>> release date and start working accordingly. Because, If we
>> wait
>>>>>>> for
>>>>>>>>>>> all the
>>>>>>>>>>>>> listed items to be reviewed and committed, then it might take
>> a
>>>>>>> lot
>>>>>>>>>>> more
>>>>>>>>>>>>> time. I am thinking we should target third week of August to
>>>>>>>>> complete
>>>>>>>>>>> and
>>>>>>>>>>>>> release.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I looked into the available patches and prepared the list. I
>>>>>>> have
>>>>>>>>>>> put,
>>>>>>>>>>>>> scope content which we have discussed till now, into a word
>>>>>>>>> document.
>>>>>>>>>>> (For
>>>>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two
>>>>>> more
>>>>>>>>> Jira
>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Please take a look at the document (link below) and provide
>>>>>> your
>>>>>>>>>>>>> comments.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Scope Document:
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
>>> mFsuHQZmrT63Ow/edit#
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>>>>> rkanter@cloudera.com
>>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of is
>>>>>>>>>>> OOZIE-2306
>>>>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
>>>>>>>>> sense to
>>>>>>>>>>>>> let it
>>>>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
>>>>>> yet.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
>>>>>> other
>>>>>>>>>>> fixes)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
>>>>>>> list
>>>>>>>>>>>>> (including
>>>>>>>>>>>>>>> the ones I mentioned above).  While we don't need to include
>>>>>>>>>>> everything
>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
>>>>>>>>>>> bandwidth),
>>>>>>>>>>>>> did
>>>>>>>>>>>>>>> you consider each of them when creating the list?  I
>>>>>> currently
>>>>>>>>> see
>>>>>>>>>>> 97
>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>>>>> I think we should include all Spark Action
>>>>>> fixes/improvements,
>>>>>>>>> given
>>>>>>>>>>>>> how
>>>>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>>>>> available
>>>>>>>>>>> patches.
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is anything,
>>>>>>>>> please
>>>>>>>>>>>>> reply
>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
>>>>>>>>> Important
>>>>>>>>>>>>> is,
>>>>>>>>>>>>>>>> how many of these we will be able to review and commit.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I will starting looking into these issues and provide
>>>>>>> feedback.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> ------------------------------------------------------------
>>> ---------------
>>>>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
>>>>>> slow
>>>>>>> in
>>>>>>>>>>> some
>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action rerun
>>>>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
>>>>>>>>> variable
>>>>>>>>>>> name
>>>>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
>>>>>>>>> failures
>>>>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
>>>>>>>>> endCreatedTime
>>>>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
>>>>>> jar
>>>>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
>>>>>>> running
>>>>>>>>> for
>>>>>>>>>>>>> longer
>>>>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
>>>>>> 90%
>>>>>>>>> cpu
>>>>>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
>>>>>>> urls
>>>>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
>>>>>>>>>>> information
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
>>>>>> be
>>>>>>>>>>> handled
>>>>>>>>>>>>>>>> rightly
>>>>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>>>>>>>>>>> mapred-site
>>>>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
>>>>>> work
>>>>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
>>>>>> log4j
>>>>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return port
>>>>>>> for
>>>>>>>>>>> admin
>>>>>>>>>>>>>>>> -servers
>>>>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
>>>>>>>>>>> endtime/pausetime
>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for downstream
>>>>>>>>>>> components
>>>>>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
>>>>>> API
>>>>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
>>>>>> minioozie
>>>>>>>>> doc
>>>>>>>>>>>>> does not
>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
>>>>>> error
>>>>>>>>> and
>>>>>>>>>>>>> audit
>>>>>>>>>>>>>>>> logs
>>>>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
>>>>>> querying
>>>>>>>>>>> should
>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>>>>> directories
>>>>>>> in
>>>>>>>>>>> /tmp
>>>>>>>>>>>>> per
>>>>>>>>>>>>>>>> day
>>>>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
>>>>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
>>>>>> action
>>>>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
>>>>>> ensure
>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing completed
>>>>>>>>> output
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
>>>>>>> kill
>>>>>>>>>>>>> control
>>>>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
>>>>>> when
>>>>>>>>>>> logging
>>>>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> ------------------------------------------------------------
>>> ---------------
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
>>>>>> to
>>>>>>> me
>>>>>>>>> and
>>>>>>>>>>>>>>>> should be included into the release. If there are things,
>>>>>>> which
>>>>>>>>>>> needs
>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in progress
>>>>>>>>> status.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
>>>>>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
>>>>>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
>>>>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
>>>>>>>>> working
>>>>>>>>>>> on
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
>>>>>>>>> HttpClient,
>>>>>>>>>>> Pig
>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
>>>>>>>>> internally as
>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
>>>>>>>>> version
>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
>>>>>> for
>>>>>>>>> over
>>>>>>>>>>> a
>>>>>>>>>>>>> year
>>>>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
>>>>>> lines
>>>>>>> in
>>>>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>>>>> 
>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>>>>>>>>>>>>>>   public static void resetDefaultDBCreation() throws
>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>       // Need to do this, else default db will not be
>>>>>>> created
>>>>>>>>>>> for
>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>>>>       // TestHiveMain will fail with
>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
>>>>>> named
>>>>>>>>>>> default)
>>>>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
>>>>>> 0.14
>>>>>>> and
>>>>>>>>>>> later
>>>>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>>>>   }
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>   public static void resetHiveConfStaticVariables()
>>>>>> throws
>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>> 
>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>> +++
>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>>>>> AbstractHiveService
>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>> 
>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>>>>> 
>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
>>>>>> and
>>>>>>>>> later
>>>>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>>>>>>>>>>>>>>   fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
>>>>>> to
>>>>>>>>> 1.x.
>>>>>>>>>>>>> 0.13
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>> <mailto:
>>>>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
>>>>>>> about
>>>>>>>>>>> when
>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
>>>>>> need
>>>>>>> to
>>>>>>>>>>> wait
>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
>>>>>> in
>>>>>>> a
>>>>>>>>>>> minor
>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
>>>>>> it in
>>>>>>>>> 4.3
>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
>>>>>> out
>>>>>>> in
>>>>>>>>>>> our
>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>>>>> should
>>>>>>>>> leave
>>>>>>>>>>> out
>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
>>>>>> in
>>>>>>>>> trunk
>>>>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
>>>>>> should
>>>>>>>>> make
>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
>>>>>>>>> time to
>>>>>>>>>>>>> bake
>>>>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>>>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
>>>>>> release?
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
>>>>>> 4.3.0.
>>>>>>>>> This
>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
>>>>>>>>> provide
>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
>>>>>>>>> exclusions)
>>>>>>>>>>> in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
>>>>>>> trunk.
>>>>>>>>>>> Based
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
>>>>>>> Database
>>>>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
>>>>>> above
>>>>>>>>>>> points
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
>>>>>> work
>>>>>>>>>>> which
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
>>>>>>>>>>>>> availability
>>>>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>> 
>>> 
>> 
> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Purshotam Shah <pu...@yahoo-inc.com.INVALID>]

Yes. I also have some pending patches. 

    On Friday, August 12, 2016 2:11 AM, Robert Kanter <rk...@cloudera.com> wrote:
 

 Ya, I could use some more time as well.  I have some deadlines early next
week, which has kept me from doing more reviews.

thanks
- Robert

On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
jaydeepmailv@gmail.com> wrote:

> Let see if we can wait for another week. So we can merge few more important
> jiras.  Few of them I personally want to get reviewed and merged.
>
> Regards,
> Jaydeep
>
> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
> wrote:
>
> > As previously discussed, we will be moving with whatever patches can be
> > reviewed/merged till end of this week.
> >
> > But there are some which needs to be reviewed and merged, mainly
> regarding
> > the upgrades. Some of the patches require Java 1.7, so I think we should
> > review OOZIE-2036 and merged it.
> >
> > ~Abhishek
> > > On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
> rohini.aditya@gmail.com>
> > wrote:
> > >
> > > Cleaned up that list and now no jira refers to "trunk".  Patch
> Available
> > > jiras targeted for trunk now point to 4.4.0.
> > >
> > > https://issues.apache.org/jira/issues/?jql=project%20%
> > 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
> > 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
> > >
> > > There are 42 jiras to go. We can take a look mid next week to see if
> any
> > > pending ones needs to be pushed out. There are quite a few which are
> > > unassigned.
> > >
> > > If there is any jira, someone would like to include in 4.3.0 please
> > update
> > > the jira.
> > >
> > > Regards,
> > > Rohini
> > >
> > > On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
> > rohini.aditya@gmail.com
> > >> wrote:
> > >
> > >> Unset the open ones without sending emails. Will update the resolved,
> > >> closed and Patch Available ones sending emails.
> > >>
> > >> Regards,
> > >> Rohini
> > >>
> > >> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rk...@cloudera.com>
> > >> wrote:
> > >>
> > >>> Ya, let's unset those remaining open trunk ones.  I agree, we need to
> > put
> > >>> in some effort to clean out old JIRAs and to review Patch Available
> > JIRAs.
> > >>>
> > >>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
> > discuss
> > >>> the issue more there, but I listed a few reasons on OOZIE-2625 about
> > why.
> > >>>
> > >>> - Robert
> > >>>
> > >>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> > >>> rohini.aditya@gmail.com>
> > >>> wrote:
> > >>>
> > >>>>
> > >>>>
> > >>> https://issues.apache.org/jira/issues/?jql=project%20%
> > 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
> > 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
> > 20status%20DESC&startIndex=50
> > >>>>
> > >>>> These are jiras open and marked trunk. Should we unset fix version
> on
> > >>> these
> > >>>> or mark them for next release? Don't see majority of them being
> worked
> > >>> on
> > >>>> even for next release. We need to do bug scrub and close many of
> these
> > >>> some
> > >>>> time.
> > >>>>
> > >>>>> Instead
> > >>>> of deleting "trunk", I think we're better off just renaming it to
> the
> > >>>> release number.
> > >>>> There is some amount of relabeling either way. If we go with
> unsetting
> > >>> fix
> > >>>> version, then we can as well delete trunk. Also, I did create a
> 4.4.0
> > >>>> yesterday to move some of the patch available jiras from 4.3.0.
> > >>>>
> > >>>>
> > >>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> > >>>> rohini.aditya@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>> I think it would be better to send out email for the bulk change.
> If
> > >>>> there
> > >>>>> are any issues or jiras wrongly marked someone can respond. We do
> > that
> > >>>> with
> > >>>>> Pig and there are always couple of jiras that folks come back with
> > >>>> feedback
> > >>>>> on pulling back to release.
> > >>>>>
> > >>>>>
> > >>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
> rkanter@cloudera.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Abhishek, next time you do a bulk change, please remember to tell
> it
> > >>> not
> > >>>>>> to
> > >>>>>> send out an email for every change :)
> > >>>>>>
> > >>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
> rkanter@cloudera.com
> > >
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Rohini, that sounds good to me.  Especially because we're on
> > >>> "master"
> > >>>>>>> branch but "trunk" version in JIRA and it gets a little
> confusing.
> > >>>>>> Instead
> > >>>>>>> of deleting "trunk", I think we're better off just renaming it to
> > >>> the
> > >>>>>>> release number.  Otherwise, we'll have to re-label any current
> > >>> "trunk"
> > >>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
> > >>> release
> > >>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
> > >>> rename it
> > >>>>>>> later.
> > >>>>>>>
> > >>>>>>> I'm not sure if there's a specific rule here, but I suppose we
> > >>> should
> > >>>>>>> probably have a vote in the private list to make things official
> as
> > >>>>>> this is
> > >>>>>>> a release-management-related change.  Can you send an email?
> > >>>>>>>
> > >>>>>>> Abhishek, thanks for putting this all together.  I think Rohini's
> > >>>>>>> suggestion of updating the JIRAs is the right next step.  You're
> > >>>>>> probably
> > >>>>>>> best off using the bulk change feature instead of doing each JIRA
> > >>>>>>> individually.  I should have time to do a bunch of reviews this
> > >>> week,
> > >>>>>> I've
> > >>>>>>> actually already started.  I should have some time to continue
> > >>> doing
> > >>>>>> more
> > >>>>>>> reviews.  As a general thing, we've been pretty slow about
> getting
> > >>>> these
> > >>>>>>> done :(
> > >>>>>>>
> > >>>>>>> - Robert
> > >>>>>>>
> > >>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> > >>>>>>> rohini.aditya@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> I also did mark some of the enhancement jiras that Satish was
> > >>> working
> > >>>>>> on
> > >>>>>>>> and which might take time to review to 4.4.0
> > >>>>>>>>
> > >>>>>>>> Regards,
> > >>>>>>>> Rohini
> > >>>>>>>>
> > >>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> > >>>>>>>> rohini.aditya@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Abhishek,
> > >>>>>>>>>    Can you mark all jiras already committed to trunk as 4.3.0
> > >>> and
> > >>>>>> those
> > >>>>>>>>> planned in the scope document as 4.3.0? This will help filter
> on
> > >>>>>> those
> > >>>>>>>>> and review instead of referring to a google doc. I can pick up
> > >>>> 10-15
> > >>>>>>>>> jiras for review end of this week. We should try to get
> > >>> majority of
> > >>>>>>>> patches
> > >>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
> the
> > >>>> 3rd
> > >>>>>>>> week
> > >>>>>>>>> for any pending critical issues and rolling the release
> > >>> candidate.
> > >>>>>>>>>
> > >>>>>>>>> Robert,
> > >>>>>>>>>  I am thinking, that we should get rid of the trunk version and
> > >>>> use
> > >>>>>>>>> release numbers when committing from the beginning like all
> > >>> other
> > >>>>>>>> projects
> > >>>>>>>>> do. Unnecessary step to change from trunk to release version
> for
> > >>>>>> every
> > >>>>>>>>> release. If folks are in agreement, then will delete the trunk
> > >>>>>> version
> > >>>>>>>>> altogether. Does that require a mail to private?
> > >>>>>>>>>
> > >>>>>>>>> Regards,
> > >>>>>>>>> Rohini
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> > >>>>>> bafna.iitr@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi All,
> > >>>>>>>>>>
> > >>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> > mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> > >>>>>>>>>
> > >>>>>>>>>> and listed all the things which we wanted to include in the
> > >>>> release.
> > >>>>>>>> In the
> > >>>>>>>>>> last few days, I spent some time and reviewed some of the
> > >>> patches
> > >>>>>> and
> > >>>>>>>>>> provided my feedback. I will continue my efforts to review
> more
> > >>>>>>>> patches.
> > >>>>>>>>>> These patches needs to be reviewed by committers and get
> > >>>> committed.
> > >>>>>> I
> > >>>>>>>>>> request other contributors also, review some of the patches
> and
> > >>>>>> provide
> > >>>>>>>>>> their feedback.
> > >>>>>>>>>>
> > >>>>>>>>>> I am thinking, may be, we can decide upon some fixed
> > >>> (approximate)
> > >>>>>>>>>> release date and start working accordingly. Because, If we
> wait
> > >>>> for
> > >>>>>>>> all the
> > >>>>>>>>>> listed items to be reviewed and committed, then it might take
> a
> > >>>> lot
> > >>>>>>>> more
> > >>>>>>>>>> time. I am thinking we should target third week of August to
> > >>>>>> complete
> > >>>>>>>> and
> > >>>>>>>>>> release.
> > >>>>>>>>>>
> > >>>>>>>>>> Feel free to provide your feedback and suggestions.
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> Abhishek
> > >>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> > >>>> bafna.iitr@gmail.com
> > >>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hi,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I looked into the available patches and prepared the list. I
> > >>>> have
> > >>>>>>>> put,
> > >>>>>>>>>> scope content which we have discussed till now, into a word
> > >>>>>> document.
> > >>>>>>>> (For
> > >>>>>>>>>> the better formatting and easy to read and comment).
> > >>>>>>>>>>>
> > >>>>>>>>>>> I have noted the additional Jira into the list. Added two
> > >>> more
> > >>>>>> Jira
> > >>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
> > >>>>>>>>>>>
> > >>>>>>>>>>> Please take a look at the document (link below) and provide
> > >>> your
> > >>>>>>>>>> comments.
> > >>>>>>>>>>>
> > >>>>>>>>>>> If this looks good, we can start the release process.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Scope Document:
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> > mFsuHQZmrT63Ow/edit#
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> Abhishek
> > >>>>>>>>>>>
> > >>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> > >>>> rkanter@cloudera.com
> > >>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of is
> > >>>>>>>> OOZIE-2306
> > >>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
> > >>>>>> sense to
> > >>>>>>>>>> let it
> > >>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
> > >>> yet.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I'd also like to add a few others to the list:
> > >>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> > >>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> > >>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> > >>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> > >>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
> > >>> other
> > >>>>>>>> fixes)
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
> > >>>> list
> > >>>>>>>>>> (including
> > >>>>>>>>>>>> the ones I mentioned above).  While we don't need to include
> > >>>>>>>> everything
> > >>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
> > >>>>>>>> bandwidth),
> > >>>>>>>>>> did
> > >>>>>>>>>>>> you consider each of them when creating the list?  I
> > >>> currently
> > >>>>>> see
> > >>>>>>>> 97
> > >>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> > >>>>>>>>>>>> I think we should include all Spark Action
> > >>> fixes/improvements,
> > >>>>>> given
> > >>>>>>>>>> how
> > >>>>>>>>>>>> popular it seems to be.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> thanks
> > >>>>>>>>>>>> - Robert
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> > >>>>>>>> bafna.iitr@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Below is the list of issues, picked from currently
> > >>> available
> > >>>>>>>> patches.
> > >>>>>>>>>> I
> > >>>>>>>>>>>>> could not decide up any 'open' issue, If there is anything,
> > >>>>>> please
> > >>>>>>>>>> reply
> > >>>>>>>>>>>>> here.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
> > >>>>>> Important
> > >>>>>>>>>> is,
> > >>>>>>>>>>>>> how many of these we will be able to review and commit.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I will starting looking into these issues and provide
> > >>>> feedback.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>> ------------------------------------------------------------
> > ---------------
> > >>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
> > >>> slow
> > >>>> in
> > >>>>>>>> some
> > >>>>>>>>>> cases
> > >>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action rerun
> > >>>>>>>>>> [OOZIE-1735]
> > >>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
> > >>>>>> variable
> > >>>>>>>> name
> > >>>>>>>>>>>>> containing dot
> > >>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
> > >>>>>> failures
> > >>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
> > >>>>>> endCreatedTime
> > >>>>>>>>>>>>> filters for coord and bundles
> > >>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
> > >>> jar
> > >>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
> > >>>> running
> > >>>>>> for
> > >>>>>>>>>> longer
> > >>>>>>>>>>>>> than expected time
> > >>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
> > >>> 90%
> > >>>>>> cpu
> > >>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
> > >>>> urls
> > >>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
> > >>>>>>>> information
> > >>>>>>>>>> about
> > >>>>>>>>>>>>> queued commnads
> > >>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
> > >>> be
> > >>>>>>>> handled
> > >>>>>>>>>>>>> rightly
> > >>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
> > >>>>>>>> mapred-site
> > >>>>>>>>>>>>> into spark launcher
> > >>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
> > >>> work
> > >>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
> > >>> log4j
> > >>>>>>>>>>>>> configuration file for execution mode'
> > >>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return port
> > >>>> for
> > >>>>>>>> admin
> > >>>>>>>>>>>>> -servers
> > >>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
> > >>>>>>>> endtime/pausetime
> > >>>>>>>>>> of a
> > >>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
> > >>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for downstream
> > >>>>>>>> components
> > >>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
> > >>> API
> > >>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
> > >>> minioozie
> > >>>>>> doc
> > >>>>>>>>>> does not
> > >>>>>>>>>>>>> work
> > >>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
> > >>> error
> > >>>>>> and
> > >>>>>>>>>> audit
> > >>>>>>>>>>>>> logs
> > >>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
> > >>> querying
> > >>>>>>>> should
> > >>>>>>>>>> have
> > >>>>>>>>>>>>> last modified filter
> > >>>>>>>>>>>>> OOZIE-2306      Oozie health check
> > >>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
> > >>> directories
> > >>>> in
> > >>>>>>>> /tmp
> > >>>>>>>>>> per
> > >>>>>>>>>>>>> day
> > >>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> > >>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
> > >>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
> > >>> action
> > >>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
> > >>> ensure
> > >>>>>>>>>> uploaded
> > >>>>>>>>>>>>> jars are world readable
> > >>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing completed
> > >>>>>> output
> > >>>>>>>>>> data
> > >>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
> > >>>> kill
> > >>>>>>>>>> control
> > >>>>>>>>>>>>> node cannot resolve variable in message
> > >>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
> > >>> when
> > >>>>>>>> logging
> > >>>>>>>>>>>>> command arguments
> > >>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>> ------------------------------------------------------------
> > ---------------
> > >>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> > >>>>>> bafna.iitr@gmail.com
> > >>>>>>>>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
> > >>> to
> > >>>> me
> > >>>>>> and
> > >>>>>>>>>>>>> should be included into the release. If there are things,
> > >>>> which
> > >>>>>>>> needs
> > >>>>>>>>>> to be
> > >>>>>>>>>>>>> excluded, provide your comments.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> We still need to decide on the patch which are in progress
> > >>>>>> status.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On the upgrade side:
> > >>>>>>>>>>>>>> Java version          1.6 to 1.7
> > >>>>>>>>>>>>>> OpenJPA version      2.2.2 to 2.4.1
> > >>>>>>>>>>>>>> Curator version      2.5.0 to 2.11.0
> > >>>>>>>>>>>>>> Pig version          0.12.1 to 0.16.0
> > >>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
> > >>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
> > >>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
> > >>>>>> working
> > >>>>>>>> on
> > >>>>>>>>>> the
> > >>>>>>>>>>>>> completing/testing the release.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> > >>>>>>>> bafna.iitr@gmail.com
> > >>>>>>>>>>>>> <ma...@gmail.com>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
> > >>>>>> HttpClient,
> > >>>>>>>> Pig
> > >>>>>>>>>> etc.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
> > >>>>>> internally as
> > >>>>>>>>>> well.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
> > >>>>>> version
> > >>>>>>>>>> also
> > >>>>>>>>>>>>> can be upgraded to 2.11.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> > >>>>>>>> rkanter@cloudera.com
> > >>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
> > >>> for
> > >>>>>> over
> > >>>>>>>> a
> > >>>>>>>>>> year
> > >>>>>>>>>>>>> now
> > >>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
> > >>> lines
> > >>>> in
> > >>>>>>>>>>>>> MiniHCatServer
> > >>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> diff --git
> > >>>>>>>>>>>>>
> > >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >>>>>>>>>>>>>>>>
> > >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
> > >>>>>>>>>>>>>>>> ---
> > >>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >>>>>>>>>>>>>>>> +++
> > >>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> > >>>>>>>>>>>>>>>>  public static void resetDefaultDBCreation() throws
> > >>>>>> Exception {
> > >>>>>>>>>>>>>>>>      // Need to do this, else default db will not be
> > >>>> created
> > >>>>>>>> for
> > >>>>>>>>>>>>> local
> > >>>>>>>>>>>>>>>> metastores.
> > >>>>>>>>>>>>>>>>      // TestHiveMain will fail with
> > >>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
> > >>> named
> > >>>>>>>> default)
> > >>>>>>>>>>>>>>>> -        Field declaredField =
> > >>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> > >>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
> > >>>>>>>>>>>>>>>> -        declaredField.set(null, false);
> > >>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
> > >>> 0.14
> > >>>> and
> > >>>>>>>> later
> > >>>>>>>>>>>>>>>> +        //Field declaredField =
> > >>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> > >>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
> > >>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
> > >>>>>>>>>>>>>>>>  }
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>  public static void resetHiveConfStaticVariables()
> > >>> throws
> > >>>>>>>>>> Exception
> > >>>>>>>>>>>>> {
> > >>>>>>>>>>>>>>>> diff --git
> > >>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >>>>>>>>>>>>>>>>
> > >>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
> > >>>>>>>>>>>>>>>> ---
> > >>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >>>>>>>>>>>>>>>> +++
> > >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> > >>>>>>>> AbstractHiveService
> > >>>>>>>>>> {
> > >>>>>>>>>>>>>>>>
> > >>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> > >>>>>>>>>> getHost());
> > >>>>>>>>>>>>>>>>  hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> > >>>>>>>>>>>>> getBinaryPort());
> > >>>>>>>>>>>>>>>>
> > >>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> > >>>>>>>>>>>>>>>> getHttpPort());
> > >>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> > >>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
> > >>> and
> > >>>>>> later
> > >>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>  Path scratchDir = new Path(baseDfsDir, "scratch");
> > >>>>>>>>>>>>>>>>  fs.mkdirs(scratchDir);
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> > >>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
> > >>>>>>>> vranganathan@hortonworks.com
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
> > >>> to
> > >>>>>> 1.x.
> > >>>>>>>>>> 0.13
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>> old
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Thanks
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Venkat
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> > >>>> rkanter@cloudera.com
> > >>>>>>>>>> <mailto:
> > >>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
> > >>>> about
> > >>>>>>>> when
> > >>>>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>>> okay
> > >>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
> > >>> need
> > >>>> to
> > >>>>>>>> wait
> > >>>>>>>>>>>>> until
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
> > >>> in
> > >>>> a
> > >>>>>>>> minor
> > >>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
> > >>> it in
> > >>>>>> 4.3
> > >>>>>>>> if
> > >>>>>>>>>>>>>>>>> nobody has
> > >>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
> > >>> out
> > >>>> in
> > >>>>>>>> our
> > >>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>> notes.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
> > >>> should
> > >>>>>> leave
> > >>>>>>>> out
> > >>>>>>>>>>>>> other
> > >>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
> > >>> in
> > >>>>>> trunk
> > >>>>>>>>>>>>> anyway.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
> > >>> should
> > >>>>>> make
> > >>>>>>>>>> sure
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>> include all of these and any pending ones.
> > >>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
> > >>>>>> time to
> > >>>>>>>>>> bake
> > >>>>>>>>>>>>>>>>> that we
> > >>>>>>>>>>>>>>>>> should hold off on including in the release?
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> thanks
> > >>>>>>>>>>>>>>>>> - Robert
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> > >>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
> > >>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
> > >>> release?
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> > >>>>>>>>>> bafna.iitr@gmail.com
> > >>>>>>>>>>>>> <ma...@gmail.com>>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
> > >>> 4.3.0.
> > >>>>>> This
> > >>>>>>>>>> thread
> > >>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
> > >>>>>> provide
> > >>>>>>>>>> your
> > >>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
> > >>>>>> exclusions)
> > >>>>>>>> in
> > >>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> release.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
> > >>>> trunk.
> > >>>>>>>> Based
> > >>>>>>>>>> on
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
> > >>>>>>>>>>>>>>>>>>> Version Upgrade
> > >>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
> > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> > >>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
> > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> > >>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
> > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
> > >>>>>>>>>>>>>>>>>>> ),
> > >>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
> > >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> > >>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
> > >>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
> > >>>>>>>>>>>>>>>>>> ).
> > >>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
> > >>>> Database
> > >>>>>>>>>> schema
> > >>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
> > >>> etc.
> > >>>>>>>>>>>>>>>>>>> Bug Fixes.
> > >>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
> > >>> above
> > >>>>>>>> points
> > >>>>>>>>>> or
> > >>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
> > >>> work
> > >>>>>>>> which
> > >>>>>>>>>> is
> > >>>>>>>>>>>>>>>>> already
> > >>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
> > >>>>>>>>>> availability
> > >>>>>>>>>>>>>>>>> of the
> > >>>>>>>>>>>>>>>>>> reviewers).
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >>
> >
> >
>


  

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

Ya, I could use some more time as well.  I have some deadlines early next
week, which has kept me from doing more reviews.

thanks
- Robert

On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma <
jaydeepmailv@gmail.com> wrote:

> Let see if we can wait for another week. So we can merge few more important
> jiras.  Few of them I personally want to get reviewed and merged.
>
> Regards,
> Jaydeep
>
> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
> wrote:
>
> > As previously discussed, we will be moving with whatever patches can be
> > reviewed/merged till end of this week.
> >
> > But there are some which needs to be reviewed and merged, mainly
> regarding
> > the upgrades. Some of the patches require Java 1.7, so I think we should
> > review OOZIE-2036 and merged it.
> >
> > ~Abhishek
> > > On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <
> rohini.aditya@gmail.com>
> > wrote:
> > >
> > > Cleaned up that list and now no jira refers to "trunk".  Patch
> Available
> > > jiras targeted for trunk now point to 4.4.0.
> > >
> > > https://issues.apache.org/jira/issues/?jql=project%20%
> > 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
> > 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
> > >
> > > There are 42 jiras to go. We can take a look mid next week to see if
> any
> > > pending ones needs to be pushed out. There are quite a few which are
> > > unassigned.
> > >
> > > If there is any jira, someone would like to include in 4.3.0 please
> > update
> > > the jira.
> > >
> > > Regards,
> > > Rohini
> > >
> > > On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
> > rohini.aditya@gmail.com
> > >> wrote:
> > >
> > >> Unset the open ones without sending emails. Will update the resolved,
> > >> closed and Patch Available ones sending emails.
> > >>
> > >> Regards,
> > >> Rohini
> > >>
> > >> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rk...@cloudera.com>
> > >> wrote:
> > >>
> > >>> Ya, let's unset those remaining open trunk ones.  I agree, we need to
> > put
> > >>> in some effort to clean out old JIRAs and to review Patch Available
> > JIRAs.
> > >>>
> > >>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
> > discuss
> > >>> the issue more there, but I listed a few reasons on OOZIE-2625 about
> > why.
> > >>>
> > >>> - Robert
> > >>>
> > >>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> > >>> rohini.aditya@gmail.com>
> > >>> wrote:
> > >>>
> > >>>>
> > >>>>
> > >>> https://issues.apache.org/jira/issues/?jql=project%20%
> > 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
> > 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
> > 20status%20DESC&startIndex=50
> > >>>>
> > >>>> These are jiras open and marked trunk. Should we unset fix version
> on
> > >>> these
> > >>>> or mark them for next release? Don't see majority of them being
> worked
> > >>> on
> > >>>> even for next release. We need to do bug scrub and close many of
> these
> > >>> some
> > >>>> time.
> > >>>>
> > >>>>> Instead
> > >>>> of deleting "trunk", I think we're better off just renaming it to
> the
> > >>>> release number.
> > >>>> There is some amount of relabeling either way. If we go with
> unsetting
> > >>> fix
> > >>>> version, then we can as well delete trunk. Also, I did create a
> 4.4.0
> > >>>> yesterday to move some of the patch available jiras from 4.3.0.
> > >>>>
> > >>>>
> > >>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> > >>>> rohini.aditya@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>> I think it would be better to send out email for the bulk change.
> If
> > >>>> there
> > >>>>> are any issues or jiras wrongly marked someone can respond. We do
> > that
> > >>>> with
> > >>>>> Pig and there are always couple of jiras that folks come back with
> > >>>> feedback
> > >>>>> on pulling back to release.
> > >>>>>
> > >>>>>
> > >>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <
> rkanter@cloudera.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Abhishek, next time you do a bulk change, please remember to tell
> it
> > >>> not
> > >>>>>> to
> > >>>>>> send out an email for every change :)
> > >>>>>>
> > >>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <
> rkanter@cloudera.com
> > >
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Rohini, that sounds good to me.  Especially because we're on
> > >>> "master"
> > >>>>>>> branch but "trunk" version in JIRA and it gets a little
> confusing.
> > >>>>>> Instead
> > >>>>>>> of deleting "trunk", I think we're better off just renaming it to
> > >>> the
> > >>>>>>> release number.  Otherwise, we'll have to re-label any current
> > >>> "trunk"
> > >>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
> > >>> release
> > >>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
> > >>> rename it
> > >>>>>>> later.
> > >>>>>>>
> > >>>>>>> I'm not sure if there's a specific rule here, but I suppose we
> > >>> should
> > >>>>>>> probably have a vote in the private list to make things official
> as
> > >>>>>> this is
> > >>>>>>> a release-management-related change.  Can you send an email?
> > >>>>>>>
> > >>>>>>> Abhishek, thanks for putting this all together.  I think Rohini's
> > >>>>>>> suggestion of updating the JIRAs is the right next step.  You're
> > >>>>>> probably
> > >>>>>>> best off using the bulk change feature instead of doing each JIRA
> > >>>>>>> individually.  I should have time to do a bunch of reviews this
> > >>> week,
> > >>>>>> I've
> > >>>>>>> actually already started.  I should have some time to continue
> > >>> doing
> > >>>>>> more
> > >>>>>>> reviews.  As a general thing, we've been pretty slow about
> getting
> > >>>> these
> > >>>>>>> done :(
> > >>>>>>>
> > >>>>>>> - Robert
> > >>>>>>>
> > >>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> > >>>>>>> rohini.aditya@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> I also did mark some of the enhancement jiras that Satish was
> > >>> working
> > >>>>>> on
> > >>>>>>>> and which might take time to review to 4.4.0
> > >>>>>>>>
> > >>>>>>>> Regards,
> > >>>>>>>> Rohini
> > >>>>>>>>
> > >>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> > >>>>>>>> rohini.aditya@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Abhishek,
> > >>>>>>>>>    Can you mark all jiras already committed to trunk as 4.3.0
> > >>> and
> > >>>>>> those
> > >>>>>>>>> planned in the scope document as 4.3.0? This will help filter
> on
> > >>>>>> those
> > >>>>>>>>> and review instead of referring to a google doc. I can pick up
> > >>>> 10-15
> > >>>>>>>>> jiras for review end of this week. We should try to get
> > >>> majority of
> > >>>>>>>> patches
> > >>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use
> the
> > >>>> 3rd
> > >>>>>>>> week
> > >>>>>>>>> for any pending critical issues and rolling the release
> > >>> candidate.
> > >>>>>>>>>
> > >>>>>>>>> Robert,
> > >>>>>>>>>  I am thinking, that we should get rid of the trunk version and
> > >>>> use
> > >>>>>>>>> release numbers when committing from the beginning like all
> > >>> other
> > >>>>>>>> projects
> > >>>>>>>>> do. Unnecessary step to change from trunk to release version
> for
> > >>>>>> every
> > >>>>>>>>> release. If folks are in agreement, then will delete the trunk
> > >>>>>> version
> > >>>>>>>>> altogether. Does that require a mail to private?
> > >>>>>>>>>
> > >>>>>>>>> Regards,
> > >>>>>>>>> Rohini
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> > >>>>>> bafna.iitr@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi All,
> > >>>>>>>>>>
> > >>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> > mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> > >>>>>>>>>
> > >>>>>>>>>> and listed all the things which we wanted to include in the
> > >>>> release.
> > >>>>>>>> In the
> > >>>>>>>>>> last few days, I spent some time and reviewed some of the
> > >>> patches
> > >>>>>> and
> > >>>>>>>>>> provided my feedback. I will continue my efforts to review
> more
> > >>>>>>>> patches.
> > >>>>>>>>>> These patches needs to be reviewed by committers and get
> > >>>> committed.
> > >>>>>> I
> > >>>>>>>>>> request other contributors also, review some of the patches
> and
> > >>>>>> provide
> > >>>>>>>>>> their feedback.
> > >>>>>>>>>>
> > >>>>>>>>>> I am thinking, may be, we can decide upon some fixed
> > >>> (approximate)
> > >>>>>>>>>> release date and start working accordingly. Because, If we
> wait
> > >>>> for
> > >>>>>>>> all the
> > >>>>>>>>>> listed items to be reviewed and committed, then it might take
> a
> > >>>> lot
> > >>>>>>>> more
> > >>>>>>>>>> time. I am thinking we should target third week of August to
> > >>>>>> complete
> > >>>>>>>> and
> > >>>>>>>>>> release.
> > >>>>>>>>>>
> > >>>>>>>>>> Feel free to provide your feedback and suggestions.
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> Abhishek
> > >>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> > >>>> bafna.iitr@gmail.com
> > >>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hi,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I looked into the available patches and prepared the list. I
> > >>>> have
> > >>>>>>>> put,
> > >>>>>>>>>> scope content which we have discussed till now, into a word
> > >>>>>> document.
> > >>>>>>>> (For
> > >>>>>>>>>> the better formatting and easy to read and comment).
> > >>>>>>>>>>>
> > >>>>>>>>>>> I have noted the additional Jira into the list. Added two
> > >>> more
> > >>>>>> Jira
> > >>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
> > >>>>>>>>>>>
> > >>>>>>>>>>> Please take a look at the document (link below) and provide
> > >>> your
> > >>>>>>>>>> comments.
> > >>>>>>>>>>>
> > >>>>>>>>>>> If this looks good, we can start the release process.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Scope Document:
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> > mFsuHQZmrT63Ow/edit#
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> Abhishek
> > >>>>>>>>>>>
> > >>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> > >>>> rkanter@cloudera.com
> > >>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of is
> > >>>>>>>> OOZIE-2306
> > >>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
> > >>>>>> sense to
> > >>>>>>>>>> let it
> > >>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
> > >>> yet.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I'd also like to add a few others to the list:
> > >>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> > >>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> > >>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> > >>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> > >>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
> > >>> other
> > >>>>>>>> fixes)
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
> > >>>> list
> > >>>>>>>>>> (including
> > >>>>>>>>>>>> the ones I mentioned above).  While we don't need to include
> > >>>>>>>> everything
> > >>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
> > >>>>>>>> bandwidth),
> > >>>>>>>>>> did
> > >>>>>>>>>>>> you consider each of them when creating the list?  I
> > >>> currently
> > >>>>>> see
> > >>>>>>>> 97
> > >>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> > >>>>>>>>>>>> I think we should include all Spark Action
> > >>> fixes/improvements,
> > >>>>>> given
> > >>>>>>>>>> how
> > >>>>>>>>>>>> popular it seems to be.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> thanks
> > >>>>>>>>>>>> - Robert
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> > >>>>>>>> bafna.iitr@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Below is the list of issues, picked from currently
> > >>> available
> > >>>>>>>> patches.
> > >>>>>>>>>> I
> > >>>>>>>>>>>>> could not decide up any 'open' issue, If there is anything,
> > >>>>>> please
> > >>>>>>>>>> reply
> > >>>>>>>>>>>>> here.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
> > >>>>>> Important
> > >>>>>>>>>> is,
> > >>>>>>>>>>>>> how many of these we will be able to review and commit.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I will starting looking into these issues and provide
> > >>>> feedback.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>> ------------------------------------------------------------
> > ---------------
> > >>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
> > >>> slow
> > >>>> in
> > >>>>>>>> some
> > >>>>>>>>>> cases
> > >>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action rerun
> > >>>>>>>>>> [OOZIE-1735]
> > >>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
> > >>>>>> variable
> > >>>>>>>> name
> > >>>>>>>>>>>>> containing dot
> > >>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
> > >>>>>> failures
> > >>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
> > >>>>>> endCreatedTime
> > >>>>>>>>>>>>> filters for coord and bundles
> > >>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
> > >>> jar
> > >>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
> > >>>> running
> > >>>>>> for
> > >>>>>>>>>> longer
> > >>>>>>>>>>>>> than expected time
> > >>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
> > >>> 90%
> > >>>>>> cpu
> > >>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
> > >>>> urls
> > >>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
> > >>>>>>>> information
> > >>>>>>>>>> about
> > >>>>>>>>>>>>> queued commnads
> > >>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
> > >>> be
> > >>>>>>>> handled
> > >>>>>>>>>>>>> rightly
> > >>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
> > >>>>>>>> mapred-site
> > >>>>>>>>>>>>> into spark launcher
> > >>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
> > >>> work
> > >>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
> > >>> log4j
> > >>>>>>>>>>>>> configuration file for execution mode'
> > >>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return port
> > >>>> for
> > >>>>>>>> admin
> > >>>>>>>>>>>>> -servers
> > >>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
> > >>>>>>>> endtime/pausetime
> > >>>>>>>>>> of a
> > >>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
> > >>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for downstream
> > >>>>>>>> components
> > >>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
> > >>> API
> > >>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
> > >>> minioozie
> > >>>>>> doc
> > >>>>>>>>>> does not
> > >>>>>>>>>>>>> work
> > >>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
> > >>> error
> > >>>>>> and
> > >>>>>>>>>> audit
> > >>>>>>>>>>>>> logs
> > >>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
> > >>> querying
> > >>>>>>>> should
> > >>>>>>>>>> have
> > >>>>>>>>>>>>> last modified filter
> > >>>>>>>>>>>>> OOZIE-2306      Oozie health check
> > >>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
> > >>> directories
> > >>>> in
> > >>>>>>>> /tmp
> > >>>>>>>>>> per
> > >>>>>>>>>>>>> day
> > >>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> > >>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
> > >>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
> > >>> action
> > >>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
> > >>> ensure
> > >>>>>>>>>> uploaded
> > >>>>>>>>>>>>> jars are world readable
> > >>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing completed
> > >>>>>> output
> > >>>>>>>>>> data
> > >>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
> > >>>> kill
> > >>>>>>>>>> control
> > >>>>>>>>>>>>> node cannot resolve variable in message
> > >>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
> > >>> when
> > >>>>>>>> logging
> > >>>>>>>>>>>>> command arguments
> > >>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>> ------------------------------------------------------------
> > ---------------
> > >>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> > >>>>>> bafna.iitr@gmail.com
> > >>>>>>>>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
> > >>> to
> > >>>> me
> > >>>>>> and
> > >>>>>>>>>>>>> should be included into the release. If there are things,
> > >>>> which
> > >>>>>>>> needs
> > >>>>>>>>>> to be
> > >>>>>>>>>>>>> excluded, provide your comments.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> We still need to decide on the patch which are in progress
> > >>>>>> status.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On the upgrade side:
> > >>>>>>>>>>>>>> Java version          1.6 to 1.7
> > >>>>>>>>>>>>>> OpenJPA version       2.2.2 to 2.4.1
> > >>>>>>>>>>>>>> Curator version       2.5.0 to 2.11.0
> > >>>>>>>>>>>>>> Pig version           0.12.1 to 0.16.0
> > >>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
> > >>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
> > >>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
> > >>>>>> working
> > >>>>>>>> on
> > >>>>>>>>>> the
> > >>>>>>>>>>>>> completing/testing the release.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> > >>>>>>>> bafna.iitr@gmail.com
> > >>>>>>>>>>>>> <ma...@gmail.com>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
> > >>>>>> HttpClient,
> > >>>>>>>> Pig
> > >>>>>>>>>> etc.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
> > >>>>>> internally as
> > >>>>>>>>>> well.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
> > >>>>>> version
> > >>>>>>>>>> also
> > >>>>>>>>>>>>> can be upgraded to 2.11.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> > >>>>>>>> rkanter@cloudera.com
> > >>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
> > >>> for
> > >>>>>> over
> > >>>>>>>> a
> > >>>>>>>>>> year
> > >>>>>>>>>>>>> now
> > >>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
> > >>> lines
> > >>>> in
> > >>>>>>>>>>>>> MiniHCatServer
> > >>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> diff --git
> > >>>>>>>>>>>>>
> > >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >>>>>>>>>>>>>>>>
> > >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
> > >>>>>>>>>>>>>>>> ---
> > >>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >>>>>>>>>>>>>>>> +++
> > >>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> > >>>>>>>>>>>>>>>>  public static void resetDefaultDBCreation() throws
> > >>>>>> Exception {
> > >>>>>>>>>>>>>>>>      // Need to do this, else default db will not be
> > >>>> created
> > >>>>>>>> for
> > >>>>>>>>>>>>> local
> > >>>>>>>>>>>>>>>> metastores.
> > >>>>>>>>>>>>>>>>      // TestHiveMain will fail with
> > >>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
> > >>> named
> > >>>>>>>> default)
> > >>>>>>>>>>>>>>>> -        Field declaredField =
> > >>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> > >>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
> > >>>>>>>>>>>>>>>> -        declaredField.set(null, false);
> > >>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
> > >>> 0.14
> > >>>> and
> > >>>>>>>> later
> > >>>>>>>>>>>>>>>> +        //Field declaredField =
> > >>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> > >>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
> > >>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
> > >>>>>>>>>>>>>>>>  }
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>  public static void resetHiveConfStaticVariables()
> > >>> throws
> > >>>>>>>>>> Exception
> > >>>>>>>>>>>>> {
> > >>>>>>>>>>>>>>>> diff --git
> > >>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >>>>>>>>>>>>>>>>
> > >>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
> > >>>>>>>>>>>>>>>> ---
> > >>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >>>>>>>>>>>>>>>> +++
> > >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> > >>>>>>>> AbstractHiveService
> > >>>>>>>>>> {
> > >>>>>>>>>>>>>>>>
> > >>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> > >>>>>>>>>> getHost());
> > >>>>>>>>>>>>>>>>  hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> > >>>>>>>>>>>>> getBinaryPort());
> > >>>>>>>>>>>>>>>>
> > >>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> > >>>>>>>>>>>>>>>> getHttpPort());
> > >>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> > >>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
> > >>> and
> > >>>>>> later
> > >>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>  Path scratchDir = new Path(baseDfsDir, "scratch");
> > >>>>>>>>>>>>>>>>  fs.mkdirs(scratchDir);
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> > >>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
> > >>>>>>>> vranganathan@hortonworks.com
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
> > >>> to
> > >>>>>> 1.x.
> > >>>>>>>>>> 0.13
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>> old
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Thanks
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Venkat
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> > >>>> rkanter@cloudera.com
> > >>>>>>>>>> <mailto:
> > >>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
> > >>>> about
> > >>>>>>>> when
> > >>>>>>>>>>>>> it's
> > >>>>>>>>>>>>>>>>> okay
> > >>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
> > >>> need
> > >>>> to
> > >>>>>>>> wait
> > >>>>>>>>>>>>> until
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
> > >>> in
> > >>>> a
> > >>>>>>>> minor
> > >>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
> > >>> it in
> > >>>>>> 4.3
> > >>>>>>>> if
> > >>>>>>>>>>>>>>>>> nobody has
> > >>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
> > >>> out
> > >>>> in
> > >>>>>>>> our
> > >>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>> notes.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
> > >>> should
> > >>>>>> leave
> > >>>>>>>> out
> > >>>>>>>>>>>>> other
> > >>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
> > >>> in
> > >>>>>> trunk
> > >>>>>>>>>>>>> anyway.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
> > >>> should
> > >>>>>> make
> > >>>>>>>>>> sure
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>> include all of these and any pending ones.
> > >>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
> > >>>>>> time to
> > >>>>>>>>>> bake
> > >>>>>>>>>>>>>>>>> that we
> > >>>>>>>>>>>>>>>>> should hold off on including in the release?
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> thanks
> > >>>>>>>>>>>>>>>>> - Robert
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> > >>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
> > >>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
> > >>> release?
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> > >>>>>>>>>> bafna.iitr@gmail.com
> > >>>>>>>>>>>>> <ma...@gmail.com>>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
> > >>> 4.3.0.
> > >>>>>> This
> > >>>>>>>>>> thread
> > >>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
> > >>>>>> provide
> > >>>>>>>>>> your
> > >>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
> > >>>>>> exclusions)
> > >>>>>>>> in
> > >>>>>>>>>> the
> > >>>>>>>>>>>>>>>>> release.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
> > >>>> trunk.
> > >>>>>>>> Based
> > >>>>>>>>>> on
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
> > >>>>>>>>>>>>>>>>>>> Version Upgrade
> > >>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
> > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> > >>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
> > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> > >>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
> > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
> > >>>>>>>>>>>>>>>>>>> ),
> > >>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
> > >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> > >>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
> > >>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
> > >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
> > >>>>>>>>>>>>>>>>>> ).
> > >>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
> > >>>> Database
> > >>>>>>>>>> schema
> > >>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
> > >>> etc.
> > >>>>>>>>>>>>>>>>>>> Bug Fixes.
> > >>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
> > >>> above
> > >>>>>>>> points
> > >>>>>>>>>> or
> > >>>>>>>>>>>>>>>>> more
> > >>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
> > >>> work
> > >>>>>>>> which
> > >>>>>>>>>> is
> > >>>>>>>>>>>>>>>>> already
> > >>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
> > >>>>>>>>>> availability
> > >>>>>>>>>>>>>>>>> of the
> > >>>>>>>>>>>>>>>>>> reviewers).
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>> Abhishek
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >>
> >
> >
>

Re: [SCOPE] Oozie 4.3.0 Release

[jaydeep vishwakarma <ja...@gmail.com>]

Let see if we can wait for another week. So we can merge few more important
jiras.  Few of them I personally want to get reviewed and merged.

Regards,
Jaydeep

On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <ba...@gmail.com>
wrote:

> As previously discussed, we will be moving with whatever patches can be
> reviewed/merged till end of this week.
>
> But there are some which needs to be reviewed and merged, mainly regarding
> the upgrades. Some of the patches require Java 1.7, so I think we should
> review OOZIE-2036 and merged it.
>
> ~Abhishek
> > On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <ro...@gmail.com>
> wrote:
> >
> > Cleaned up that list and now no jira refers to "trunk".  Patch Available
> > jiras targeted for trunk now point to 4.4.0.
> >
> > https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%
> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
> >
> > There are 42 jiras to go. We can take a look mid next week to see if any
> > pending ones needs to be pushed out. There are quite a few which are
> > unassigned.
> >
> > If there is any jira, someone would like to include in 4.3.0 please
> update
> > the jira.
> >
> > Regards,
> > Rohini
> >
> > On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <
> rohini.aditya@gmail.com
> >> wrote:
> >
> >> Unset the open ones without sending emails. Will update the resolved,
> >> closed and Patch Available ones sending emails.
> >>
> >> Regards,
> >> Rohini
> >>
> >> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rk...@cloudera.com>
> >> wrote:
> >>
> >>> Ya, let's unset those remaining open trunk ones.  I agree, we need to
> put
> >>> in some effort to clean out old JIRAs and to review Patch Available
> JIRAs.
> >>>
> >>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can
> discuss
> >>> the issue more there, but I listed a few reasons on OOZIE-2625 about
> why.
> >>>
> >>> - Robert
> >>>
> >>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> >>> rohini.aditya@gmail.com>
> >>> wrote:
> >>>
> >>>>
> >>>>
> >>> https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%
> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%
> 20status%20DESC&startIndex=50
> >>>>
> >>>> These are jiras open and marked trunk. Should we unset fix version on
> >>> these
> >>>> or mark them for next release? Don't see majority of them being worked
> >>> on
> >>>> even for next release. We need to do bug scrub and close many of these
> >>> some
> >>>> time.
> >>>>
> >>>>> Instead
> >>>> of deleting "trunk", I think we're better off just renaming it to the
> >>>> release number.
> >>>> There is some amount of relabeling either way. If we go with unsetting
> >>> fix
> >>>> version, then we can as well delete trunk. Also, I did create a 4.4.0
> >>>> yesterday to move some of the patch available jiras from 4.3.0.
> >>>>
> >>>>
> >>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> >>>> rohini.aditya@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> I think it would be better to send out email for the bulk change. If
> >>>> there
> >>>>> are any issues or jiras wrongly marked someone can respond. We do
> that
> >>>> with
> >>>>> Pig and there are always couple of jiras that folks come back with
> >>>> feedback
> >>>>> on pulling back to release.
> >>>>>
> >>>>>
> >>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <rk...@cloudera.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Abhishek, next time you do a bulk change, please remember to tell it
> >>> not
> >>>>>> to
> >>>>>> send out an email for every change :)
> >>>>>>
> >>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rkanter@cloudera.com
> >
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Rohini, that sounds good to me.  Especially because we're on
> >>> "master"
> >>>>>>> branch but "trunk" version in JIRA and it gets a little confusing.
> >>>>>> Instead
> >>>>>>> of deleting "trunk", I think we're better off just renaming it to
> >>> the
> >>>>>>> release number.  Otherwise, we'll have to re-label any current
> >>> "trunk"
> >>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
> >>> release
> >>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
> >>> rename it
> >>>>>>> later.
> >>>>>>>
> >>>>>>> I'm not sure if there's a specific rule here, but I suppose we
> >>> should
> >>>>>>> probably have a vote in the private list to make things official as
> >>>>>> this is
> >>>>>>> a release-management-related change.  Can you send an email?
> >>>>>>>
> >>>>>>> Abhishek, thanks for putting this all together.  I think Rohini's
> >>>>>>> suggestion of updating the JIRAs is the right next step.  You're
> >>>>>> probably
> >>>>>>> best off using the bulk change feature instead of doing each JIRA
> >>>>>>> individually.  I should have time to do a bunch of reviews this
> >>> week,
> >>>>>> I've
> >>>>>>> actually already started.  I should have some time to continue
> >>> doing
> >>>>>> more
> >>>>>>> reviews.  As a general thing, we've been pretty slow about getting
> >>>> these
> >>>>>>> done :(
> >>>>>>>
> >>>>>>> - Robert
> >>>>>>>
> >>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> >>>>>>> rohini.aditya@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> I also did mark some of the enhancement jiras that Satish was
> >>> working
> >>>>>> on
> >>>>>>>> and which might take time to review to 4.4.0
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Rohini
> >>>>>>>>
> >>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> >>>>>>>> rohini.aditya@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Abhishek,
> >>>>>>>>>    Can you mark all jiras already committed to trunk as 4.3.0
> >>> and
> >>>>>> those
> >>>>>>>>> planned in the scope document as 4.3.0? This will help filter on
> >>>>>> those
> >>>>>>>>> and review instead of referring to a google doc. I can pick up
> >>>> 10-15
> >>>>>>>>> jiras for review end of this week. We should try to get
> >>> majority of
> >>>>>>>> patches
> >>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use the
> >>>> 3rd
> >>>>>>>> week
> >>>>>>>>> for any pending critical issues and rolling the release
> >>> candidate.
> >>>>>>>>>
> >>>>>>>>> Robert,
> >>>>>>>>>  I am thinking, that we should get rid of the trunk version and
> >>>> use
> >>>>>>>>> release numbers when committing from the beginning like all
> >>> other
> >>>>>>>> projects
> >>>>>>>>> do. Unnecessary step to change from trunk to release version for
> >>>>>> every
> >>>>>>>>> release. If folks are in agreement, then will delete the trunk
> >>>>>> version
> >>>>>>>>> altogether. Does that require a mail to private?
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Rohini
> >>>>>>>>>
> >>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> >>>>>> bafna.iitr@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi All,
> >>>>>>>>>>
> >>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >>>>>>>>>
> >>>>>>>>>> and listed all the things which we wanted to include in the
> >>>> release.
> >>>>>>>> In the
> >>>>>>>>>> last few days, I spent some time and reviewed some of the
> >>> patches
> >>>>>> and
> >>>>>>>>>> provided my feedback. I will continue my efforts to review more
> >>>>>>>> patches.
> >>>>>>>>>> These patches needs to be reviewed by committers and get
> >>>> committed.
> >>>>>> I
> >>>>>>>>>> request other contributors also, review some of the patches and
> >>>>>> provide
> >>>>>>>>>> their feedback.
> >>>>>>>>>>
> >>>>>>>>>> I am thinking, may be, we can decide upon some fixed
> >>> (approximate)
> >>>>>>>>>> release date and start working accordingly. Because, If we wait
> >>>> for
> >>>>>>>> all the
> >>>>>>>>>> listed items to be reviewed and committed, then it might take a
> >>>> lot
> >>>>>>>> more
> >>>>>>>>>> time. I am thinking we should target third week of August to
> >>>>>> complete
> >>>>>>>> and
> >>>>>>>>>> release.
> >>>>>>>>>>
> >>>>>>>>>> Feel free to provide your feedback and suggestions.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Abhishek
> >>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> >>>> bafna.iitr@gmail.com
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> I looked into the available patches and prepared the list. I
> >>>> have
> >>>>>>>> put,
> >>>>>>>>>> scope content which we have discussed till now, into a word
> >>>>>> document.
> >>>>>>>> (For
> >>>>>>>>>> the better formatting and easy to read and comment).
> >>>>>>>>>>>
> >>>>>>>>>>> I have noted the additional Jira into the list. Added two
> >>> more
> >>>>>> Jira
> >>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
> >>>>>>>>>>>
> >>>>>>>>>>> Please take a look at the document (link below) and provide
> >>> your
> >>>>>>>>>> comments.
> >>>>>>>>>>>
> >>>>>>>>>>> If this looks good, we can start the release process.
> >>>>>>>>>>>
> >>>>>>>>>>> Scope Document:
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G
> mFsuHQZmrT63Ow/edit#
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Abhishek
> >>>>>>>>>>>
> >>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> >>>> rkanter@cloudera.com
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of is
> >>>>>>>> OOZIE-2306
> >>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
> >>>>>> sense to
> >>>>>>>>>> let it
> >>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
> >>> yet.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'd also like to add a few others to the list:
> >>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> >>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> >>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> >>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
> >>> other
> >>>>>>>> fixes)
> >>>>>>>>>>>>
> >>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
> >>>> list
> >>>>>>>>>> (including
> >>>>>>>>>>>> the ones I mentioned above).  While we don't need to include
> >>>>>>>> everything
> >>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
> >>>>>>>> bandwidth),
> >>>>>>>>>> did
> >>>>>>>>>>>> you consider each of them when creating the list?  I
> >>> currently
> >>>>>> see
> >>>>>>>> 97
> >>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
> >>>>>>>>>>>> I think we should include all Spark Action
> >>> fixes/improvements,
> >>>>>> given
> >>>>>>>>>> how
> >>>>>>>>>>>> popular it seems to be.
> >>>>>>>>>>>>
> >>>>>>>>>>>> thanks
> >>>>>>>>>>>> - Robert
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> >>>>>>>> bafna.iitr@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Below is the list of issues, picked from currently
> >>> available
> >>>>>>>> patches.
> >>>>>>>>>> I
> >>>>>>>>>>>>> could not decide up any 'open' issue, If there is anything,
> >>>>>> please
> >>>>>>>>>> reply
> >>>>>>>>>>>>> here.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
> >>>>>> Important
> >>>>>>>>>> is,
> >>>>>>>>>>>>> how many of these we will be able to review and commit.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I will starting looking into these issues and provide
> >>>> feedback.
> >>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>> ------------------------------------------------------------
> ---------------
> >>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
> >>> slow
> >>>> in
> >>>>>>>> some
> >>>>>>>>>> cases
> >>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action rerun
> >>>>>>>>>> [OOZIE-1735]
> >>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
> >>>>>> variable
> >>>>>>>> name
> >>>>>>>>>>>>> containing dot
> >>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
> >>>>>> failures
> >>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
> >>>>>> endCreatedTime
> >>>>>>>>>>>>> filters for coord and bundles
> >>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
> >>> jar
> >>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
> >>>> running
> >>>>>> for
> >>>>>>>>>> longer
> >>>>>>>>>>>>> than expected time
> >>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
> >>> 90%
> >>>>>> cpu
> >>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
> >>>> urls
> >>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
> >>>>>>>> information
> >>>>>>>>>> about
> >>>>>>>>>>>>> queued commnads
> >>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
> >>> be
> >>>>>>>> handled
> >>>>>>>>>>>>> rightly
> >>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
> >>>>>>>> mapred-site
> >>>>>>>>>>>>> into spark launcher
> >>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
> >>> work
> >>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
> >>> log4j
> >>>>>>>>>>>>> configuration file for execution mode'
> >>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return port
> >>>> for
> >>>>>>>> admin
> >>>>>>>>>>>>> -servers
> >>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
> >>>>>>>> endtime/pausetime
> >>>>>>>>>> of a
> >>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
> >>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for downstream
> >>>>>>>> components
> >>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
> >>> API
> >>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
> >>> minioozie
> >>>>>> doc
> >>>>>>>>>> does not
> >>>>>>>>>>>>> work
> >>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
> >>> error
> >>>>>> and
> >>>>>>>>>> audit
> >>>>>>>>>>>>> logs
> >>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
> >>> querying
> >>>>>>>> should
> >>>>>>>>>> have
> >>>>>>>>>>>>> last modified filter
> >>>>>>>>>>>>> OOZIE-2306      Oozie health check
> >>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
> >>> directories
> >>>> in
> >>>>>>>> /tmp
> >>>>>>>>>> per
> >>>>>>>>>>>>> day
> >>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> >>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
> >>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
> >>> action
> >>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
> >>> ensure
> >>>>>>>>>> uploaded
> >>>>>>>>>>>>> jars are world readable
> >>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing completed
> >>>>>> output
> >>>>>>>>>> data
> >>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
> >>>> kill
> >>>>>>>>>> control
> >>>>>>>>>>>>> node cannot resolve variable in message
> >>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
> >>> when
> >>>>>>>> logging
> >>>>>>>>>>>>> command arguments
> >>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>> ------------------------------------------------------------
> ---------------
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> >>>>>> bafna.iitr@gmail.com
> >>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
> >>> to
> >>>> me
> >>>>>> and
> >>>>>>>>>>>>> should be included into the release. If there are things,
> >>>> which
> >>>>>>>> needs
> >>>>>>>>>> to be
> >>>>>>>>>>>>> excluded, provide your comments.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We still need to decide on the patch which are in progress
> >>>>>> status.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On the upgrade side:
> >>>>>>>>>>>>>> Java version          1.6 to 1.7
> >>>>>>>>>>>>>> OpenJPA version       2.2.2 to 2.4.1
> >>>>>>>>>>>>>> Curator version       2.5.0 to 2.11.0
> >>>>>>>>>>>>>> Pig version           0.12.1 to 0.16.0
> >>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
> >>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
> >>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
> >>>>>> working
> >>>>>>>> on
> >>>>>>>>>> the
> >>>>>>>>>>>>> completing/testing the release.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> >>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>> <ma...@gmail.com>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
> >>>>>> HttpClient,
> >>>>>>>> Pig
> >>>>>>>>>> etc.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
> >>>>>> internally as
> >>>>>>>>>> well.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
> >>>>>> version
> >>>>>>>>>> also
> >>>>>>>>>>>>> can be upgraded to 2.11.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> >>>>>>>> rkanter@cloudera.com
> >>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
> >>> for
> >>>>>> over
> >>>>>>>> a
> >>>>>>>>>> year
> >>>>>>>>>>>>> now
> >>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
> >>> lines
> >>>> in
> >>>>>>>>>>>>> MiniHCatServer
> >>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>>>>>
> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>>>>>>>>>>>>
> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
> >>>>>>>>>>>>>>>> ---
> >>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>>>>>>>>>>>> +++
> >>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> >>>>>>>>>>>>>>>>  public static void resetDefaultDBCreation() throws
> >>>>>> Exception {
> >>>>>>>>>>>>>>>>      // Need to do this, else default db will not be
> >>>> created
> >>>>>>>> for
> >>>>>>>>>>>>> local
> >>>>>>>>>>>>>>>> metastores.
> >>>>>>>>>>>>>>>>      // TestHiveMain will fail with
> >>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
> >>> named
> >>>>>>>> default)
> >>>>>>>>>>>>>>>> -        Field declaredField =
> >>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>> -        declaredField.set(null, false);
> >>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
> >>> 0.14
> >>>> and
> >>>>>>>> later
> >>>>>>>>>>>>>>>> +        //Field declaredField =
> >>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
> >>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
> >>>>>>>>>>>>>>>>  }
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>  public static void resetHiveConfStaticVariables()
> >>> throws
> >>>>>>>>>> Exception
> >>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>> diff --git
> >>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>>
> >>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
> >>>>>>>>>>>>>>>> ---
> >>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>> +++
> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> >>>>>>>> AbstractHiveService
> >>>>>>>>>> {
> >>>>>>>>>>>>>>>>
> >>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> >>>>>>>>>> getHost());
> >>>>>>>>>>>>>>>>  hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> >>>>>>>>>>>>> getBinaryPort());
> >>>>>>>>>>>>>>>>
> >>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >>>>>>>>>>>>>>>> getHttpPort());
> >>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
> >>> and
> >>>>>> later
> >>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>  Path scratchDir = new Path(baseDfsDir, "scratch");
> >>>>>>>>>>>>>>>>  fs.mkdirs(scratchDir);
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> >>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
> >>>>>>>> vranganathan@hortonworks.com
> >>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
> >>> to
> >>>>>> 1.x.
> >>>>>>>>>> 0.13
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> old
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Venkat
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> >>>> rkanter@cloudera.com
> >>>>>>>>>> <mailto:
> >>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
> >>>> about
> >>>>>>>> when
> >>>>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>> okay
> >>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
> >>> need
> >>>> to
> >>>>>>>> wait
> >>>>>>>>>>>>> until
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
> >>> in
> >>>> a
> >>>>>>>> minor
> >>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
> >>> it in
> >>>>>> 4.3
> >>>>>>>> if
> >>>>>>>>>>>>>>>>> nobody has
> >>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
> >>> out
> >>>> in
> >>>>>>>> our
> >>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>> notes.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
> >>> should
> >>>>>> leave
> >>>>>>>> out
> >>>>>>>>>>>>> other
> >>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
> >>> in
> >>>>>> trunk
> >>>>>>>>>>>>> anyway.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
> >>> should
> >>>>>> make
> >>>>>>>>>> sure
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> include all of these and any pending ones.
> >>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
> >>>>>> time to
> >>>>>>>>>> bake
> >>>>>>>>>>>>>>>>> that we
> >>>>>>>>>>>>>>>>> should hold off on including in the release?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> thanks
> >>>>>>>>>>>>>>>>> - Robert
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> >>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
> >>> release?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> >>>>>>>>>> bafna.iitr@gmail.com
> >>>>>>>>>>>>> <ma...@gmail.com>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
> >>> 4.3.0.
> >>>>>> This
> >>>>>>>>>> thread
> >>>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
> >>>>>> provide
> >>>>>>>>>> your
> >>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
> >>>>>> exclusions)
> >>>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
> >>>> trunk.
> >>>>>>>> Based
> >>>>>>>>>> on
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
> >>>>>>>>>>>>>>>>>>> Version Upgrade
> >>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
> >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> >>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
> >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> >>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
> >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
> >>>>>>>>>>>>>>>>>>> ),
> >>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> >>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
> >>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
> >>>>>>>>>>>>>>>>>> ).
> >>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
> >>>> Database
> >>>>>>>>>> schema
> >>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
> >>> etc.
> >>>>>>>>>>>>>>>>>>> Bug Fixes.
> >>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
> >>> above
> >>>>>>>> points
> >>>>>>>>>> or
> >>>>>>>>>>>>>>>>> more
> >>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
> >>> work
> >>>>>>>> which
> >>>>>>>>>> is
> >>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
> >>>>>>>>>> availability
> >>>>>>>>>>>>>>>>> of the
> >>>>>>>>>>>>>>>>>> reviewers).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>>> Abhishek
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

As previously discussed, we will be moving with whatever patches can be reviewed/merged till end of this week.

But there are some which needs to be reviewed and merged, mainly regarding the upgrades. Some of the patches require Java 1.7, so I think we should review OOZIE-2036 and merged it.

~Abhishek
> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy <ro...@gmail.com> wrote:
> 
> Cleaned up that list and now no jira refers to "trunk".  Patch Available
> jiras targeted for trunk now point to 4.4.0.
> 
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC
> 
> There are 42 jiras to go. We can take a look mid next week to see if any
> pending ones needs to be pushed out. There are quite a few which are
> unassigned.
> 
> If there is any jira, someone would like to include in 4.3.0 please update
> the jira.
> 
> Regards,
> Rohini
> 
> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <rohini.aditya@gmail.com
>> wrote:
> 
>> Unset the open ones without sending emails. Will update the resolved,
>> closed and Patch Available ones sending emails.
>> 
>> Regards,
>> Rohini
>> 
>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>> 
>>> Ya, let's unset those remaining open trunk ones.  I agree, we need to put
>>> in some effort to clean out old JIRAs and to review Patch Available JIRAs.
>>> 
>>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can discuss
>>> the issue more there, but I listed a few reasons on OOZIE-2625 about why.
>>> 
>>> - Robert
>>> 
>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>>> rohini.aditya@gmail.com>
>>> wrote:
>>> 
>>>> 
>>>> 
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%20status%20DESC&startIndex=50
>>>> 
>>>> These are jiras open and marked trunk. Should we unset fix version on
>>> these
>>>> or mark them for next release? Don't see majority of them being worked
>>> on
>>>> even for next release. We need to do bug scrub and close many of these
>>> some
>>>> time.
>>>> 
>>>>> Instead
>>>> of deleting "trunk", I think we're better off just renaming it to the
>>>> release number.
>>>> There is some amount of relabeling either way. If we go with unsetting
>>> fix
>>>> version, then we can as well delete trunk. Also, I did create a 4.4.0
>>>> yesterday to move some of the patch available jiras from 4.3.0.
>>>> 
>>>> 
>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>>>> rohini.aditya@gmail.com>
>>>> wrote:
>>>> 
>>>>> I think it would be better to send out email for the bulk change. If
>>>> there
>>>>> are any issues or jiras wrongly marked someone can respond. We do that
>>>> with
>>>>> Pig and there are always couple of jiras that folks come back with
>>>> feedback
>>>>> on pulling back to release.
>>>>> 
>>>>> 
>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <rk...@cloudera.com>
>>>>> wrote:
>>>>> 
>>>>>> Abhishek, next time you do a bulk change, please remember to tell it
>>> not
>>>>>> to
>>>>>> send out an email for every change :)
>>>>>> 
>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rk...@cloudera.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Rohini, that sounds good to me.  Especially because we're on
>>> "master"
>>>>>>> branch but "trunk" version in JIRA and it gets a little confusing.
>>>>>> Instead
>>>>>>> of deleting "trunk", I think we're better off just renaming it to
>>> the
>>>>>>> release number.  Otherwise, we'll have to re-label any current
>>> "trunk"
>>>>>>> fields.  We can use 4.4.0 for now, but I imagine that the next
>>> release
>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always
>>> rename it
>>>>>>> later.
>>>>>>> 
>>>>>>> I'm not sure if there's a specific rule here, but I suppose we
>>> should
>>>>>>> probably have a vote in the private list to make things official as
>>>>>> this is
>>>>>>> a release-management-related change.  Can you send an email?
>>>>>>> 
>>>>>>> Abhishek, thanks for putting this all together.  I think Rohini's
>>>>>>> suggestion of updating the JIRAs is the right next step.  You're
>>>>>> probably
>>>>>>> best off using the bulk change feature instead of doing each JIRA
>>>>>>> individually.  I should have time to do a bunch of reviews this
>>> week,
>>>>>> I've
>>>>>>> actually already started.  I should have some time to continue
>>> doing
>>>>>> more
>>>>>>> reviews.  As a general thing, we've been pretty slow about getting
>>>> these
>>>>>>> done :(
>>>>>>> 
>>>>>>> - Robert
>>>>>>> 
>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>>>>>>> rohini.aditya@gmail.com> wrote:
>>>>>>> 
>>>>>>>> I also did mark some of the enhancement jiras that Satish was
>>> working
>>>>>> on
>>>>>>>> and which might take time to review to 4.4.0
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Rohini
>>>>>>>> 
>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>>>>>>>> rohini.aditya@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Abhishek,
>>>>>>>>>    Can you mark all jiras already committed to trunk as 4.3.0
>>> and
>>>>>> those
>>>>>>>>> planned in the scope document as 4.3.0? This will help filter on
>>>>>> those
>>>>>>>>> and review instead of referring to a google doc. I can pick up
>>>> 10-15
>>>>>>>>> jiras for review end of this week. We should try to get
>>> majority of
>>>>>>>> patches
>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use the
>>>> 3rd
>>>>>>>> week
>>>>>>>>> for any pending critical issues and rolling the release
>>> candidate.
>>>>>>>>> 
>>>>>>>>> Robert,
>>>>>>>>>  I am thinking, that we should get rid of the trunk version and
>>>> use
>>>>>>>>> release numbers when committing from the beginning like all
>>> other
>>>>>>>> projects
>>>>>>>>> do. Unnecessary step to change from trunk to release version for
>>>>>> every
>>>>>>>>> release. If folks are in agreement, then will delete the trunk
>>>>>> version
>>>>>>>>> altogether. Does that require a mail to private?
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Rohini
>>>>>>>>> 
>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>>>>>> bafna.iitr@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Hi All,
>>>>>>>>>> 
>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document <
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>>>>>>>>> 
>>>>>>>>>> and listed all the things which we wanted to include in the
>>>> release.
>>>>>>>> In the
>>>>>>>>>> last few days, I spent some time and reviewed some of the
>>> patches
>>>>>> and
>>>>>>>>>> provided my feedback. I will continue my efforts to review more
>>>>>>>> patches.
>>>>>>>>>> These patches needs to be reviewed by committers and get
>>>> committed.
>>>>>> I
>>>>>>>>>> request other contributors also, review some of the patches and
>>>>>> provide
>>>>>>>>>> their feedback.
>>>>>>>>>> 
>>>>>>>>>> I am thinking, may be, we can decide upon some fixed
>>> (approximate)
>>>>>>>>>> release date and start working accordingly. Because, If we wait
>>>> for
>>>>>>>> all the
>>>>>>>>>> listed items to be reviewed and committed, then it might take a
>>>> lot
>>>>>>>> more
>>>>>>>>>> time. I am thinking we should target third week of August to
>>>>>> complete
>>>>>>>> and
>>>>>>>>>> release.
>>>>>>>>>> 
>>>>>>>>>> Feel free to provide your feedback and suggestions.
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> Abhishek
>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>>>> bafna.iitr@gmail.com
>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi,
>>>>>>>>>>> 
>>>>>>>>>>> I looked into the available patches and prepared the list. I
>>>> have
>>>>>>>> put,
>>>>>>>>>> scope content which we have discussed till now, into a word
>>>>>> document.
>>>>>>>> (For
>>>>>>>>>> the better formatting and easy to read and comment).
>>>>>>>>>>> 
>>>>>>>>>>> I have noted the additional Jira into the list. Added two
>>> more
>>>>>> Jira
>>>>>>>>>> related to spark (Scala version and Spark 2.0 property).
>>>>>>>>>>> 
>>>>>>>>>>> Please take a look at the document (link below) and provide
>>> your
>>>>>>>>>> comments.
>>>>>>>>>>> 
>>>>>>>>>>> If this looks good, we can start the release process.
>>>>>>>>>>> 
>>>>>>>>>>> Scope Document:
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Abhishek
>>>>>>>>>>> 
>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>>>> rkanter@cloudera.com
>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> That list looks good to me.  The only one I'm not sure of is
>>>>>>>> OOZIE-2306
>>>>>>>>>>>> (Oozie Health Check).  It's a new feature and it might make
>>>>>> sense to
>>>>>>>>>> let it
>>>>>>>>>>>> "bake" a while first?  I'm also not sure it's quite ready
>>> yet.
>>>>>>>>>>>> 
>>>>>>>>>>>> I'd also like to add a few others to the list:
>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and
>>> other
>>>>>>>> fixes)
>>>>>>>>>>>> 
>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your
>>>> list
>>>>>>>>>> (including
>>>>>>>>>>>> the ones I mentioned above).  While we don't need to include
>>>>>>>> everything
>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review
>>>>>>>> bandwidth),
>>>>>>>>>> did
>>>>>>>>>>>> you consider each of them when creating the list?  I
>>> currently
>>>>>> see
>>>>>>>> 97
>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available
>>>>>>>>>>>> I think we should include all Spark Action
>>> fixes/improvements,
>>>>>> given
>>>>>>>>>> how
>>>>>>>>>>>> popular it seems to be.
>>>>>>>>>>>> 
>>>>>>>>>>>> thanks
>>>>>>>>>>>> - Robert
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>>>>>>>> bafna.iitr@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Below is the list of issues, picked from currently
>>> available
>>>>>>>> patches.
>>>>>>>>>> I
>>>>>>>>>>>>> could not decide up any 'open' issue, If there is anything,
>>>>>> please
>>>>>>>>>> reply
>>>>>>>>>>>>> here.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback.
>>>>>> Important
>>>>>>>>>> is,
>>>>>>>>>>>>> how many of these we will be able to review and commit.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I will starting looking into these issues and provide
>>>> feedback.
>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> ---------------------------------------------------------------------------
>>>>>>>>>>>>> OOZIE-1978      Forkjoin validation code is ridiculously
>>> slow
>>>> in
>>>>>>>> some
>>>>>>>>>> cases
>>>>>>>>>>>>> OOZIE-2508      Documentation change for Coord action rerun
>>>>>>>>>> [OOZIE-1735]
>>>>>>>>>>>>> OOZIE-2583      oozie throws EL Exception when reference
>>>>>> variable
>>>>>>>> name
>>>>>>>>>>>>> containing dot
>>>>>>>>>>>>> OOZIE-2582      Populating external child Ids for action
>>>>>> failures
>>>>>>>>>>>>> OOZIE-2517      Add support for startCreatedTime and
>>>>>> endCreatedTime
>>>>>>>>>>>>> filters for coord and bundles
>>>>>>>>>>>>> OOZIE-2512      ShareLibservice returns incorrect path for
>>> jar
>>>>>>>>>>>>> OOZIE-2572      SLA DURATION miss not shown when job is
>>>> running
>>>>>> for
>>>>>>>>>> longer
>>>>>>>>>>>>> than expected time
>>>>>>>>>>>>> OOZIE-2457      Oozie log parsing regex consume more than
>>> 90%
>>>>>> cpu
>>>>>>>>>>>>> OOZIE-2574      Oozie to support replication-enabled mysql
>>>> urls
>>>>>>>>>>>>> OOZIE-2506      Add logs into RecoverService for logging
>>>>>>>> information
>>>>>>>>>> about
>>>>>>>>>>>>> queued commnads
>>>>>>>>>>>>> OOZIE-2573      dataset url which contains spaces can not
>>> be
>>>>>>>> handled
>>>>>>>>>>>>> rightly
>>>>>>>>>>>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>>>>>>>> mapred-site
>>>>>>>>>>>>> into spark launcher
>>>>>>>>>>>>> OOZIE-2491      oozie acl cannot specify group,it does`t
>>> work
>>>>>>>>>>>>> OOZIE-2539      Incorrect property key is used for 'hive
>>> log4j
>>>>>>>>>>>>> configuration file for execution mode'
>>>>>>>>>>>>> OOZIE-2555      Oozie SSL enable setup does not return port
>>>> for
>>>>>>>> admin
>>>>>>>>>>>>> -servers
>>>>>>>>>>>>> OOZIE-2515      Duplicate information for "Changing
>>>>>>>> endtime/pausetime
>>>>>>>>>> of a
>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki
>>>>>>>>>>>>> OOZIE-2498      Oozie CallerId configuration for downstream
>>>>>>>> components
>>>>>>>>>>>>> OOZIE-2516      Update web service documentation for jobs
>>> API
>>>>>>>>>>>>> OOZIE-2500      -DtestJarSimple option mentioned in
>>> minioozie
>>>>>> doc
>>>>>>>>>> does not
>>>>>>>>>>>>> work
>>>>>>>>>>>>> OOZIE-2513      log.scan.duration should not be used for
>>> error
>>>>>> and
>>>>>>>>>> audit
>>>>>>>>>>>>> logs
>>>>>>>>>>>>> OOZIE-2461      Workflow, Coordinator and Bundle job
>>> querying
>>>>>>>> should
>>>>>>>>>> have
>>>>>>>>>>>>> last modified filter
>>>>>>>>>>>>> OOZIE-2306      Oozie health check
>>>>>>>>>>>>> OOZIE-2518      Oozie is creating many files and
>>> directories
>>>> in
>>>>>>>> /tmp
>>>>>>>>>> per
>>>>>>>>>>>>> day
>>>>>>>>>>>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>>>>>>>>>>>> OOZIE-2473      Connection pool for SMTP connection
>>>>>>>>>>>>> OOZIE-2440      Exponential re-try policy for workflow
>>> action
>>>>>>>>>>>>> OOZIE-2458      'oozie-setup.sh sharelib create' should
>>> ensure
>>>>>>>>>> uploaded
>>>>>>>>>>>>> jars are world readable
>>>>>>>>>>>>> OOZIE-2390      Rerun with failed option removing completed
>>>>>> output
>>>>>>>>>> data
>>>>>>>>>>>>> OOZIE-1810      Workflow cannot get into Failed state when
>>>> kill
>>>>>>>>>> control
>>>>>>>>>>>>> node cannot resolve variable in message
>>>>>>>>>>>>> OOZIE-2244      Oozie should mask passwords in the logs
>>> when
>>>>>>>> logging
>>>>>>>>>>>>> command arguments
>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> ---------------------------------------------------------------------------
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>>>>>> bafna.iitr@gmail.com
>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good
>>> to
>>>> me
>>>>>> and
>>>>>>>>>>>>> should be included into the release. If there are things,
>>>> which
>>>>>>>> needs
>>>>>>>>>> to be
>>>>>>>>>>>>> excluded, provide your comments.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> We still need to decide on the patch which are in progress
>>>>>> status.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On the upgrade side:
>>>>>>>>>>>>>> Java version          1.6 to 1.7
>>>>>>>>>>>>>> OpenJPA version       2.2.2 to 2.4.1
>>>>>>>>>>>>>> Curator version       2.5.0 to 2.11.0
>>>>>>>>>>>>>> Pig version           0.12.1 to 0.16.0
>>>>>>>>>>>>>> Hive version          0.13.1 to 1.2.0
>>>>>>>>>>>>>> HttpClient version    4.3 to 4.3.6
>>>>>>>>>>>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Please provide your feedback on the above upgrades.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start
>>>>>> working
>>>>>>>> on
>>>>>>>>>> the
>>>>>>>>>>>>> completing/testing the release.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>> <ma...@gmail.com>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator,
>>>>>> HttpClient,
>>>>>>>> Pig
>>>>>>>>>> etc.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done
>>>>>> internally as
>>>>>>>>>> well.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 <
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
>>>>>> version
>>>>>>>>>> also
>>>>>>>>>>>>> can be upgraded to 2.11.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>>>>>>>> rkanter@cloudera.com
>>>>>>>>>>>>> <ma...@cloudera.com>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0
>>> for
>>>>>> over
>>>>>>>> a
>>>>>>>>>> year
>>>>>>>>>>>>> now
>>>>>>>>>>>>>>>> with no problems.  We did have to comment out a few
>>> lines
>>>> in
>>>>>>>>>>>>> MiniHCatServer
>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>>>>> 
>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>> 
>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644
>>>>>>>>>>>>>>>> ---
>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>> +++
>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>>>>>>>>>>>  public static void resetDefaultDBCreation() throws
>>>>>> Exception {
>>>>>>>>>>>>>>>>      // Need to do this, else default db will not be
>>>> created
>>>>>>>> for
>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>> metastores.
>>>>>>>>>>>>>>>>      // TestHiveMain will fail with
>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database
>>> named
>>>>>>>> default)
>>>>>>>>>>>>>>>> -        Field declaredField =
>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>> -        declaredField.setAccessible(true);
>>>>>>>>>>>>>>>> -        declaredField.set(null, false);
>>>>>>>>>>>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
>>> 0.14
>>>> and
>>>>>>>> later
>>>>>>>>>>>>>>>> +        //Field declaredField =
>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>>>>>>>>>>>> +        //declaredField.setAccessible(true);
>>>>>>>>>>>>>>>> +        //declaredField.set(null, false);
>>>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>  public static void resetHiveConfStaticVariables()
>>> throws
>>>>>>>>>> Exception
>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>> diff --git
>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>> 
>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644
>>>>>>>>>>>>>>>> ---
>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>> +++
>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>>>>>>>> AbstractHiveService
>>>>>>>>>> {
>>>>>>>>>>>>>>>> 
>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>>>>>>>>>> getHost());
>>>>>>>>>>>>>>>>  hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
>>>>>>>>>>>>> getBinaryPort());
>>>>>>>>>>>>>>>> 
>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>>>>>>>>>>>> getHttpPort());
>>>>>>>>>>>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
>>> and
>>>>>> later
>>>>>>>>>>>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>  Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>>>>>>>>>>>  fs.mkdirs(scratchDir);
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>>>>>>>>>>>> vranganathan@hortonworks.com <mailto:
>>>>>>>> vranganathan@hortonworks.com
>>>>>>>>>>>> 
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also
>>> to
>>>>>> 1.x.
>>>>>>>>>> 0.13
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> old
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Venkat
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>>>> rkanter@cloudera.com
>>>>>>>>>> <mailto:
>>>>>>>>>>>>> rkanter@cloudera.com>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere
>>>> about
>>>>>>>> when
>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>> okay
>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we
>>> need
>>>> to
>>>>>>>> wait
>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> next major release (5.0)?  That said, Hadoop did this
>>> in
>>>> a
>>>>>>>> minor
>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013.  So I'm fine with doing
>>> it in
>>>>>> 4.3
>>>>>>>> if
>>>>>>>>>>>>>>>>> nobody has
>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that
>>> out
>>>> in
>>>>>>>> our
>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>> notes.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we
>>> should
>>>>>> leave
>>>>>>>> out
>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently
>>> in
>>>>>> trunk
>>>>>>>>>>>>> anyway.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we
>>> should
>>>>>> make
>>>>>>>>>> sure
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> include all of these and any pending ones.
>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more
>>>>>> time to
>>>>>>>>>> bake
>>>>>>>>>>>>>>>>> that we
>>>>>>>>>>>>>>>>> should hold off on including in the release?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>> - Robert
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>>>>>>>>>>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the
>>> release?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>>>>>>>>>> bafna.iitr@gmail.com
>>>>>>>>>>>>> <ma...@gmail.com>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie
>>> 4.3.0.
>>>>>> This
>>>>>>>>>> thread
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please
>>>>>> provide
>>>>>>>>>> your
>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any
>>>>>> exclusions)
>>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to
>>>> trunk.
>>>>>>>> Based
>>>>>>>>>> on
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>>>>>>>>>>>> Version Upgrade
>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>>>>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>>>>>>>>>>>> ),
>>>>>>>>>>>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 <
>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>>>>>>>>>>>> ).
>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN,
>>>> Database
>>>>>>>>>> schema
>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade
>>> etc.
>>>>>>>>>>>>>>>>>>> Bug Fixes.
>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the
>>> above
>>>>>>>> points
>>>>>>>>>> or
>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some
>>> work
>>>>>>>> which
>>>>>>>>>> is
>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the
>>>>>>>>>> availability
>>>>>>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>>>> reviewers).
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Abhishek
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy <ro...@gmail.com>]

Cleaned up that list and now no jira refers to "trunk".  Patch Available
jiras targeted for trunk now point to 4.4.0.

https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status%20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC

There are 42 jiras to go. We can take a look mid next week to see if any
pending ones needs to be pushed out. There are quite a few which are
unassigned.

If there is any jira, someone would like to include in 4.3.0 please update
the jira.

Regards,
Rohini

On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy <rohini.aditya@gmail.com
> wrote:

> Unset the open ones without sending emails. Will update the resolved,
> closed and Patch Available ones sending emails.
>
> Regards,
> Rohini
>
> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
>> Ya, let's unset those remaining open trunk ones.  I agree, we need to put
>> in some effort to clean out old JIRAs and to review Patch Available JIRAs.
>>
>> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can discuss
>> the issue more there, but I listed a few reasons on OOZIE-2625 about why.
>>
>> - Robert
>>
>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
>> rohini.aditya@gmail.com>
>> wrote:
>>
>> >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%20status%20DESC&startIndex=50
>> >
>> > These are jiras open and marked trunk. Should we unset fix version on
>> these
>> > or mark them for next release? Don't see majority of them being worked
>> on
>> > even for next release. We need to do bug scrub and close many of these
>> some
>> > time.
>> >
>> > > Instead
>> > of deleting "trunk", I think we're better off just renaming it to the
>> > release number.
>> >  There is some amount of relabeling either way. If we go with unsetting
>> fix
>> > version, then we can as well delete trunk. Also, I did create a 4.4.0
>> > yesterday to move some of the patch available jiras from 4.3.0.
>> >
>> >
>> > On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
>> > rohini.aditya@gmail.com>
>> > wrote:
>> >
>> > > I think it would be better to send out email for the bulk change. If
>> > there
>> > > are any issues or jiras wrongly marked someone can respond. We do that
>> > with
>> > > Pig and there are always couple of jiras that folks come back with
>> > feedback
>> > > on pulling back to release.
>> > >
>> > >
>> > > On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <rk...@cloudera.com>
>> > > wrote:
>> > >
>> > >> Abhishek, next time you do a bulk change, please remember to tell it
>> not
>> > >> to
>> > >> send out an email for every change :)
>> > >>
>> > >> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rk...@cloudera.com>
>> > >> wrote:
>> > >>
>> > >> > Rohini, that sounds good to me.  Especially because we're on
>> "master"
>> > >> > branch but "trunk" version in JIRA and it gets a little confusing.
>> > >> Instead
>> > >> > of deleting "trunk", I think we're better off just renaming it to
>> the
>> > >> > release number.  Otherwise, we'll have to re-label any current
>> "trunk"
>> > >> > fields.  We can use 4.4.0 for now, but I imagine that the next
>> release
>> > >> > might end up being 5.0.0 (with Oozie on Yarn); we can always
>> rename it
>> > >> > later.
>> > >> >
>> > >> > I'm not sure if there's a specific rule here, but I suppose we
>> should
>> > >> > probably have a vote in the private list to make things official as
>> > >> this is
>> > >> > a release-management-related change.  Can you send an email?
>> > >> >
>> > >> > Abhishek, thanks for putting this all together.  I think Rohini's
>> > >> > suggestion of updating the JIRAs is the right next step.  You're
>> > >> probably
>> > >> > best off using the bulk change feature instead of doing each JIRA
>> > >> > individually.  I should have time to do a bunch of reviews this
>> week,
>> > >> I've
>> > >> > actually already started.  I should have some time to continue
>> doing
>> > >> more
>> > >> > reviews.  As a general thing, we've been pretty slow about getting
>> > these
>> > >> > done :(
>> > >> >
>> > >> > - Robert
>> > >> >
>> > >> > On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>> > >> > rohini.aditya@gmail.com> wrote:
>> > >> >
>> > >> >> I also did mark some of the enhancement jiras that Satish was
>> working
>> > >> on
>> > >> >> and which might take time to review to 4.4.0
>> > >> >>
>> > >> >> Regards,
>> > >> >> Rohini
>> > >> >>
>> > >> >> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>> > >> >> rohini.aditya@gmail.com>
>> > >> >> wrote:
>> > >> >>
>> > >> >> > Abhishek,
>> > >> >> >     Can you mark all jiras already committed to trunk as 4.3.0
>> and
>> > >> those
>> > >> >> > planned in the scope document as 4.3.0? This will help filter on
>> > >> those
>> > >> >> > and review instead of referring to a google doc. I can pick up
>> > 10-15
>> > >> >> > jiras for review end of this week. We should try to get
>> majority of
>> > >> >> patches
>> > >> >> > in the next 10 days and branch by Aug 12th. You can then use the
>> > 3rd
>> > >> >> week
>> > >> >> > for any pending critical issues and rolling the release
>> candidate.
>> > >> >> >
>> > >> >> > Robert,
>> > >> >> >   I am thinking, that we should get rid of the trunk version and
>> > use
>> > >> >> > release numbers when committing from the beginning like all
>> other
>> > >> >> projects
>> > >> >> > do. Unnecessary step to change from trunk to release version for
>> > >> every
>> > >> >> > release. If folks are in agreement, then will delete the trunk
>> > >> version
>> > >> >> > altogether. Does that require a mail to private?
>> > >> >> >
>> > >> >> > Regards,
>> > >> >> > Rohini
>> > >> >> >
>> > >> >> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>> > >> bafna.iitr@gmail.com>
>> > >> >> > wrote:
>> > >> >> >
>> > >> >> >> Hi All,
>> > >> >> >>
>> > >> >> >> As we prepared a Oozie 4.3.0 scope document <
>> > >> >> >>
>> > >> >>
>> > >>
>> >
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>> > >> >> >
>> > >> >> >> and listed all the things which we wanted to include in the
>> > release.
>> > >> >> In the
>> > >> >> >> last few days, I spent some time and reviewed some of the
>> patches
>> > >> and
>> > >> >> >> provided my feedback. I will continue my efforts to review more
>> > >> >> patches.
>> > >> >> >> These patches needs to be reviewed by committers and get
>> > committed.
>> > >> I
>> > >> >> >> request other contributors also, review some of the patches and
>> > >> provide
>> > >> >> >> their feedback.
>> > >> >> >>
>> > >> >> >> I am thinking, may be, we can decide upon some fixed
>> (approximate)
>> > >> >> >> release date and start working accordingly. Because, If we wait
>> > for
>> > >> >> all the
>> > >> >> >> listed items to be reviewed and committed, then it might take a
>> > lot
>> > >> >> more
>> > >> >> >> time. I am thinking we should target third week of August to
>> > >> complete
>> > >> >> and
>> > >> >> >> release.
>> > >> >> >>
>> > >> >> >> Feel free to provide your feedback and suggestions.
>> > >> >> >>
>> > >> >> >> Thanks,
>> > >> >> >> Abhishek
>> > >> >> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
>> > bafna.iitr@gmail.com
>> > >> >
>> > >> >> >> wrote:
>> > >> >> >> >
>> > >> >> >> > Hi,
>> > >> >> >> >
>> > >> >> >> > I looked into the available patches and prepared the list. I
>> > have
>> > >> >> put,
>> > >> >> >> scope content which we have discussed till now, into a word
>> > >> document.
>> > >> >> (For
>> > >> >> >> the better formatting and easy to read and comment).
>> > >> >> >> >
>> > >> >> >> > I have noted the additional Jira into the list. Added two
>> more
>> > >> Jira
>> > >> >> >> related to spark (Scala version and Spark 2.0 property).
>> > >> >> >> >
>> > >> >> >> > Please take a look at the document (link below) and provide
>> your
>> > >> >> >> comments.
>> > >> >> >> >
>> > >> >> >> > If this looks good, we can start the release process.
>> > >> >> >> >
>> > >> >> >> > Scope Document:
>> > >> >> >>
>> > >> >>
>> > >>
>> >
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
>> > >> >> >> >
>> > >> >> >> > Thanks,
>> > >> >> >> > Abhishek
>> > >> >> >> >
>> > >> >> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
>> > rkanter@cloudera.com
>> > >> >
>> > >> >> >> wrote:
>> > >> >> >> >>
>> > >> >> >> >> That list looks good to me.  The only one I'm not sure of is
>> > >> >> OOZIE-2306
>> > >> >> >> >> (Oozie Health Check).  It's a new feature and it might make
>> > >> sense to
>> > >> >> >> let it
>> > >> >> >> >> "bake" a while first?  I'm also not sure it's quite ready
>> yet.
>> > >> >> >> >>
>> > >> >> >> >> I'd also like to add a few others to the list:
>> > >> >> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>> > >> >> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>> > >> >> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>> > >> >> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>> > >> >> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and
>> other
>> > >> >> fixes)
>> > >> >> >> >>
>> > >> >> >> >> There's a number of other Patch Available JIRAs not on your
>> > list
>> > >> >> >> (including
>> > >> >> >> >> the ones I mentioned above).  While we don't need to include
>> > >> >> everything
>> > >> >> >> >> here (and we probably shouldn't nor do we have the review
>> > >> >> bandwidth),
>> > >> >> >> did
>> > >> >> >> >> you consider each of them when creating the list?  I
>> currently
>> > >> see
>> > >> >> 97
>> > >> >> >> >> JIRAs: https://s.apache.org/oozie-patch-available
>> > >> >> >> >> I think we should include all Spark Action
>> fixes/improvements,
>> > >> given
>> > >> >> >> how
>> > >> >> >> >> popular it seems to be.
>> > >> >> >> >>
>> > >> >> >> >> thanks
>> > >> >> >> >> - Robert
>> > >> >> >> >>
>> > >> >> >> >>
>> > >> >> >> >>
>> > >> >> >> >>
>> > >> >> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>> > >> >> bafna.iitr@gmail.com>
>> > >> >> >> >> wrote:
>> > >> >> >> >>
>> > >> >> >> >>> Hi All,
>> > >> >> >> >>>
>> > >> >> >> >>> Below is the list of issues, picked from currently
>> available
>> > >> >> patches.
>> > >> >> >> I
>> > >> >> >> >>> could not decide up any 'open' issue, If there is anything,
>> > >> please
>> > >> >> >> reply
>> > >> >> >> >>> here.
>> > >> >> >> >>>
>> > >> >> >> >>> We can 'drop/add' more if required based on the feedback.
>> > >> Important
>> > >> >> >> is,
>> > >> >> >> >>> how many of these we will be able to review and commit.
>> > >> >> >> >>>
>> > >> >> >> >>> I will starting looking into these issues and provide
>> > feedback.
>> > >> >> >> >>>
>> > >> >> >>
>> > >> >>
>> > >>
>> >
>> ---------------------------------------------------------------------------
>> > >> >> >> >>> OOZIE-1978      Forkjoin validation code is ridiculously
>> slow
>> > in
>> > >> >> some
>> > >> >> >> cases
>> > >> >> >> >>> OOZIE-2508      Documentation change for Coord action rerun
>> > >> >> >> [OOZIE-1735]
>> > >> >> >> >>> OOZIE-2583      oozie throws EL Exception when reference
>> > >> variable
>> > >> >> name
>> > >> >> >> >>> containing dot
>> > >> >> >> >>> OOZIE-2582      Populating external child Ids for action
>> > >> failures
>> > >> >> >> >>> OOZIE-2517      Add support for startCreatedTime and
>> > >> endCreatedTime
>> > >> >> >> >>> filters for coord and bundles
>> > >> >> >> >>> OOZIE-2512      ShareLibservice returns incorrect path for
>> jar
>> > >> >> >> >>> OOZIE-2572      SLA DURATION miss not shown when job is
>> > running
>> > >> for
>> > >> >> >> longer
>> > >> >> >> >>> than expected time
>> > >> >> >> >>> OOZIE-2457      Oozie log parsing regex consume more than
>> 90%
>> > >> cpu
>> > >> >> >> >>> OOZIE-2574      Oozie to support replication-enabled mysql
>> > urls
>> > >> >> >> >>> OOZIE-2506      Add logs into RecoverService for logging
>> > >> >> information
>> > >> >> >> about
>> > >> >> >> >>> queued commnads
>> > >> >> >> >>> OOZIE-2573      dataset url which contains spaces can not
>> be
>> > >> >> handled
>> > >> >> >> >>> rightly
>> > >> >> >> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>> > >> >> mapred-site
>> > >> >> >> >>> into spark launcher
>> > >> >> >> >>> OOZIE-2491      oozie acl cannot specify group,it does`t
>> work
>> > >> >> >> >>> OOZIE-2539      Incorrect property key is used for 'hive
>> log4j
>> > >> >> >> >>> configuration file for execution mode'
>> > >> >> >> >>> OOZIE-2555      Oozie SSL enable setup does not return port
>> > for
>> > >> >> admin
>> > >> >> >> >>> -servers
>> > >> >> >> >>> OOZIE-2515      Duplicate information for "Changing
>> > >> >> endtime/pausetime
>> > >> >> >> of a
>> > >> >> >> >>> Bundle Job" in CommandLineTool wiki
>> > >> >> >> >>> OOZIE-2498      Oozie CallerId configuration for downstream
>> > >> >> components
>> > >> >> >> >>> OOZIE-2516      Update web service documentation for jobs
>> API
>> > >> >> >> >>> OOZIE-2500      -DtestJarSimple option mentioned in
>> minioozie
>> > >> doc
>> > >> >> >> does not
>> > >> >> >> >>> work
>> > >> >> >> >>> OOZIE-2513      log.scan.duration should not be used for
>> error
>> > >> and
>> > >> >> >> audit
>> > >> >> >> >>> logs
>> > >> >> >> >>> OOZIE-2461      Workflow, Coordinator and Bundle job
>> querying
>> > >> >> should
>> > >> >> >> have
>> > >> >> >> >>> last modified filter
>> > >> >> >> >>> OOZIE-2306      Oozie health check
>> > >> >> >> >>> OOZIE-2518      Oozie is creating many files and
>> directories
>> > in
>> > >> >> /tmp
>> > >> >> >> per
>> > >> >> >> >>> day
>> > >> >> >> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>> > >> >> >> >>> OOZIE-2473      Connection pool for SMTP connection
>> > >> >> >> >>> OOZIE-2440      Exponential re-try policy for workflow
>> action
>> > >> >> >> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should
>> ensure
>> > >> >> >> uploaded
>> > >> >> >> >>> jars are world readable
>> > >> >> >> >>> OOZIE-2390      Rerun with failed option removing completed
>> > >> output
>> > >> >> >> data
>> > >> >> >> >>> OOZIE-1810      Workflow cannot get into Failed state when
>> > kill
>> > >> >> >> control
>> > >> >> >> >>> node cannot resolve variable in message
>> > >> >> >> >>> OOZIE-2244      Oozie should mask passwords in the logs
>> when
>> > >> >> logging
>> > >> >> >> >>> command arguments
>> > >> >> >> >>>
>> > >> >> >>
>> > >> >>
>> > >>
>> >
>> ---------------------------------------------------------------------------
>> > >> >> >> >>> Thanks,
>> > >> >> >> >>> Abhishek
>> > >> >> >> >>>
>> > >> >> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>> > >> bafna.iitr@gmail.com
>> > >> >> >
>> > >> >> >> >>> wrote:
>> > >> >> >> >>>>
>> > >> >> >> >>>> Hi All,
>> > >> >> >> >>>>
>> > >> >> >> >>>> I looked into the commits (approx. 160), all looked good
>> to
>> > me
>> > >> and
>> > >> >> >> >>> should be included into the release. If there are things,
>> > which
>> > >> >> needs
>> > >> >> >> to be
>> > >> >> >> >>> excluded, provide your comments.
>> > >> >> >> >>>>
>> > >> >> >> >>>> We still need to decide on the patch which are in progress
>> > >> status.
>> > >> >> >> >>>>
>> > >> >> >> >>>> On the upgrade side:
>> > >> >> >> >>>> Java version          1.6 to 1.7
>> > >> >> >> >>>> OpenJPA version       2.2.2 to 2.4.1
>> > >> >> >> >>>> Curator version       2.5.0 to 2.11.0
>> > >> >> >> >>>> Pig version           0.12.1 to 0.16.0
>> > >> >> >> >>>> Hive version          0.13.1 to 1.2.0
>> > >> >> >> >>>> HttpClient version    4.3 to 4.3.6
>> > >> >> >> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>> > >> >> >> >>>>
>> > >> >> >> >>>> Please provide your feedback on the above upgrades.
>> > >> >> >> >>>>
>> > >> >> >> >>>> Once we decide on the scope of the release. We will start
>> > >> working
>> > >> >> on
>> > >> >> >> the
>> > >> >> >> >>> completing/testing the release.
>> > >> >> >> >>>>
>> > >> >> >> >>>> Thanks,
>> > >> >> >> >>>> Abhishek
>> > >> >> >> >>>>
>> > >> >> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>> > >> >> bafna.iitr@gmail.com
>> > >> >> >> >>> <ma...@gmail.com>> wrote:
>> > >> >> >> >>>>>
>> > >> >> >> >>>>> Okay. We will make the Java version to 1.7.
>> > >> >> >> >>>>>
>> > >> >> >> >>>>> There are upgrades we can do like OpenJPA, Curator,
>> > >> HttpClient,
>> > >> >> Pig
>> > >> >> >> etc.
>> > >> >> >> >>>>>
>> > >> >> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done
>> > >> internally as
>> > >> >> >> well.
>> > >> >> >> >>>>>
>> > >> >> >> >>>>> As per the comments on the OOZIE-2231 <
>> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
>> > >> version
>> > >> >> >> also
>> > >> >> >> >>> can be upgraded to 2.11.
>> > >> >> >> >>>>>
>> > >> >> >> >>>>> Thanks,
>> > >> >> >> >>>>> Abhishek
>> > >> >> >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>> > >> >> rkanter@cloudera.com
>> > >> >> >> >>> <ma...@cloudera.com>> wrote:
>> > >> >> >> >>>>>>
>> > >> >> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0
>> for
>> > >> over
>> > >> >> a
>> > >> >> >> year
>> > >> >> >> >>> now
>> > >> >> >> >>>>>> with no problems.  We did have to comment out a few
>> lines
>> > in
>> > >> >> >> >>> MiniHCatServer
>> > >> >> >> >>>>>> and MiniHS2 that were no longer needed:
>> > >> >> >> >>>>>>
>> > >> >> >> >>>>>> diff --git
>> > >> >> >> >>>
>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> > >> >> >> >>>>>>
>> > >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> > >> >> >> >>>>>> index 8699ff8..d694977 100644
>> > >> >> >> >>>>>> ---
>> > >> >> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> > >> >> >> >>>>>> +++
>> > >> >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> > >> >> >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>> > >> >> >> >>>>>>   public static void resetDefaultDBCreation() throws
>> > >> Exception {
>> > >> >> >> >>>>>>       // Need to do this, else default db will not be
>> > created
>> > >> >> for
>> > >> >> >> >>> local
>> > >> >> >> >>>>>> metastores.
>> > >> >> >> >>>>>>       // TestHiveMain will fail with
>> > >> >> >> >>>>>> InvalidObjectException(message:There is no database
>> named
>> > >> >> default)
>> > >> >> >> >>>>>> -        Field declaredField =
>> > >> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> > >> >> >> >>>>>> -        declaredField.setAccessible(true);
>> > >> >> >> >>>>>> -        declaredField.set(null, false);
>> > >> >> >> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive
>> 0.14
>> > and
>> > >> >> later
>> > >> >> >> >>>>>> +        //Field declaredField =
>> > >> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> > >> >> >> >>>>>> +        //declaredField.setAccessible(true);
>> > >> >> >> >>>>>> +        //declaredField.set(null, false);
>> > >> >> >> >>>>>>   }
>> > >> >> >> >>>>>>
>> > >> >> >> >>>>>>   public static void resetHiveConfStaticVariables()
>> throws
>> > >> >> >> Exception
>> > >> >> >> >>> {
>> > >> >> >> >>>>>> diff --git
>> > >> >> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> > >> >> >> >>>>>>
>> > b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> > >> >> >> >>>>>> index c9dfd56..505dc55 100644
>> > >> >> >> >>>>>> ---
>> > >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> > >> >> >> >>>>>> +++
>> > >> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> > >> >> >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>> > >> >> AbstractHiveService
>> > >> >> >> {
>> > >> >> >> >>>>>>
>>  hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>> > >> >> >> getHost());
>> > >> >> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
>> > >> >> >> >>> getBinaryPort());
>> > >> >> >> >>>>>>
>> >  hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>> > >> >> >> >>>>>> getHttpPort());
>> > >> >> >> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> > >> >> >> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14
>> and
>> > >> later
>> > >> >> >> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> > >> >> >> >>>>>>
>> > >> >> >> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
>> > >> >> >> >>>>>>   fs.mkdirs(scratchDir);
>> > >> >> >> >>>>>>
>> > >> >> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>> > >> >> >> >>>>>> vranganathan@hortonworks.com <mailto:
>> > >> >> vranganathan@hortonworks.com
>> > >> >> >> >>
>> > >> >> >> >>> wrote:
>> > >> >> >> >>>>>>
>> > >> >> >> >>>>>>> Let us explore changing the default hive version also
>> to
>> > >> 1.x.
>> > >> >> >>  0.13
>> > >> >> >> >>> is
>> > >> >> >> >>>>>>> old
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>> Thanks
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>> Venkat
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
>> > rkanter@cloudera.com
>> > >> >> >> <mailto:
>> > >> >> >> >>> rkanter@cloudera.com>> wrote:
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>  I don't think we have any guidelines defined anywhere
>> > about
>> > >> >> when
>> > >> >> >> >>> it's
>> > >> >> >> >>>>>>> okay
>> > >> >> >> >>>>>>>  to change the minimum Java version, but perhaps we
>> need
>> > to
>> > >> >> wait
>> > >> >> >> >>> until
>> > >> >> >> >>>>>>> the
>> > >> >> >> >>>>>>>  next major release (5.0)?  That said, Hadoop did this
>> in
>> > a
>> > >> >> minor
>> > >> >> >> >>>>>>> release
>> > >> >> >> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing
>> it in
>> > >> 4.3
>> > >> >> if
>> > >> >> >> >>>>>>> nobody has
>> > >> >> >> >>>>>>>  an objections; we should just make sure to call that
>> out
>> > in
>> > >> >> our
>> > >> >> >> >>> release
>> > >> >> >> >>>>>>>  notes.
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>  Otherwise, nothing comes to mind for anything we
>> should
>> > >> leave
>> > >> >> out
>> > >> >> >> >>> other
>> > >> >> >> >>>>>>>  than the Oozie on Yarn stuff, which is not currently
>> in
>> > >> trunk
>> > >> >> >> >>> anyway.
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>  I know there's a number of Spark Action JIRAs; we
>> should
>> > >> make
>> > >> >> >> sure
>> > >> >> >> >>> to
>> > >> >> >> >>>>>>>  include all of these and any pending ones.
>> > >> >> >> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more
>> > >> time to
>> > >> >> >> bake
>> > >> >> >> >>>>>>> that we
>> > >> >> >> >>>>>>>  should hold off on including in the release?
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>  thanks
>> > >> >> >> >>>>>>>  - Robert
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>> > >> >> >> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
>> > >> >> >> >>>>>>>  wrote:
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>> Hi All,
>> > >> >> >> >>>>>>>>
>> > >> >> >> >>>>>>>> Any suggestions or comments on the scope for the
>> release?
>> > >> >> >> >>>>>>>>
>> > >> >> >> >>>>>>>> Thanks,
>> > >> >> >> >>>>>>>> Abhishek
>> > >> >> >> >>>>>>>>
>> > >> >> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>> > >> >> >> bafna.iitr@gmail.com
>> > >> >> >> >>> <ma...@gmail.com>>
>> > >> >> >> >>>>>>>> wrote:
>> > >> >> >> >>>>>>>>>
>> > >> >> >> >>>>>>>>> Hi All,
>> > >> >> >> >>>>>>>>>
>> > >> >> >> >>>>>>>>> Oozie is starting the release process for Oozie
>> 4.3.0.
>> > >> This
>> > >> >> >> thread
>> > >> >> >> >>>>>>> is
>> > >> >> >> >>>>>>>> for discussion about the scope of the release. Please
>> > >> provide
>> > >> >> >> your
>> > >> >> >> >>>>>>>> suggestion about what should be included (or any
>> > >> exclusions)
>> > >> >> in
>> > >> >> >> the
>> > >> >> >> >>>>>>> release.
>> > >> >> >> >>>>>>>>>
>> > >> >> >> >>>>>>>>> Currently, we have around 158 patches submitted to
>> > trunk.
>> > >> >> Based
>> > >> >> >> on
>> > >> >> >> >>>>>>> the
>> > >> >> >> >>>>>>>> scope, we will include (exclude) more if required.
>> > >> >> >> >>>>>>>>> Version Upgrade
>> > >> >> >> >>>>>>>>> Java 1.7 (OOZIE-2036 <
>> > >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>> > >> >> >> >>>>>>>>> OpenJPA (OOZIE-2488 <
>> > >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>> > >> >> >> >>>>>>>>> Curator  (OOZIE-2231 <
>> > >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
>> > >> >> >> >>>>>>>>> ),
>> > >> >> >> >>>>>>>>> HttpClient  (OOZIE-2538 <
>> > >> >> >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>> > >> >> >> >>>>>>>>> Pig (OOZIE-2602 <
>> > >> >> >> https://issues.apache.org/jira/browse/OOZIE-2602 <
>> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
>> > >> >> >> >>>>>>>> ).
>> > >> >> >> >>>>>>>>> New Features/Improvements: Add default JT and NN,
>> > Database
>> > >> >> >> schema
>> > >> >> >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade
>> etc.
>> > >> >> >> >>>>>>>>> Bug Fixes.
>> > >> >> >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the
>> above
>> > >> >> points
>> > >> >> >> or
>> > >> >> >> >>>>>>> more
>> > >> >> >> >>>>>>>> for the scope of the release. We can also, take some
>> work
>> > >> >> which
>> > >> >> >> is
>> > >> >> >> >>>>>>> already
>> > >> >> >> >>>>>>>> done and review is pending for them (depending on the
>> > >> >> >> availability
>> > >> >> >> >>>>>>> of the
>> > >> >> >> >>>>>>>> reviewers).
>> > >> >> >> >>>>>>>>>
>> > >> >> >> >>>>>>>>> Thanks,
>> > >> >> >> >>>>>>>>> Abhishek
>> > >> >> >> >>>>>>>>
>> > >> >> >> >>>>>>>>
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>>>
>> > >> >> >> >>>>>
>> > >> >> >> >>>>
>> > >> >> >> >>>
>> > >> >> >> >>>
>> > >> >> >> >
>> > >> >> >>
>> > >> >> >>
>> > >> >> >
>> > >> >>
>> > >> >
>> > >> >
>> > >>
>> > >
>> > >
>> >
>>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy <ro...@gmail.com>]

Unset the open ones without sending emails. Will update the resolved,
closed and Patch Available ones sending emails.

Regards,
Rohini

On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <rk...@cloudera.com> wrote:

> Ya, let's unset those remaining open trunk ones.  I agree, we need to put
> in some effort to clean out old JIRAs and to review Patch Available JIRAs.
>
> Also, I just created OOZIE-2625 to drop workflowgenerator.  We can discuss
> the issue more there, but I listed a few reasons on OOZIE-2625 about why.
>
> - Robert
>
> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <
> rohini.aditya@gmail.com>
> wrote:
>
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%20status%20DESC&startIndex=50
> >
> > These are jiras open and marked trunk. Should we unset fix version on
> these
> > or mark them for next release? Don't see majority of them being worked on
> > even for next release. We need to do bug scrub and close many of these
> some
> > time.
> >
> > > Instead
> > of deleting "trunk", I think we're better off just renaming it to the
> > release number.
> >  There is some amount of relabeling either way. If we go with unsetting
> fix
> > version, then we can as well delete trunk. Also, I did create a 4.4.0
> > yesterday to move some of the patch available jiras from 4.3.0.
> >
> >
> > On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> > rohini.aditya@gmail.com>
> > wrote:
> >
> > > I think it would be better to send out email for the bulk change. If
> > there
> > > are any issues or jiras wrongly marked someone can respond. We do that
> > with
> > > Pig and there are always couple of jiras that folks come back with
> > feedback
> > > on pulling back to release.
> > >
> > >
> > > On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <rk...@cloudera.com>
> > > wrote:
> > >
> > >> Abhishek, next time you do a bulk change, please remember to tell it
> not
> > >> to
> > >> send out an email for every change :)
> > >>
> > >> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rk...@cloudera.com>
> > >> wrote:
> > >>
> > >> > Rohini, that sounds good to me.  Especially because we're on
> "master"
> > >> > branch but "trunk" version in JIRA and it gets a little confusing.
> > >> Instead
> > >> > of deleting "trunk", I think we're better off just renaming it to
> the
> > >> > release number.  Otherwise, we'll have to re-label any current
> "trunk"
> > >> > fields.  We can use 4.4.0 for now, but I imagine that the next
> release
> > >> > might end up being 5.0.0 (with Oozie on Yarn); we can always rename
> it
> > >> > later.
> > >> >
> > >> > I'm not sure if there's a specific rule here, but I suppose we
> should
> > >> > probably have a vote in the private list to make things official as
> > >> this is
> > >> > a release-management-related change.  Can you send an email?
> > >> >
> > >> > Abhishek, thanks for putting this all together.  I think Rohini's
> > >> > suggestion of updating the JIRAs is the right next step.  You're
> > >> probably
> > >> > best off using the bulk change feature instead of doing each JIRA
> > >> > individually.  I should have time to do a bunch of reviews this
> week,
> > >> I've
> > >> > actually already started.  I should have some time to continue doing
> > >> more
> > >> > reviews.  As a general thing, we've been pretty slow about getting
> > these
> > >> > done :(
> > >> >
> > >> > - Robert
> > >> >
> > >> > On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> > >> > rohini.aditya@gmail.com> wrote:
> > >> >
> > >> >> I also did mark some of the enhancement jiras that Satish was
> working
> > >> on
> > >> >> and which might take time to review to 4.4.0
> > >> >>
> > >> >> Regards,
> > >> >> Rohini
> > >> >>
> > >> >> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> > >> >> rohini.aditya@gmail.com>
> > >> >> wrote:
> > >> >>
> > >> >> > Abhishek,
> > >> >> >     Can you mark all jiras already committed to trunk as 4.3.0
> and
> > >> those
> > >> >> > planned in the scope document as 4.3.0? This will help filter on
> > >> those
> > >> >> > and review instead of referring to a google doc. I can pick up
> > 10-15
> > >> >> > jiras for review end of this week. We should try to get majority
> of
> > >> >> patches
> > >> >> > in the next 10 days and branch by Aug 12th. You can then use the
> > 3rd
> > >> >> week
> > >> >> > for any pending critical issues and rolling the release
> candidate.
> > >> >> >
> > >> >> > Robert,
> > >> >> >   I am thinking, that we should get rid of the trunk version and
> > use
> > >> >> > release numbers when committing from the beginning like all other
> > >> >> projects
> > >> >> > do. Unnecessary step to change from trunk to release version for
> > >> every
> > >> >> > release. If folks are in agreement, then will delete the trunk
> > >> version
> > >> >> > altogether. Does that require a mail to private?
> > >> >> >
> > >> >> > Regards,
> > >> >> > Rohini
> > >> >> >
> > >> >> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> > >> bafna.iitr@gmail.com>
> > >> >> > wrote:
> > >> >> >
> > >> >> >> Hi All,
> > >> >> >>
> > >> >> >> As we prepared a Oozie 4.3.0 scope document <
> > >> >> >>
> > >> >>
> > >>
> >
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> > >> >> >
> > >> >> >> and listed all the things which we wanted to include in the
> > release.
> > >> >> In the
> > >> >> >> last few days, I spent some time and reviewed some of the
> patches
> > >> and
> > >> >> >> provided my feedback. I will continue my efforts to review more
> > >> >> patches.
> > >> >> >> These patches needs to be reviewed by committers and get
> > committed.
> > >> I
> > >> >> >> request other contributors also, review some of the patches and
> > >> provide
> > >> >> >> their feedback.
> > >> >> >>
> > >> >> >> I am thinking, may be, we can decide upon some fixed
> (approximate)
> > >> >> >> release date and start working accordingly. Because, If we wait
> > for
> > >> >> all the
> > >> >> >> listed items to be reviewed and committed, then it might take a
> > lot
> > >> >> more
> > >> >> >> time. I am thinking we should target third week of August to
> > >> complete
> > >> >> and
> > >> >> >> release.
> > >> >> >>
> > >> >> >> Feel free to provide your feedback and suggestions.
> > >> >> >>
> > >> >> >> Thanks,
> > >> >> >> Abhishek
> > >> >> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> > bafna.iitr@gmail.com
> > >> >
> > >> >> >> wrote:
> > >> >> >> >
> > >> >> >> > Hi,
> > >> >> >> >
> > >> >> >> > I looked into the available patches and prepared the list. I
> > have
> > >> >> put,
> > >> >> >> scope content which we have discussed till now, into a word
> > >> document.
> > >> >> (For
> > >> >> >> the better formatting and easy to read and comment).
> > >> >> >> >
> > >> >> >> > I have noted the additional Jira into the list. Added two more
> > >> Jira
> > >> >> >> related to spark (Scala version and Spark 2.0 property).
> > >> >> >> >
> > >> >> >> > Please take a look at the document (link below) and provide
> your
> > >> >> >> comments.
> > >> >> >> >
> > >> >> >> > If this looks good, we can start the release process.
> > >> >> >> >
> > >> >> >> > Scope Document:
> > >> >> >>
> > >> >>
> > >>
> >
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
> > >> >> >> >
> > >> >> >> > Thanks,
> > >> >> >> > Abhishek
> > >> >> >> >
> > >> >> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> > rkanter@cloudera.com
> > >> >
> > >> >> >> wrote:
> > >> >> >> >>
> > >> >> >> >> That list looks good to me.  The only one I'm not sure of is
> > >> >> OOZIE-2306
> > >> >> >> >> (Oozie Health Check).  It's a new feature and it might make
> > >> sense to
> > >> >> >> let it
> > >> >> >> >> "bake" a while first?  I'm also not sure it's quite ready
> yet.
> > >> >> >> >>
> > >> >> >> >> I'd also like to add a few others to the list:
> > >> >> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> > >> >> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> > >> >> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> > >> >> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> > >> >> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and
> other
> > >> >> fixes)
> > >> >> >> >>
> > >> >> >> >> There's a number of other Patch Available JIRAs not on your
> > list
> > >> >> >> (including
> > >> >> >> >> the ones I mentioned above).  While we don't need to include
> > >> >> everything
> > >> >> >> >> here (and we probably shouldn't nor do we have the review
> > >> >> bandwidth),
> > >> >> >> did
> > >> >> >> >> you consider each of them when creating the list?  I
> currently
> > >> see
> > >> >> 97
> > >> >> >> >> JIRAs: https://s.apache.org/oozie-patch-available
> > >> >> >> >> I think we should include all Spark Action
> fixes/improvements,
> > >> given
> > >> >> >> how
> > >> >> >> >> popular it seems to be.
> > >> >> >> >>
> > >> >> >> >> thanks
> > >> >> >> >> - Robert
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> > >> >> bafna.iitr@gmail.com>
> > >> >> >> >> wrote:
> > >> >> >> >>
> > >> >> >> >>> Hi All,
> > >> >> >> >>>
> > >> >> >> >>> Below is the list of issues, picked from currently available
> > >> >> patches.
> > >> >> >> I
> > >> >> >> >>> could not decide up any 'open' issue, If there is anything,
> > >> please
> > >> >> >> reply
> > >> >> >> >>> here.
> > >> >> >> >>>
> > >> >> >> >>> We can 'drop/add' more if required based on the feedback.
> > >> Important
> > >> >> >> is,
> > >> >> >> >>> how many of these we will be able to review and commit.
> > >> >> >> >>>
> > >> >> >> >>> I will starting looking into these issues and provide
> > feedback.
> > >> >> >> >>>
> > >> >> >>
> > >> >>
> > >>
> >
> ---------------------------------------------------------------------------
> > >> >> >> >>> OOZIE-1978      Forkjoin validation code is ridiculously
> slow
> > in
> > >> >> some
> > >> >> >> cases
> > >> >> >> >>> OOZIE-2508      Documentation change for Coord action rerun
> > >> >> >> [OOZIE-1735]
> > >> >> >> >>> OOZIE-2583      oozie throws EL Exception when reference
> > >> variable
> > >> >> name
> > >> >> >> >>> containing dot
> > >> >> >> >>> OOZIE-2582      Populating external child Ids for action
> > >> failures
> > >> >> >> >>> OOZIE-2517      Add support for startCreatedTime and
> > >> endCreatedTime
> > >> >> >> >>> filters for coord and bundles
> > >> >> >> >>> OOZIE-2512      ShareLibservice returns incorrect path for
> jar
> > >> >> >> >>> OOZIE-2572      SLA DURATION miss not shown when job is
> > running
> > >> for
> > >> >> >> longer
> > >> >> >> >>> than expected time
> > >> >> >> >>> OOZIE-2457      Oozie log parsing regex consume more than
> 90%
> > >> cpu
> > >> >> >> >>> OOZIE-2574      Oozie to support replication-enabled mysql
> > urls
> > >> >> >> >>> OOZIE-2506      Add logs into RecoverService for logging
> > >> >> information
> > >> >> >> about
> > >> >> >> >>> queued commnads
> > >> >> >> >>> OOZIE-2573      dataset url which contains spaces can not be
> > >> >> handled
> > >> >> >> >>> rightly
> > >> >> >> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
> > >> >> mapred-site
> > >> >> >> >>> into spark launcher
> > >> >> >> >>> OOZIE-2491      oozie acl cannot specify group,it does`t
> work
> > >> >> >> >>> OOZIE-2539      Incorrect property key is used for 'hive
> log4j
> > >> >> >> >>> configuration file for execution mode'
> > >> >> >> >>> OOZIE-2555      Oozie SSL enable setup does not return port
> > for
> > >> >> admin
> > >> >> >> >>> -servers
> > >> >> >> >>> OOZIE-2515      Duplicate information for "Changing
> > >> >> endtime/pausetime
> > >> >> >> of a
> > >> >> >> >>> Bundle Job" in CommandLineTool wiki
> > >> >> >> >>> OOZIE-2498      Oozie CallerId configuration for downstream
> > >> >> components
> > >> >> >> >>> OOZIE-2516      Update web service documentation for jobs
> API
> > >> >> >> >>> OOZIE-2500      -DtestJarSimple option mentioned in
> minioozie
> > >> doc
> > >> >> >> does not
> > >> >> >> >>> work
> > >> >> >> >>> OOZIE-2513      log.scan.duration should not be used for
> error
> > >> and
> > >> >> >> audit
> > >> >> >> >>> logs
> > >> >> >> >>> OOZIE-2461      Workflow, Coordinator and Bundle job
> querying
> > >> >> should
> > >> >> >> have
> > >> >> >> >>> last modified filter
> > >> >> >> >>> OOZIE-2306      Oozie health check
> > >> >> >> >>> OOZIE-2518      Oozie is creating many files and directories
> > in
> > >> >> /tmp
> > >> >> >> per
> > >> >> >> >>> day
> > >> >> >> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> > >> >> >> >>> OOZIE-2473      Connection pool for SMTP connection
> > >> >> >> >>> OOZIE-2440      Exponential re-try policy for workflow
> action
> > >> >> >> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should
> ensure
> > >> >> >> uploaded
> > >> >> >> >>> jars are world readable
> > >> >> >> >>> OOZIE-2390      Rerun with failed option removing completed
> > >> output
> > >> >> >> data
> > >> >> >> >>> OOZIE-1810      Workflow cannot get into Failed state when
> > kill
> > >> >> >> control
> > >> >> >> >>> node cannot resolve variable in message
> > >> >> >> >>> OOZIE-2244      Oozie should mask passwords in the logs when
> > >> >> logging
> > >> >> >> >>> command arguments
> > >> >> >> >>>
> > >> >> >>
> > >> >>
> > >>
> >
> ---------------------------------------------------------------------------
> > >> >> >> >>> Thanks,
> > >> >> >> >>> Abhishek
> > >> >> >> >>>
> > >> >> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> > >> bafna.iitr@gmail.com
> > >> >> >
> > >> >> >> >>> wrote:
> > >> >> >> >>>>
> > >> >> >> >>>> Hi All,
> > >> >> >> >>>>
> > >> >> >> >>>> I looked into the commits (approx. 160), all looked good to
> > me
> > >> and
> > >> >> >> >>> should be included into the release. If there are things,
> > which
> > >> >> needs
> > >> >> >> to be
> > >> >> >> >>> excluded, provide your comments.
> > >> >> >> >>>>
> > >> >> >> >>>> We still need to decide on the patch which are in progress
> > >> status.
> > >> >> >> >>>>
> > >> >> >> >>>> On the upgrade side:
> > >> >> >> >>>> Java version          1.6 to 1.7
> > >> >> >> >>>> OpenJPA version       2.2.2 to 2.4.1
> > >> >> >> >>>> Curator version       2.5.0 to 2.11.0
> > >> >> >> >>>> Pig version           0.12.1 to 0.16.0
> > >> >> >> >>>> Hive version          0.13.1 to 1.2.0
> > >> >> >> >>>> HttpClient version    4.3 to 4.3.6
> > >> >> >> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> > >> >> >> >>>>
> > >> >> >> >>>> Please provide your feedback on the above upgrades.
> > >> >> >> >>>>
> > >> >> >> >>>> Once we decide on the scope of the release. We will start
> > >> working
> > >> >> on
> > >> >> >> the
> > >> >> >> >>> completing/testing the release.
> > >> >> >> >>>>
> > >> >> >> >>>> Thanks,
> > >> >> >> >>>> Abhishek
> > >> >> >> >>>>
> > >> >> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> > >> >> bafna.iitr@gmail.com
> > >> >> >> >>> <ma...@gmail.com>> wrote:
> > >> >> >> >>>>>
> > >> >> >> >>>>> Okay. We will make the Java version to 1.7.
> > >> >> >> >>>>>
> > >> >> >> >>>>> There are upgrades we can do like OpenJPA, Curator,
> > >> HttpClient,
> > >> >> Pig
> > >> >> >> etc.
> > >> >> >> >>>>>
> > >> >> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done
> > >> internally as
> > >> >> >> well.
> > >> >> >> >>>>>
> > >> >> >> >>>>> As per the comments on the OOZIE-2231 <
> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
> > >> version
> > >> >> >> also
> > >> >> >> >>> can be upgraded to 2.11.
> > >> >> >> >>>>>
> > >> >> >> >>>>> Thanks,
> > >> >> >> >>>>> Abhishek
> > >> >> >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> > >> >> rkanter@cloudera.com
> > >> >> >> >>> <ma...@cloudera.com>> wrote:
> > >> >> >> >>>>>>
> > >> >> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0
> for
> > >> over
> > >> >> a
> > >> >> >> year
> > >> >> >> >>> now
> > >> >> >> >>>>>> with no problems.  We did have to comment out a few lines
> > in
> > >> >> >> >>> MiniHCatServer
> > >> >> >> >>>>>> and MiniHS2 that were no longer needed:
> > >> >> >> >>>>>>
> > >> >> >> >>>>>> diff --git
> > >> >> >> >>>
> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >> >> >> >>>>>>
> > >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >> >> >> >>>>>> index 8699ff8..d694977 100644
> > >> >> >> >>>>>> ---
> > >> >> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >> >> >> >>>>>> +++
> > >> >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> > >> >> >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> > >> >> >> >>>>>>   public static void resetDefaultDBCreation() throws
> > >> Exception {
> > >> >> >> >>>>>>       // Need to do this, else default db will not be
> > created
> > >> >> for
> > >> >> >> >>> local
> > >> >> >> >>>>>> metastores.
> > >> >> >> >>>>>>       // TestHiveMain will fail with
> > >> >> >> >>>>>> InvalidObjectException(message:There is no database named
> > >> >> default)
> > >> >> >> >>>>>> -        Field declaredField =
> > >> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> > >> >> >> >>>>>> -        declaredField.setAccessible(true);
> > >> >> >> >>>>>> -        declaredField.set(null, false);
> > >> >> >> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14
> > and
> > >> >> later
> > >> >> >> >>>>>> +        //Field declaredField =
> > >> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> > >> >> >> >>>>>> +        //declaredField.setAccessible(true);
> > >> >> >> >>>>>> +        //declaredField.set(null, false);
> > >> >> >> >>>>>>   }
> > >> >> >> >>>>>>
> > >> >> >> >>>>>>   public static void resetHiveConfStaticVariables()
> throws
> > >> >> >> Exception
> > >> >> >> >>> {
> > >> >> >> >>>>>> diff --git
> > >> >> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >> >> >> >>>>>>
> > b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >> >> >> >>>>>> index c9dfd56..505dc55 100644
> > >> >> >> >>>>>> ---
> > >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >> >> >> >>>>>> +++
> > >> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> > >> >> >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> > >> >> AbstractHiveService
> > >> >> >> {
> > >> >> >> >>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> > >> >> >> getHost());
> > >> >> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> > >> >> >> >>> getBinaryPort());
> > >> >> >> >>>>>>
> >  hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> > >> >> >> >>>>>> getHttpPort());
> > >> >> >> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> > >> >> >> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and
> > >> later
> > >> >> >> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> > >> >> >> >>>>>>
> > >> >> >> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
> > >> >> >> >>>>>>   fs.mkdirs(scratchDir);
> > >> >> >> >>>>>>
> > >> >> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> > >> >> >> >>>>>> vranganathan@hortonworks.com <mailto:
> > >> >> vranganathan@hortonworks.com
> > >> >> >> >>
> > >> >> >> >>> wrote:
> > >> >> >> >>>>>>
> > >> >> >> >>>>>>> Let us explore changing the default hive version also
> to
> > >> 1.x.
> > >> >> >>  0.13
> > >> >> >> >>> is
> > >> >> >> >>>>>>> old
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>> Thanks
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>> Venkat
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> > rkanter@cloudera.com
> > >> >> >> <mailto:
> > >> >> >> >>> rkanter@cloudera.com>> wrote:
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>  I don't think we have any guidelines defined anywhere
> > about
> > >> >> when
> > >> >> >> >>> it's
> > >> >> >> >>>>>>> okay
> > >> >> >> >>>>>>>  to change the minimum Java version, but perhaps we need
> > to
> > >> >> wait
> > >> >> >> >>> until
> > >> >> >> >>>>>>> the
> > >> >> >> >>>>>>>  next major release (5.0)?  That said, Hadoop did this
> in
> > a
> > >> >> minor
> > >> >> >> >>>>>>> release
> > >> >> >> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it
> in
> > >> 4.3
> > >> >> if
> > >> >> >> >>>>>>> nobody has
> > >> >> >> >>>>>>>  an objections; we should just make sure to call that
> out
> > in
> > >> >> our
> > >> >> >> >>> release
> > >> >> >> >>>>>>>  notes.
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>  Otherwise, nothing comes to mind for anything we should
> > >> leave
> > >> >> out
> > >> >> >> >>> other
> > >> >> >> >>>>>>>  than the Oozie on Yarn stuff, which is not currently in
> > >> trunk
> > >> >> >> >>> anyway.
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>  I know there's a number of Spark Action JIRAs; we
> should
> > >> make
> > >> >> >> sure
> > >> >> >> >>> to
> > >> >> >> >>>>>>>  include all of these and any pending ones.
> > >> >> >> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more
> > >> time to
> > >> >> >> bake
> > >> >> >> >>>>>>> that we
> > >> >> >> >>>>>>>  should hold off on including in the release?
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>  thanks
> > >> >> >> >>>>>>>  - Robert
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> > >> >> >> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
> > >> >> >> >>>>>>>  wrote:
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>> Hi All,
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>> Any suggestions or comments on the scope for the
> release?
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>> Thanks,
> > >> >> >> >>>>>>>> Abhishek
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> > >> >> >> bafna.iitr@gmail.com
> > >> >> >> >>> <ma...@gmail.com>>
> > >> >> >> >>>>>>>> wrote:
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>> Hi All,
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0.
> > >> This
> > >> >> >> thread
> > >> >> >> >>>>>>> is
> > >> >> >> >>>>>>>> for discussion about the scope of the release. Please
> > >> provide
> > >> >> >> your
> > >> >> >> >>>>>>>> suggestion about what should be included (or any
> > >> exclusions)
> > >> >> in
> > >> >> >> the
> > >> >> >> >>>>>>> release.
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>> Currently, we have around 158 patches submitted to
> > trunk.
> > >> >> Based
> > >> >> >> on
> > >> >> >> >>>>>>> the
> > >> >> >> >>>>>>>> scope, we will include (exclude) more if required.
> > >> >> >> >>>>>>>>> Version Upgrade
> > >> >> >> >>>>>>>>> Java 1.7 (OOZIE-2036 <
> > >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> > >> >> >> >>>>>>>>> OpenJPA (OOZIE-2488 <
> > >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> > >> >> >> >>>>>>>>> Curator  (OOZIE-2231 <
> > >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
> > >> >> >> >>>>>>>>> ),
> > >> >> >> >>>>>>>>> HttpClient  (OOZIE-2538 <
> > >> >> >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> > >> >> >> >>>>>>>>> Pig (OOZIE-2602 <
> > >> >> >> https://issues.apache.org/jira/browse/OOZIE-2602 <
> > >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
> > >> >> >> >>>>>>>> ).
> > >> >> >> >>>>>>>>> New Features/Improvements: Add default JT and NN,
> > Database
> > >> >> >> schema
> > >> >> >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade
> etc.
> > >> >> >> >>>>>>>>> Bug Fixes.
> > >> >> >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the
> above
> > >> >> points
> > >> >> >> or
> > >> >> >> >>>>>>> more
> > >> >> >> >>>>>>>> for the scope of the release. We can also, take some
> work
> > >> >> which
> > >> >> >> is
> > >> >> >> >>>>>>> already
> > >> >> >> >>>>>>>> done and review is pending for them (depending on the
> > >> >> >> availability
> > >> >> >> >>>>>>> of the
> > >> >> >> >>>>>>>> reviewers).
> > >> >> >> >>>>>>>>>
> > >> >> >> >>>>>>>>> Thanks,
> > >> >> >> >>>>>>>>> Abhishek
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>>
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>>>
> > >> >> >> >>>>>
> > >> >> >> >>>>
> > >> >> >> >>>
> > >> >> >> >>>
> > >> >> >> >
> > >> >> >>
> > >> >> >>
> > >> >> >
> > >> >>
> > >> >
> > >> >
> > >>
> > >
> > >
> >
>

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

Ya, let's unset those remaining open trunk ones.  I agree, we need to put
in some effort to clean out old JIRAs and to review Patch Available JIRAs.

Also, I just created OOZIE-2625 to drop workflowgenerator.  We can discuss
the issue more there, but I listed a few reasons on OOZIE-2625 about why.

- Robert

On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy <ro...@gmail.com>
wrote:

>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%20status%20DESC&startIndex=50
>
> These are jiras open and marked trunk. Should we unset fix version on these
> or mark them for next release? Don't see majority of them being worked on
> even for next release. We need to do bug scrub and close many of these some
> time.
>
> > Instead
> of deleting "trunk", I think we're better off just renaming it to the
> release number.
>  There is some amount of relabeling either way. If we go with unsetting fix
> version, then we can as well delete trunk. Also, I did create a 4.4.0
> yesterday to move some of the patch available jiras from 4.3.0.
>
>
> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <
> rohini.aditya@gmail.com>
> wrote:
>
> > I think it would be better to send out email for the bulk change. If
> there
> > are any issues or jiras wrongly marked someone can respond. We do that
> with
> > Pig and there are always couple of jiras that folks come back with
> feedback
> > on pulling back to release.
> >
> >
> > On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <rk...@cloudera.com>
> > wrote:
> >
> >> Abhishek, next time you do a bulk change, please remember to tell it not
> >> to
> >> send out an email for every change :)
> >>
> >> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rk...@cloudera.com>
> >> wrote:
> >>
> >> > Rohini, that sounds good to me.  Especially because we're on "master"
> >> > branch but "trunk" version in JIRA and it gets a little confusing.
> >> Instead
> >> > of deleting "trunk", I think we're better off just renaming it to the
> >> > release number.  Otherwise, we'll have to re-label any current "trunk"
> >> > fields.  We can use 4.4.0 for now, but I imagine that the next release
> >> > might end up being 5.0.0 (with Oozie on Yarn); we can always rename it
> >> > later.
> >> >
> >> > I'm not sure if there's a specific rule here, but I suppose we should
> >> > probably have a vote in the private list to make things official as
> >> this is
> >> > a release-management-related change.  Can you send an email?
> >> >
> >> > Abhishek, thanks for putting this all together.  I think Rohini's
> >> > suggestion of updating the JIRAs is the right next step.  You're
> >> probably
> >> > best off using the bulk change feature instead of doing each JIRA
> >> > individually.  I should have time to do a bunch of reviews this week,
> >> I've
> >> > actually already started.  I should have some time to continue doing
> >> more
> >> > reviews.  As a general thing, we've been pretty slow about getting
> these
> >> > done :(
> >> >
> >> > - Robert
> >> >
> >> > On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> >> > rohini.aditya@gmail.com> wrote:
> >> >
> >> >> I also did mark some of the enhancement jiras that Satish was working
> >> on
> >> >> and which might take time to review to 4.4.0
> >> >>
> >> >> Regards,
> >> >> Rohini
> >> >>
> >> >> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> >> >> rohini.aditya@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Abhishek,
> >> >> >     Can you mark all jiras already committed to trunk as 4.3.0 and
> >> those
> >> >> > planned in the scope document as 4.3.0? This will help filter on
> >> those
> >> >> > and review instead of referring to a google doc. I can pick up
> 10-15
> >> >> > jiras for review end of this week. We should try to get majority of
> >> >> patches
> >> >> > in the next 10 days and branch by Aug 12th. You can then use the
> 3rd
> >> >> week
> >> >> > for any pending critical issues and rolling the release candidate.
> >> >> >
> >> >> > Robert,
> >> >> >   I am thinking, that we should get rid of the trunk version and
> use
> >> >> > release numbers when committing from the beginning like all other
> >> >> projects
> >> >> > do. Unnecessary step to change from trunk to release version for
> >> every
> >> >> > release. If folks are in agreement, then will delete the trunk
> >> version
> >> >> > altogether. Does that require a mail to private?
> >> >> >
> >> >> > Regards,
> >> >> > Rohini
> >> >> >
> >> >> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
> >> bafna.iitr@gmail.com>
> >> >> > wrote:
> >> >> >
> >> >> >> Hi All,
> >> >> >>
> >> >> >> As we prepared a Oozie 4.3.0 scope document <
> >> >> >>
> >> >>
> >>
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >> >> >
> >> >> >> and listed all the things which we wanted to include in the
> release.
> >> >> In the
> >> >> >> last few days, I spent some time and reviewed some of the patches
> >> and
> >> >> >> provided my feedback. I will continue my efforts to review more
> >> >> patches.
> >> >> >> These patches needs to be reviewed by committers and get
> committed.
> >> I
> >> >> >> request other contributors also, review some of the patches and
> >> provide
> >> >> >> their feedback.
> >> >> >>
> >> >> >> I am thinking, may be, we can decide upon some fixed (approximate)
> >> >> >> release date and start working accordingly. Because, If we wait
> for
> >> >> all the
> >> >> >> listed items to be reviewed and committed, then it might take a
> lot
> >> >> more
> >> >> >> time. I am thinking we should target third week of August to
> >> complete
> >> >> and
> >> >> >> release.
> >> >> >>
> >> >> >> Feel free to provide your feedback and suggestions.
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Abhishek
> >> >> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <
> bafna.iitr@gmail.com
> >> >
> >> >> >> wrote:
> >> >> >> >
> >> >> >> > Hi,
> >> >> >> >
> >> >> >> > I looked into the available patches and prepared the list. I
> have
> >> >> put,
> >> >> >> scope content which we have discussed till now, into a word
> >> document.
> >> >> (For
> >> >> >> the better formatting and easy to read and comment).
> >> >> >> >
> >> >> >> > I have noted the additional Jira into the list. Added two more
> >> Jira
> >> >> >> related to spark (Scala version and Spark 2.0 property).
> >> >> >> >
> >> >> >> > Please take a look at the document (link below) and provide your
> >> >> >> comments.
> >> >> >> >
> >> >> >> > If this looks good, we can start the release process.
> >> >> >> >
> >> >> >> > Scope Document:
> >> >> >>
> >> >>
> >>
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> > Abhishek
> >> >> >> >
> >> >> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <
> rkanter@cloudera.com
> >> >
> >> >> >> wrote:
> >> >> >> >>
> >> >> >> >> That list looks good to me.  The only one I'm not sure of is
> >> >> OOZIE-2306
> >> >> >> >> (Oozie Health Check).  It's a new feature and it might make
> >> sense to
> >> >> >> let it
> >> >> >> >> "bake" a while first?  I'm also not sure it's quite ready yet.
> >> >> >> >>
> >> >> >> >> I'd also like to add a few others to the list:
> >> >> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> >> >> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> >> >> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >> >> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> >> >> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other
> >> >> fixes)
> >> >> >> >>
> >> >> >> >> There's a number of other Patch Available JIRAs not on your
> list
> >> >> >> (including
> >> >> >> >> the ones I mentioned above).  While we don't need to include
> >> >> everything
> >> >> >> >> here (and we probably shouldn't nor do we have the review
> >> >> bandwidth),
> >> >> >> did
> >> >> >> >> you consider each of them when creating the list?  I currently
> >> see
> >> >> 97
> >> >> >> >> JIRAs: https://s.apache.org/oozie-patch-available
> >> >> >> >> I think we should include all Spark Action fixes/improvements,
> >> given
> >> >> >> how
> >> >> >> >> popular it seems to be.
> >> >> >> >>
> >> >> >> >> thanks
> >> >> >> >> - Robert
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> >> >> bafna.iitr@gmail.com>
> >> >> >> >> wrote:
> >> >> >> >>
> >> >> >> >>> Hi All,
> >> >> >> >>>
> >> >> >> >>> Below is the list of issues, picked from currently available
> >> >> patches.
> >> >> >> I
> >> >> >> >>> could not decide up any 'open' issue, If there is anything,
> >> please
> >> >> >> reply
> >> >> >> >>> here.
> >> >> >> >>>
> >> >> >> >>> We can 'drop/add' more if required based on the feedback.
> >> Important
> >> >> >> is,
> >> >> >> >>> how many of these we will be able to review and commit.
> >> >> >> >>>
> >> >> >> >>> I will starting looking into these issues and provide
> feedback.
> >> >> >> >>>
> >> >> >>
> >> >>
> >>
> ---------------------------------------------------------------------------
> >> >> >> >>> OOZIE-1978      Forkjoin validation code is ridiculously slow
> in
> >> >> some
> >> >> >> cases
> >> >> >> >>> OOZIE-2508      Documentation change for Coord action rerun
> >> >> >> [OOZIE-1735]
> >> >> >> >>> OOZIE-2583      oozie throws EL Exception when reference
> >> variable
> >> >> name
> >> >> >> >>> containing dot
> >> >> >> >>> OOZIE-2582      Populating external child Ids for action
> >> failures
> >> >> >> >>> OOZIE-2517      Add support for startCreatedTime and
> >> endCreatedTime
> >> >> >> >>> filters for coord and bundles
> >> >> >> >>> OOZIE-2512      ShareLibservice returns incorrect path for jar
> >> >> >> >>> OOZIE-2572      SLA DURATION miss not shown when job is
> running
> >> for
> >> >> >> longer
> >> >> >> >>> than expected time
> >> >> >> >>> OOZIE-2457      Oozie log parsing regex consume more than 90%
> >> cpu
> >> >> >> >>> OOZIE-2574      Oozie to support replication-enabled mysql
> urls
> >> >> >> >>> OOZIE-2506      Add logs into RecoverService for logging
> >> >> information
> >> >> >> about
> >> >> >> >>> queued commnads
> >> >> >> >>> OOZIE-2573      dataset url which contains spaces can not be
> >> >> handled
> >> >> >> >>> rightly
> >> >> >> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
> >> >> mapred-site
> >> >> >> >>> into spark launcher
> >> >> >> >>> OOZIE-2491      oozie acl cannot specify group,it does`t work
> >> >> >> >>> OOZIE-2539      Incorrect property key is used for 'hive log4j
> >> >> >> >>> configuration file for execution mode'
> >> >> >> >>> OOZIE-2555      Oozie SSL enable setup does not return port
> for
> >> >> admin
> >> >> >> >>> -servers
> >> >> >> >>> OOZIE-2515      Duplicate information for "Changing
> >> >> endtime/pausetime
> >> >> >> of a
> >> >> >> >>> Bundle Job" in CommandLineTool wiki
> >> >> >> >>> OOZIE-2498      Oozie CallerId configuration for downstream
> >> >> components
> >> >> >> >>> OOZIE-2516      Update web service documentation for jobs API
> >> >> >> >>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie
> >> doc
> >> >> >> does not
> >> >> >> >>> work
> >> >> >> >>> OOZIE-2513      log.scan.duration should not be used for error
> >> and
> >> >> >> audit
> >> >> >> >>> logs
> >> >> >> >>> OOZIE-2461      Workflow, Coordinator and Bundle job querying
> >> >> should
> >> >> >> have
> >> >> >> >>> last modified filter
> >> >> >> >>> OOZIE-2306      Oozie health check
> >> >> >> >>> OOZIE-2518      Oozie is creating many files and directories
> in
> >> >> /tmp
> >> >> >> per
> >> >> >> >>> day
> >> >> >> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> >> >> >> >>> OOZIE-2473      Connection pool for SMTP connection
> >> >> >> >>> OOZIE-2440      Exponential re-try policy for workflow action
> >> >> >> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure
> >> >> >> uploaded
> >> >> >> >>> jars are world readable
> >> >> >> >>> OOZIE-2390      Rerun with failed option removing completed
> >> output
> >> >> >> data
> >> >> >> >>> OOZIE-1810      Workflow cannot get into Failed state when
> kill
> >> >> >> control
> >> >> >> >>> node cannot resolve variable in message
> >> >> >> >>> OOZIE-2244      Oozie should mask passwords in the logs when
> >> >> logging
> >> >> >> >>> command arguments
> >> >> >> >>>
> >> >> >>
> >> >>
> >>
> ---------------------------------------------------------------------------
> >> >> >> >>> Thanks,
> >> >> >> >>> Abhishek
> >> >> >> >>>
> >> >> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> >> bafna.iitr@gmail.com
> >> >> >
> >> >> >> >>> wrote:
> >> >> >> >>>>
> >> >> >> >>>> Hi All,
> >> >> >> >>>>
> >> >> >> >>>> I looked into the commits (approx. 160), all looked good to
> me
> >> and
> >> >> >> >>> should be included into the release. If there are things,
> which
> >> >> needs
> >> >> >> to be
> >> >> >> >>> excluded, provide your comments.
> >> >> >> >>>>
> >> >> >> >>>> We still need to decide on the patch which are in progress
> >> status.
> >> >> >> >>>>
> >> >> >> >>>> On the upgrade side:
> >> >> >> >>>> Java version          1.6 to 1.7
> >> >> >> >>>> OpenJPA version       2.2.2 to 2.4.1
> >> >> >> >>>> Curator version       2.5.0 to 2.11.0
> >> >> >> >>>> Pig version           0.12.1 to 0.16.0
> >> >> >> >>>> Hive version          0.13.1 to 1.2.0
> >> >> >> >>>> HttpClient version    4.3 to 4.3.6
> >> >> >> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> >> >> >> >>>>
> >> >> >> >>>> Please provide your feedback on the above upgrades.
> >> >> >> >>>>
> >> >> >> >>>> Once we decide on the scope of the release. We will start
> >> working
> >> >> on
> >> >> >> the
> >> >> >> >>> completing/testing the release.
> >> >> >> >>>>
> >> >> >> >>>> Thanks,
> >> >> >> >>>> Abhishek
> >> >> >> >>>>
> >> >> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> >> >> bafna.iitr@gmail.com
> >> >> >> >>> <ma...@gmail.com>> wrote:
> >> >> >> >>>>>
> >> >> >> >>>>> Okay. We will make the Java version to 1.7.
> >> >> >> >>>>>
> >> >> >> >>>>> There are upgrades we can do like OpenJPA, Curator,
> >> HttpClient,
> >> >> Pig
> >> >> >> etc.
> >> >> >> >>>>>
> >> >> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done
> >> internally as
> >> >> >> well.
> >> >> >> >>>>>
> >> >> >> >>>>> As per the comments on the OOZIE-2231 <
> >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
> >> version
> >> >> >> also
> >> >> >> >>> can be upgraded to 2.11.
> >> >> >> >>>>>
> >> >> >> >>>>> Thanks,
> >> >> >> >>>>> Abhishek
> >> >> >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> >> >> rkanter@cloudera.com
> >> >> >> >>> <ma...@cloudera.com>> wrote:
> >> >> >> >>>>>>
> >> >> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for
> >> over
> >> >> a
> >> >> >> year
> >> >> >> >>> now
> >> >> >> >>>>>> with no problems.  We did have to comment out a few lines
> in
> >> >> >> >>> MiniHCatServer
> >> >> >> >>>>>> and MiniHS2 that were no longer needed:
> >> >> >> >>>>>>
> >> >> >> >>>>>> diff --git
> >> >> >> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >> >> >>>>>>
> >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >> >> >>>>>> index 8699ff8..d694977 100644
> >> >> >> >>>>>> ---
> >> >> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >> >> >>>>>> +++
> >> >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >> >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> >> >> >> >>>>>>   public static void resetDefaultDBCreation() throws
> >> Exception {
> >> >> >> >>>>>>       // Need to do this, else default db will not be
> created
> >> >> for
> >> >> >> >>> local
> >> >> >> >>>>>> metastores.
> >> >> >> >>>>>>       // TestHiveMain will fail with
> >> >> >> >>>>>> InvalidObjectException(message:There is no database named
> >> >> default)
> >> >> >> >>>>>> -        Field declaredField =
> >> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >> >> >> >>>>>> -        declaredField.setAccessible(true);
> >> >> >> >>>>>> -        declaredField.set(null, false);
> >> >> >> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14
> and
> >> >> later
> >> >> >> >>>>>> +        //Field declaredField =
> >> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >> >> >> >>>>>> +        //declaredField.setAccessible(true);
> >> >> >> >>>>>> +        //declaredField.set(null, false);
> >> >> >> >>>>>>   }
> >> >> >> >>>>>>
> >> >> >> >>>>>>   public static void resetHiveConfStaticVariables() throws
> >> >> >> Exception
> >> >> >> >>> {
> >> >> >> >>>>>> diff --git
> >> >> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >> >> >>>>>>
> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >> >> >>>>>> index c9dfd56..505dc55 100644
> >> >> >> >>>>>> ---
> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >> >> >>>>>> +++
> >> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >> >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> >> >> AbstractHiveService
> >> >> >> {
> >> >> >> >>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> >> >> >> getHost());
> >> >> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> >> >> >> >>> getBinaryPort());
> >> >> >> >>>>>>
>  hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >> >> >> >>>>>> getHttpPort());
> >> >> >> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >> >> >> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and
> >> later
> >> >> >> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >> >> >> >>>>>>
> >> >> >> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
> >> >> >> >>>>>>   fs.mkdirs(scratchDir);
> >> >> >> >>>>>>
> >> >> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> >> >> >> >>>>>> vranganathan@hortonworks.com <mailto:
> >> >> vranganathan@hortonworks.com
> >> >> >> >>
> >> >> >> >>> wrote:
> >> >> >> >>>>>>
> >> >> >> >>>>>>> Let us explore changing the default hive version also  to
> >> 1.x.
> >> >> >>  0.13
> >> >> >> >>> is
> >> >> >> >>>>>>> old
> >> >> >> >>>>>>>
> >> >> >> >>>>>>> Thanks
> >> >> >> >>>>>>>
> >> >> >> >>>>>>> Venkat
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>
> >> >> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <
> rkanter@cloudera.com
> >> >> >> <mailto:
> >> >> >> >>> rkanter@cloudera.com>> wrote:
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>  I don't think we have any guidelines defined anywhere
> about
> >> >> when
> >> >> >> >>> it's
> >> >> >> >>>>>>> okay
> >> >> >> >>>>>>>  to change the minimum Java version, but perhaps we need
> to
> >> >> wait
> >> >> >> >>> until
> >> >> >> >>>>>>> the
> >> >> >> >>>>>>>  next major release (5.0)?  That said, Hadoop did this in
> a
> >> >> minor
> >> >> >> >>>>>>> release
> >> >> >> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it in
> >> 4.3
> >> >> if
> >> >> >> >>>>>>> nobody has
> >> >> >> >>>>>>>  an objections; we should just make sure to call that out
> in
> >> >> our
> >> >> >> >>> release
> >> >> >> >>>>>>>  notes.
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>  Otherwise, nothing comes to mind for anything we should
> >> leave
> >> >> out
> >> >> >> >>> other
> >> >> >> >>>>>>>  than the Oozie on Yarn stuff, which is not currently in
> >> trunk
> >> >> >> >>> anyway.
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>  I know there's a number of Spark Action JIRAs; we should
> >> make
> >> >> >> sure
> >> >> >> >>> to
> >> >> >> >>>>>>>  include all of these and any pending ones.
> >> >> >> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more
> >> time to
> >> >> >> bake
> >> >> >> >>>>>>> that we
> >> >> >> >>>>>>>  should hold off on including in the release?
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>  thanks
> >> >> >> >>>>>>>  - Robert
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> >> >> >> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
> >> >> >> >>>>>>>  wrote:
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>> Hi All,
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>> Any suggestions or comments on the scope for the release?
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>> Thanks,
> >> >> >> >>>>>>>> Abhishek
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> >> >> >> bafna.iitr@gmail.com
> >> >> >> >>> <ma...@gmail.com>>
> >> >> >> >>>>>>>> wrote:
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>> Hi All,
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0.
> >> This
> >> >> >> thread
> >> >> >> >>>>>>> is
> >> >> >> >>>>>>>> for discussion about the scope of the release. Please
> >> provide
> >> >> >> your
> >> >> >> >>>>>>>> suggestion about what should be included (or any
> >> exclusions)
> >> >> in
> >> >> >> the
> >> >> >> >>>>>>> release.
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>> Currently, we have around 158 patches submitted to
> trunk.
> >> >> Based
> >> >> >> on
> >> >> >> >>>>>>> the
> >> >> >> >>>>>>>> scope, we will include (exclude) more if required.
> >> >> >> >>>>>>>>> Version Upgrade
> >> >> >> >>>>>>>>> Java 1.7 (OOZIE-2036 <
> >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> >> >> >> >>>>>>>>> OpenJPA (OOZIE-2488 <
> >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> >> >> >> >>>>>>>>> Curator  (OOZIE-2231 <
> >> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
> >> >> >> >>>>>>>>> ),
> >> >> >> >>>>>>>>> HttpClient  (OOZIE-2538 <
> >> >> >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> >> >> >> >>>>>>>>> Pig (OOZIE-2602 <
> >> >> >> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
> >> >> >> >>>>>>>> ).
> >> >> >> >>>>>>>>> New Features/Improvements: Add default JT and NN,
> Database
> >> >> >> schema
> >> >> >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
> >> >> >> >>>>>>>>> Bug Fixes.
> >> >> >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above
> >> >> points
> >> >> >> or
> >> >> >> >>>>>>> more
> >> >> >> >>>>>>>> for the scope of the release. We can also, take some work
> >> >> which
> >> >> >> is
> >> >> >> >>>>>>> already
> >> >> >> >>>>>>>> done and review is pending for them (depending on the
> >> >> >> availability
> >> >> >> >>>>>>> of the
> >> >> >> >>>>>>>> reviewers).
> >> >> >> >>>>>>>>>
> >> >> >> >>>>>>>>> Thanks,
> >> >> >> >>>>>>>>> Abhishek
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>>
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>
> >> >> >> >>>>>>>
> >> >> >> >>>>>
> >> >> >> >>>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >
> >> >>
> >> >
> >> >
> >>
> >
> >
>

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy <ro...@gmail.com>]

https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and%20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C%20status%20DESC&startIndex=50

These are jiras open and marked trunk. Should we unset fix version on these
or mark them for next release? Don't see majority of them being worked on
even for next release. We need to do bug scrub and close many of these some
time.

> Instead
of deleting "trunk", I think we're better off just renaming it to the
release number.
 There is some amount of relabeling either way. If we go with unsetting fix
version, then we can as well delete trunk. Also, I did create a 4.4.0
yesterday to move some of the patch available jiras from 4.3.0.


On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy <ro...@gmail.com>
wrote:

> I think it would be better to send out email for the bulk change. If there
> are any issues or jiras wrongly marked someone can respond. We do that with
> Pig and there are always couple of jiras that folks come back with feedback
> on pulling back to release.
>
>
> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
>> Abhishek, next time you do a bulk change, please remember to tell it not
>> to
>> send out an email for every change :)
>>
>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>>
>> > Rohini, that sounds good to me.  Especially because we're on "master"
>> > branch but "trunk" version in JIRA and it gets a little confusing.
>> Instead
>> > of deleting "trunk", I think we're better off just renaming it to the
>> > release number.  Otherwise, we'll have to re-label any current "trunk"
>> > fields.  We can use 4.4.0 for now, but I imagine that the next release
>> > might end up being 5.0.0 (with Oozie on Yarn); we can always rename it
>> > later.
>> >
>> > I'm not sure if there's a specific rule here, but I suppose we should
>> > probably have a vote in the private list to make things official as
>> this is
>> > a release-management-related change.  Can you send an email?
>> >
>> > Abhishek, thanks for putting this all together.  I think Rohini's
>> > suggestion of updating the JIRAs is the right next step.  You're
>> probably
>> > best off using the bulk change feature instead of doing each JIRA
>> > individually.  I should have time to do a bunch of reviews this week,
>> I've
>> > actually already started.  I should have some time to continue doing
>> more
>> > reviews.  As a general thing, we've been pretty slow about getting these
>> > done :(
>> >
>> > - Robert
>> >
>> > On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
>> > rohini.aditya@gmail.com> wrote:
>> >
>> >> I also did mark some of the enhancement jiras that Satish was working
>> on
>> >> and which might take time to review to 4.4.0
>> >>
>> >> Regards,
>> >> Rohini
>> >>
>> >> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>> >> rohini.aditya@gmail.com>
>> >> wrote:
>> >>
>> >> > Abhishek,
>> >> >     Can you mark all jiras already committed to trunk as 4.3.0 and
>> those
>> >> > planned in the scope document as 4.3.0? This will help filter on
>> those
>> >> > and review instead of referring to a google doc. I can pick up 10-15
>> >> > jiras for review end of this week. We should try to get majority of
>> >> patches
>> >> > in the next 10 days and branch by Aug 12th. You can then use the 3rd
>> >> week
>> >> > for any pending critical issues and rolling the release candidate.
>> >> >
>> >> > Robert,
>> >> >   I am thinking, that we should get rid of the trunk version and use
>> >> > release numbers when committing from the beginning like all other
>> >> projects
>> >> > do. Unnecessary step to change from trunk to release version for
>> every
>> >> > release. If folks are in agreement, then will delete the trunk
>> version
>> >> > altogether. Does that require a mail to private?
>> >> >
>> >> > Regards,
>> >> > Rohini
>> >> >
>> >> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com>
>> >> > wrote:
>> >> >
>> >> >> Hi All,
>> >> >>
>> >> >> As we prepared a Oozie 4.3.0 scope document <
>> >> >>
>> >>
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>> >> >
>> >> >> and listed all the things which we wanted to include in the release.
>> >> In the
>> >> >> last few days, I spent some time and reviewed some of the patches
>> and
>> >> >> provided my feedback. I will continue my efforts to review more
>> >> patches.
>> >> >> These patches needs to be reviewed by committers and get committed.
>> I
>> >> >> request other contributors also, review some of the patches and
>> provide
>> >> >> their feedback.
>> >> >>
>> >> >> I am thinking, may be, we can decide upon some fixed (approximate)
>> >> >> release date and start working accordingly. Because, If we wait for
>> >> all the
>> >> >> listed items to be reviewed and committed, then it might take a lot
>> >> more
>> >> >> time. I am thinking we should target third week of August to
>> complete
>> >> and
>> >> >> release.
>> >> >>
>> >> >> Feel free to provide your feedback and suggestions.
>> >> >>
>> >> >> Thanks,
>> >> >> Abhishek
>> >> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <bafna.iitr@gmail.com
>> >
>> >> >> wrote:
>> >> >> >
>> >> >> > Hi,
>> >> >> >
>> >> >> > I looked into the available patches and prepared the list. I have
>> >> put,
>> >> >> scope content which we have discussed till now, into a word
>> document.
>> >> (For
>> >> >> the better formatting and easy to read and comment).
>> >> >> >
>> >> >> > I have noted the additional Jira into the list. Added two more
>> Jira
>> >> >> related to spark (Scala version and Spark 2.0 property).
>> >> >> >
>> >> >> > Please take a look at the document (link below) and provide your
>> >> >> comments.
>> >> >> >
>> >> >> > If this looks good, we can start the release process.
>> >> >> >
>> >> >> > Scope Document:
>> >> >>
>> >>
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Abhishek
>> >> >> >
>> >> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rkanter@cloudera.com
>> >
>> >> >> wrote:
>> >> >> >>
>> >> >> >> That list looks good to me.  The only one I'm not sure of is
>> >> OOZIE-2306
>> >> >> >> (Oozie Health Check).  It's a new feature and it might make
>> sense to
>> >> >> let it
>> >> >> >> "bake" a while first?  I'm also not sure it's quite ready yet.
>> >> >> >>
>> >> >> >> I'd also like to add a few others to the list:
>> >> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>> >> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>> >> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>> >> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>> >> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other
>> >> fixes)
>> >> >> >>
>> >> >> >> There's a number of other Patch Available JIRAs not on your list
>> >> >> (including
>> >> >> >> the ones I mentioned above).  While we don't need to include
>> >> everything
>> >> >> >> here (and we probably shouldn't nor do we have the review
>> >> bandwidth),
>> >> >> did
>> >> >> >> you consider each of them when creating the list?  I currently
>> see
>> >> 97
>> >> >> >> JIRAs: https://s.apache.org/oozie-patch-available
>> >> >> >> I think we should include all Spark Action fixes/improvements,
>> given
>> >> >> how
>> >> >> >> popular it seems to be.
>> >> >> >>
>> >> >> >> thanks
>> >> >> >> - Robert
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>> >> bafna.iitr@gmail.com>
>> >> >> >> wrote:
>> >> >> >>
>> >> >> >>> Hi All,
>> >> >> >>>
>> >> >> >>> Below is the list of issues, picked from currently available
>> >> patches.
>> >> >> I
>> >> >> >>> could not decide up any 'open' issue, If there is anything,
>> please
>> >> >> reply
>> >> >> >>> here.
>> >> >> >>>
>> >> >> >>> We can 'drop/add' more if required based on the feedback.
>> Important
>> >> >> is,
>> >> >> >>> how many of these we will be able to review and commit.
>> >> >> >>>
>> >> >> >>> I will starting looking into these issues and provide feedback.
>> >> >> >>>
>> >> >>
>> >>
>> ---------------------------------------------------------------------------
>> >> >> >>> OOZIE-1978      Forkjoin validation code is ridiculously slow in
>> >> some
>> >> >> cases
>> >> >> >>> OOZIE-2508      Documentation change for Coord action rerun
>> >> >> [OOZIE-1735]
>> >> >> >>> OOZIE-2583      oozie throws EL Exception when reference
>> variable
>> >> name
>> >> >> >>> containing dot
>> >> >> >>> OOZIE-2582      Populating external child Ids for action
>> failures
>> >> >> >>> OOZIE-2517      Add support for startCreatedTime and
>> endCreatedTime
>> >> >> >>> filters for coord and bundles
>> >> >> >>> OOZIE-2512      ShareLibservice returns incorrect path for jar
>> >> >> >>> OOZIE-2572      SLA DURATION miss not shown when job is running
>> for
>> >> >> longer
>> >> >> >>> than expected time
>> >> >> >>> OOZIE-2457      Oozie log parsing regex consume more than 90%
>> cpu
>> >> >> >>> OOZIE-2574      Oozie to support replication-enabled mysql urls
>> >> >> >>> OOZIE-2506      Add logs into RecoverService for logging
>> >> information
>> >> >> about
>> >> >> >>> queued commnads
>> >> >> >>> OOZIE-2573      dataset url which contains spaces can not be
>> >> handled
>> >> >> >>> rightly
>> >> >> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>> >> mapred-site
>> >> >> >>> into spark launcher
>> >> >> >>> OOZIE-2491      oozie acl cannot specify group,it does`t work
>> >> >> >>> OOZIE-2539      Incorrect property key is used for 'hive log4j
>> >> >> >>> configuration file for execution mode'
>> >> >> >>> OOZIE-2555      Oozie SSL enable setup does not return port for
>> >> admin
>> >> >> >>> -servers
>> >> >> >>> OOZIE-2515      Duplicate information for "Changing
>> >> endtime/pausetime
>> >> >> of a
>> >> >> >>> Bundle Job" in CommandLineTool wiki
>> >> >> >>> OOZIE-2498      Oozie CallerId configuration for downstream
>> >> components
>> >> >> >>> OOZIE-2516      Update web service documentation for jobs API
>> >> >> >>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie
>> doc
>> >> >> does not
>> >> >> >>> work
>> >> >> >>> OOZIE-2513      log.scan.duration should not be used for error
>> and
>> >> >> audit
>> >> >> >>> logs
>> >> >> >>> OOZIE-2461      Workflow, Coordinator and Bundle job querying
>> >> should
>> >> >> have
>> >> >> >>> last modified filter
>> >> >> >>> OOZIE-2306      Oozie health check
>> >> >> >>> OOZIE-2518      Oozie is creating many files and directories in
>> >> /tmp
>> >> >> per
>> >> >> >>> day
>> >> >> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>> >> >> >>> OOZIE-2473      Connection pool for SMTP connection
>> >> >> >>> OOZIE-2440      Exponential re-try policy for workflow action
>> >> >> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure
>> >> >> uploaded
>> >> >> >>> jars are world readable
>> >> >> >>> OOZIE-2390      Rerun with failed option removing completed
>> output
>> >> >> data
>> >> >> >>> OOZIE-1810      Workflow cannot get into Failed state when kill
>> >> >> control
>> >> >> >>> node cannot resolve variable in message
>> >> >> >>> OOZIE-2244      Oozie should mask passwords in the logs when
>> >> logging
>> >> >> >>> command arguments
>> >> >> >>>
>> >> >>
>> >>
>> ---------------------------------------------------------------------------
>> >> >> >>> Thanks,
>> >> >> >>> Abhishek
>> >> >> >>>
>> >> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
>> bafna.iitr@gmail.com
>> >> >
>> >> >> >>> wrote:
>> >> >> >>>>
>> >> >> >>>> Hi All,
>> >> >> >>>>
>> >> >> >>>> I looked into the commits (approx. 160), all looked good to me
>> and
>> >> >> >>> should be included into the release. If there are things, which
>> >> needs
>> >> >> to be
>> >> >> >>> excluded, provide your comments.
>> >> >> >>>>
>> >> >> >>>> We still need to decide on the patch which are in progress
>> status.
>> >> >> >>>>
>> >> >> >>>> On the upgrade side:
>> >> >> >>>> Java version          1.6 to 1.7
>> >> >> >>>> OpenJPA version       2.2.2 to 2.4.1
>> >> >> >>>> Curator version       2.5.0 to 2.11.0
>> >> >> >>>> Pig version           0.12.1 to 0.16.0
>> >> >> >>>> Hive version          0.13.1 to 1.2.0
>> >> >> >>>> HttpClient version    4.3 to 4.3.6
>> >> >> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>> >> >> >>>>
>> >> >> >>>> Please provide your feedback on the above upgrades.
>> >> >> >>>>
>> >> >> >>>> Once we decide on the scope of the release. We will start
>> working
>> >> on
>> >> >> the
>> >> >> >>> completing/testing the release.
>> >> >> >>>>
>> >> >> >>>> Thanks,
>> >> >> >>>> Abhishek
>> >> >> >>>>
>> >> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>> >> bafna.iitr@gmail.com
>> >> >> >>> <ma...@gmail.com>> wrote:
>> >> >> >>>>>
>> >> >> >>>>> Okay. We will make the Java version to 1.7.
>> >> >> >>>>>
>> >> >> >>>>> There are upgrades we can do like OpenJPA, Curator,
>> HttpClient,
>> >> Pig
>> >> >> etc.
>> >> >> >>>>>
>> >> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done
>> internally as
>> >> >> well.
>> >> >> >>>>>
>> >> >> >>>>> As per the comments on the OOZIE-2231 <
>> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
>> version
>> >> >> also
>> >> >> >>> can be upgraded to 2.11.
>> >> >> >>>>>
>> >> >> >>>>> Thanks,
>> >> >> >>>>> Abhishek
>> >> >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>> >> rkanter@cloudera.com
>> >> >> >>> <ma...@cloudera.com>> wrote:
>> >> >> >>>>>>
>> >> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for
>> over
>> >> a
>> >> >> year
>> >> >> >>> now
>> >> >> >>>>>> with no problems.  We did have to comment out a few lines in
>> >> >> >>> MiniHCatServer
>> >> >> >>>>>> and MiniHS2 that were no longer needed:
>> >> >> >>>>>>
>> >> >> >>>>>> diff --git
>> >> >> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >> >> >>>>>>
>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >> >> >>>>>> index 8699ff8..d694977 100644
>> >> >> >>>>>> ---
>> >> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >> >> >>>>>> +++
>> >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >> >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>> >> >> >>>>>>   public static void resetDefaultDBCreation() throws
>> Exception {
>> >> >> >>>>>>       // Need to do this, else default db will not be created
>> >> for
>> >> >> >>> local
>> >> >> >>>>>> metastores.
>> >> >> >>>>>>       // TestHiveMain will fail with
>> >> >> >>>>>> InvalidObjectException(message:There is no database named
>> >> default)
>> >> >> >>>>>> -        Field declaredField =
>> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> >> >> >>>>>> -        declaredField.setAccessible(true);
>> >> >> >>>>>> -        declaredField.set(null, false);
>> >> >> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and
>> >> later
>> >> >> >>>>>> +        //Field declaredField =
>> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> >> >> >>>>>> +        //declaredField.setAccessible(true);
>> >> >> >>>>>> +        //declaredField.set(null, false);
>> >> >> >>>>>>   }
>> >> >> >>>>>>
>> >> >> >>>>>>   public static void resetHiveConfStaticVariables() throws
>> >> >> Exception
>> >> >> >>> {
>> >> >> >>>>>> diff --git
>> >> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >> >> >>>>>> index c9dfd56..505dc55 100644
>> >> >> >>>>>> ---
>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >> >> >>>>>> +++
>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >> >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>> >> AbstractHiveService
>> >> >> {
>> >> >> >>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>> >> >> getHost());
>> >> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
>> >> >> >>> getBinaryPort());
>> >> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>> >> >> >>>>>> getHttpPort());
>> >> >> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> >> >> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and
>> later
>> >> >> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> >> >> >>>>>>
>> >> >> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
>> >> >> >>>>>>   fs.mkdirs(scratchDir);
>> >> >> >>>>>>
>> >> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>> >> >> >>>>>> vranganathan@hortonworks.com <mailto:
>> >> vranganathan@hortonworks.com
>> >> >> >>
>> >> >> >>> wrote:
>> >> >> >>>>>>
>> >> >> >>>>>>> Let us explore changing the default hive version also  to
>> 1.x.
>> >> >>  0.13
>> >> >> >>> is
>> >> >> >>>>>>> old
>> >> >> >>>>>>>
>> >> >> >>>>>>> Thanks
>> >> >> >>>>>>>
>> >> >> >>>>>>> Venkat
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com
>> >> >> <mailto:
>> >> >> >>> rkanter@cloudera.com>> wrote:
>> >> >> >>>>>>>
>> >> >> >>>>>>>  I don't think we have any guidelines defined anywhere about
>> >> when
>> >> >> >>> it's
>> >> >> >>>>>>> okay
>> >> >> >>>>>>>  to change the minimum Java version, but perhaps we need to
>> >> wait
>> >> >> >>> until
>> >> >> >>>>>>> the
>> >> >> >>>>>>>  next major release (5.0)?  That said, Hadoop did this in a
>> >> minor
>> >> >> >>>>>>> release
>> >> >> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it in
>> 4.3
>> >> if
>> >> >> >>>>>>> nobody has
>> >> >> >>>>>>>  an objections; we should just make sure to call that out in
>> >> our
>> >> >> >>> release
>> >> >> >>>>>>>  notes.
>> >> >> >>>>>>>
>> >> >> >>>>>>>  Otherwise, nothing comes to mind for anything we should
>> leave
>> >> out
>> >> >> >>> other
>> >> >> >>>>>>>  than the Oozie on Yarn stuff, which is not currently in
>> trunk
>> >> >> >>> anyway.
>> >> >> >>>>>>>
>> >> >> >>>>>>>  I know there's a number of Spark Action JIRAs; we should
>> make
>> >> >> sure
>> >> >> >>> to
>> >> >> >>>>>>>  include all of these and any pending ones.
>> >> >> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more
>> time to
>> >> >> bake
>> >> >> >>>>>>> that we
>> >> >> >>>>>>>  should hold off on including in the release?
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>>  thanks
>> >> >> >>>>>>>  - Robert
>> >> >> >>>>>>>
>> >> >> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>> >> >> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
>> >> >> >>>>>>>  wrote:
>> >> >> >>>>>>>
>> >> >> >>>>>>>> Hi All,
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> Any suggestions or comments on the scope for the release?
>> >> >> >>>>>>>>
>> >> >> >>>>>>>> Thanks,
>> >> >> >>>>>>>> Abhishek
>> >> >> >>>>>>>>
>> >> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>> >> >> bafna.iitr@gmail.com
>> >> >> >>> <ma...@gmail.com>>
>> >> >> >>>>>>>> wrote:
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> Hi All,
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0.
>> This
>> >> >> thread
>> >> >> >>>>>>> is
>> >> >> >>>>>>>> for discussion about the scope of the release. Please
>> provide
>> >> >> your
>> >> >> >>>>>>>> suggestion about what should be included (or any
>> exclusions)
>> >> in
>> >> >> the
>> >> >> >>>>>>> release.
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> Currently, we have around 158 patches submitted to trunk.
>> >> Based
>> >> >> on
>> >> >> >>>>>>> the
>> >> >> >>>>>>>> scope, we will include (exclude) more if required.
>> >> >> >>>>>>>>> Version Upgrade
>> >> >> >>>>>>>>> Java 1.7 (OOZIE-2036 <
>> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>> >> >> >>>>>>>>> OpenJPA (OOZIE-2488 <
>> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>> >> >> >>>>>>>>> Curator  (OOZIE-2231 <
>> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
>> >> >> >>>>>>>>> ),
>> >> >> >>>>>>>>> HttpClient  (OOZIE-2538 <
>> >> >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>> >> >> >>>>>>>>> Pig (OOZIE-2602 <
>> >> >> https://issues.apache.org/jira/browse/OOZIE-2602 <
>> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
>> >> >> >>>>>>>> ).
>> >> >> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database
>> >> >> schema
>> >> >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
>> >> >> >>>>>>>>> Bug Fixes.
>> >> >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above
>> >> points
>> >> >> or
>> >> >> >>>>>>> more
>> >> >> >>>>>>>> for the scope of the release. We can also, take some work
>> >> which
>> >> >> is
>> >> >> >>>>>>> already
>> >> >> >>>>>>>> done and review is pending for them (depending on the
>> >> >> availability
>> >> >> >>>>>>> of the
>> >> >> >>>>>>>> reviewers).
>> >> >> >>>>>>>>>
>> >> >> >>>>>>>>> Thanks,
>> >> >> >>>>>>>>> Abhishek
>> >> >> >>>>>>>>
>> >> >> >>>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>
>> >> >> >>>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >
>> >> >>
>> >> >>
>> >> >
>> >>
>> >
>> >
>>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy <ro...@gmail.com>]

I think it would be better to send out email for the bulk change. If there
are any issues or jiras wrongly marked someone can respond. We do that with
Pig and there are always couple of jiras that folks come back with feedback
on pulling back to release.

On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <rk...@cloudera.com> wrote:

> Abhishek, next time you do a bulk change, please remember to tell it not to
> send out an email for every change :)
>
> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > Rohini, that sounds good to me.  Especially because we're on "master"
> > branch but "trunk" version in JIRA and it gets a little confusing.
> Instead
> > of deleting "trunk", I think we're better off just renaming it to the
> > release number.  Otherwise, we'll have to re-label any current "trunk"
> > fields.  We can use 4.4.0 for now, but I imagine that the next release
> > might end up being 5.0.0 (with Oozie on Yarn); we can always rename it
> > later.
> >
> > I'm not sure if there's a specific rule here, but I suppose we should
> > probably have a vote in the private list to make things official as this
> is
> > a release-management-related change.  Can you send an email?
> >
> > Abhishek, thanks for putting this all together.  I think Rohini's
> > suggestion of updating the JIRAs is the right next step.  You're probably
> > best off using the bulk change feature instead of doing each JIRA
> > individually.  I should have time to do a bunch of reviews this week,
> I've
> > actually already started.  I should have some time to continue doing more
> > reviews.  As a general thing, we've been pretty slow about getting these
> > done :(
> >
> > - Robert
> >
> > On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> > rohini.aditya@gmail.com> wrote:
> >
> >> I also did mark some of the enhancement jiras that Satish was working on
> >> and which might take time to review to 4.4.0
> >>
> >> Regards,
> >> Rohini
> >>
> >> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> >> rohini.aditya@gmail.com>
> >> wrote:
> >>
> >> > Abhishek,
> >> >     Can you mark all jiras already committed to trunk as 4.3.0 and
> those
> >> > planned in the scope document as 4.3.0? This will help filter on those
> >> > and review instead of referring to a google doc. I can pick up 10-15
> >> > jiras for review end of this week. We should try to get majority of
> >> patches
> >> > in the next 10 days and branch by Aug 12th. You can then use the 3rd
> >> week
> >> > for any pending critical issues and rolling the release candidate.
> >> >
> >> > Robert,
> >> >   I am thinking, that we should get rid of the trunk version and use
> >> > release numbers when committing from the beginning like all other
> >> projects
> >> > do. Unnecessary step to change from trunk to release version for every
> >> > release. If folks are in agreement, then will delete the trunk version
> >> > altogether. Does that require a mail to private?
> >> >
> >> > Regards,
> >> > Rohini
> >> >
> >> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <bafna.iitr@gmail.com
> >
> >> > wrote:
> >> >
> >> >> Hi All,
> >> >>
> >> >> As we prepared a Oozie 4.3.0 scope document <
> >> >>
> >>
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >> >
> >> >> and listed all the things which we wanted to include in the release.
> >> In the
> >> >> last few days, I spent some time and reviewed some of the patches and
> >> >> provided my feedback. I will continue my efforts to review more
> >> patches.
> >> >> These patches needs to be reviewed by committers and get committed. I
> >> >> request other contributors also, review some of the patches and
> provide
> >> >> their feedback.
> >> >>
> >> >> I am thinking, may be, we can decide upon some fixed (approximate)
> >> >> release date and start working accordingly. Because, If we wait for
> >> all the
> >> >> listed items to be reviewed and committed, then it might take a lot
> >> more
> >> >> time. I am thinking we should target third week of August to complete
> >> and
> >> >> release.
> >> >>
> >> >> Feel free to provide your feedback and suggestions.
> >> >>
> >> >> Thanks,
> >> >> Abhishek
> >> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <ba...@gmail.com>
> >> >> wrote:
> >> >> >
> >> >> > Hi,
> >> >> >
> >> >> > I looked into the available patches and prepared the list. I have
> >> put,
> >> >> scope content which we have discussed till now, into a word document.
> >> (For
> >> >> the better formatting and easy to read and comment).
> >> >> >
> >> >> > I have noted the additional Jira into the list. Added two more Jira
> >> >> related to spark (Scala version and Spark 2.0 property).
> >> >> >
> >> >> > Please take a look at the document (link below) and provide your
> >> >> comments.
> >> >> >
> >> >> > If this looks good, we can start the release process.
> >> >> >
> >> >> > Scope Document:
> >> >>
> >>
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
> >> >> >
> >> >> > Thanks,
> >> >> > Abhishek
> >> >> >
> >> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rk...@cloudera.com>
> >> >> wrote:
> >> >> >>
> >> >> >> That list looks good to me.  The only one I'm not sure of is
> >> OOZIE-2306
> >> >> >> (Oozie Health Check).  It's a new feature and it might make sense
> to
> >> >> let it
> >> >> >> "bake" a while first?  I'm also not sure it's quite ready yet.
> >> >> >>
> >> >> >> I'd also like to add a few others to the list:
> >> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> >> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> >> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> >> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other
> >> fixes)
> >> >> >>
> >> >> >> There's a number of other Patch Available JIRAs not on your list
> >> >> (including
> >> >> >> the ones I mentioned above).  While we don't need to include
> >> everything
> >> >> >> here (and we probably shouldn't nor do we have the review
> >> bandwidth),
> >> >> did
> >> >> >> you consider each of them when creating the list?  I currently see
> >> 97
> >> >> >> JIRAs: https://s.apache.org/oozie-patch-available
> >> >> >> I think we should include all Spark Action fixes/improvements,
> given
> >> >> how
> >> >> >> popular it seems to be.
> >> >> >>
> >> >> >> thanks
> >> >> >> - Robert
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> >> bafna.iitr@gmail.com>
> >> >> >> wrote:
> >> >> >>
> >> >> >>> Hi All,
> >> >> >>>
> >> >> >>> Below is the list of issues, picked from currently available
> >> patches.
> >> >> I
> >> >> >>> could not decide up any 'open' issue, If there is anything,
> please
> >> >> reply
> >> >> >>> here.
> >> >> >>>
> >> >> >>> We can 'drop/add' more if required based on the feedback.
> Important
> >> >> is,
> >> >> >>> how many of these we will be able to review and commit.
> >> >> >>>
> >> >> >>> I will starting looking into these issues and provide feedback.
> >> >> >>>
> >> >>
> >>
> ---------------------------------------------------------------------------
> >> >> >>> OOZIE-1978      Forkjoin validation code is ridiculously slow in
> >> some
> >> >> cases
> >> >> >>> OOZIE-2508      Documentation change for Coord action rerun
> >> >> [OOZIE-1735]
> >> >> >>> OOZIE-2583      oozie throws EL Exception when reference variable
> >> name
> >> >> >>> containing dot
> >> >> >>> OOZIE-2582      Populating external child Ids for action failures
> >> >> >>> OOZIE-2517      Add support for startCreatedTime and
> endCreatedTime
> >> >> >>> filters for coord and bundles
> >> >> >>> OOZIE-2512      ShareLibservice returns incorrect path for jar
> >> >> >>> OOZIE-2572      SLA DURATION miss not shown when job is running
> for
> >> >> longer
> >> >> >>> than expected time
> >> >> >>> OOZIE-2457      Oozie log parsing regex consume more than 90% cpu
> >> >> >>> OOZIE-2574      Oozie to support replication-enabled mysql urls
> >> >> >>> OOZIE-2506      Add logs into RecoverService for logging
> >> information
> >> >> about
> >> >> >>> queued commnads
> >> >> >>> OOZIE-2573      dataset url which contains spaces can not be
> >> handled
> >> >> >>> rightly
> >> >> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
> >> mapred-site
> >> >> >>> into spark launcher
> >> >> >>> OOZIE-2491      oozie acl cannot specify group,it does`t work
> >> >> >>> OOZIE-2539      Incorrect property key is used for 'hive log4j
> >> >> >>> configuration file for execution mode'
> >> >> >>> OOZIE-2555      Oozie SSL enable setup does not return port for
> >> admin
> >> >> >>> -servers
> >> >> >>> OOZIE-2515      Duplicate information for "Changing
> >> endtime/pausetime
> >> >> of a
> >> >> >>> Bundle Job" in CommandLineTool wiki
> >> >> >>> OOZIE-2498      Oozie CallerId configuration for downstream
> >> components
> >> >> >>> OOZIE-2516      Update web service documentation for jobs API
> >> >> >>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie doc
> >> >> does not
> >> >> >>> work
> >> >> >>> OOZIE-2513      log.scan.duration should not be used for error
> and
> >> >> audit
> >> >> >>> logs
> >> >> >>> OOZIE-2461      Workflow, Coordinator and Bundle job querying
> >> should
> >> >> have
> >> >> >>> last modified filter
> >> >> >>> OOZIE-2306      Oozie health check
> >> >> >>> OOZIE-2518      Oozie is creating many files and directories in
> >> /tmp
> >> >> per
> >> >> >>> day
> >> >> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> >> >> >>> OOZIE-2473      Connection pool for SMTP connection
> >> >> >>> OOZIE-2440      Exponential re-try policy for workflow action
> >> >> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure
> >> >> uploaded
> >> >> >>> jars are world readable
> >> >> >>> OOZIE-2390      Rerun with failed option removing completed
> output
> >> >> data
> >> >> >>> OOZIE-1810      Workflow cannot get into Failed state when kill
> >> >> control
> >> >> >>> node cannot resolve variable in message
> >> >> >>> OOZIE-2244      Oozie should mask passwords in the logs when
> >> logging
> >> >> >>> command arguments
> >> >> >>>
> >> >>
> >>
> ---------------------------------------------------------------------------
> >> >> >>> Thanks,
> >> >> >>> Abhishek
> >> >> >>>
> >> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <
> bafna.iitr@gmail.com
> >> >
> >> >> >>> wrote:
> >> >> >>>>
> >> >> >>>> Hi All,
> >> >> >>>>
> >> >> >>>> I looked into the commits (approx. 160), all looked good to me
> and
> >> >> >>> should be included into the release. If there are things, which
> >> needs
> >> >> to be
> >> >> >>> excluded, provide your comments.
> >> >> >>>>
> >> >> >>>> We still need to decide on the patch which are in progress
> status.
> >> >> >>>>
> >> >> >>>> On the upgrade side:
> >> >> >>>> Java version          1.6 to 1.7
> >> >> >>>> OpenJPA version       2.2.2 to 2.4.1
> >> >> >>>> Curator version       2.5.0 to 2.11.0
> >> >> >>>> Pig version           0.12.1 to 0.16.0
> >> >> >>>> Hive version          0.13.1 to 1.2.0
> >> >> >>>> HttpClient version    4.3 to 4.3.6
> >> >> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> >> >> >>>>
> >> >> >>>> Please provide your feedback on the above upgrades.
> >> >> >>>>
> >> >> >>>> Once we decide on the scope of the release. We will start
> working
> >> on
> >> >> the
> >> >> >>> completing/testing the release.
> >> >> >>>>
> >> >> >>>> Thanks,
> >> >> >>>> Abhishek
> >> >> >>>>
> >> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
> >> bafna.iitr@gmail.com
> >> >> >>> <ma...@gmail.com>> wrote:
> >> >> >>>>>
> >> >> >>>>> Okay. We will make the Java version to 1.7.
> >> >> >>>>>
> >> >> >>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient,
> >> Pig
> >> >> etc.
> >> >> >>>>>
> >> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done internally
> as
> >> >> well.
> >> >> >>>>>
> >> >> >>>>> As per the comments on the OOZIE-2231 <
> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator
> version
> >> >> also
> >> >> >>> can be upgraded to 2.11.
> >> >> >>>>>
> >> >> >>>>> Thanks,
> >> >> >>>>> Abhishek
> >> >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
> >> rkanter@cloudera.com
> >> >> >>> <ma...@cloudera.com>> wrote:
> >> >> >>>>>>
> >> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for
> over
> >> a
> >> >> year
> >> >> >>> now
> >> >> >>>>>> with no problems.  We did have to comment out a few lines in
> >> >> >>> MiniHCatServer
> >> >> >>>>>> and MiniHS2 that were no longer needed:
> >> >> >>>>>>
> >> >> >>>>>> diff --git
> >> >> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >> >>>>>> index 8699ff8..d694977 100644
> >> >> >>>>>> ---
> >> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >> >>>>>> +++
> >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> >> >> >>>>>>   public static void resetDefaultDBCreation() throws
> Exception {
> >> >> >>>>>>       // Need to do this, else default db will not be created
> >> for
> >> >> >>> local
> >> >> >>>>>> metastores.
> >> >> >>>>>>       // TestHiveMain will fail with
> >> >> >>>>>> InvalidObjectException(message:There is no database named
> >> default)
> >> >> >>>>>> -        Field declaredField =
> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >> >> >>>>>> -        declaredField.setAccessible(true);
> >> >> >>>>>> -        declaredField.set(null, false);
> >> >> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and
> >> later
> >> >> >>>>>> +        //Field declaredField =
> >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >> >> >>>>>> +        //declaredField.setAccessible(true);
> >> >> >>>>>> +        //declaredField.set(null, false);
> >> >> >>>>>>   }
> >> >> >>>>>>
> >> >> >>>>>>   public static void resetHiveConfStaticVariables() throws
> >> >> Exception
> >> >> >>> {
> >> >> >>>>>> diff --git
> >> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >> >>>>>> index c9dfd56..505dc55 100644
> >> >> >>>>>> ---
> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >> >>>>>> +++
> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> >> AbstractHiveService
> >> >> {
> >> >> >>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> >> >> getHost());
> >> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> >> >> >>> getBinaryPort());
> >> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >> >> >>>>>> getHttpPort());
> >> >> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >> >> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and
> later
> >> >> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >> >> >>>>>>
> >> >> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
> >> >> >>>>>>   fs.mkdirs(scratchDir);
> >> >> >>>>>>
> >> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> >> >> >>>>>> vranganathan@hortonworks.com <mailto:
> >> vranganathan@hortonworks.com
> >> >> >>
> >> >> >>> wrote:
> >> >> >>>>>>
> >> >> >>>>>>> Let us explore changing the default hive version also  to
> 1.x.
> >> >>  0.13
> >> >> >>> is
> >> >> >>>>>>> old
> >> >> >>>>>>>
> >> >> >>>>>>> Thanks
> >> >> >>>>>>>
> >> >> >>>>>>> Venkat
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com
> >> >> <mailto:
> >> >> >>> rkanter@cloudera.com>> wrote:
> >> >> >>>>>>>
> >> >> >>>>>>>  I don't think we have any guidelines defined anywhere about
> >> when
> >> >> >>> it's
> >> >> >>>>>>> okay
> >> >> >>>>>>>  to change the minimum Java version, but perhaps we need to
> >> wait
> >> >> >>> until
> >> >> >>>>>>> the
> >> >> >>>>>>>  next major release (5.0)?  That said, Hadoop did this in a
> >> minor
> >> >> >>>>>>> release
> >> >> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it in
> 4.3
> >> if
> >> >> >>>>>>> nobody has
> >> >> >>>>>>>  an objections; we should just make sure to call that out in
> >> our
> >> >> >>> release
> >> >> >>>>>>>  notes.
> >> >> >>>>>>>
> >> >> >>>>>>>  Otherwise, nothing comes to mind for anything we should
> leave
> >> out
> >> >> >>> other
> >> >> >>>>>>>  than the Oozie on Yarn stuff, which is not currently in
> trunk
> >> >> >>> anyway.
> >> >> >>>>>>>
> >> >> >>>>>>>  I know there's a number of Spark Action JIRAs; we should
> make
> >> >> sure
> >> >> >>> to
> >> >> >>>>>>>  include all of these and any pending ones.
> >> >> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more time
> to
> >> >> bake
> >> >> >>>>>>> that we
> >> >> >>>>>>>  should hold off on including in the release?
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>>  thanks
> >> >> >>>>>>>  - Robert
> >> >> >>>>>>>
> >> >> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> >> >> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
> >> >> >>>>>>>  wrote:
> >> >> >>>>>>>
> >> >> >>>>>>>> Hi All,
> >> >> >>>>>>>>
> >> >> >>>>>>>> Any suggestions or comments on the scope for the release?
> >> >> >>>>>>>>
> >> >> >>>>>>>> Thanks,
> >> >> >>>>>>>> Abhishek
> >> >> >>>>>>>>
> >> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> >> >> bafna.iitr@gmail.com
> >> >> >>> <ma...@gmail.com>>
> >> >> >>>>>>>> wrote:
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> Hi All,
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This
> >> >> thread
> >> >> >>>>>>> is
> >> >> >>>>>>>> for discussion about the scope of the release. Please
> provide
> >> >> your
> >> >> >>>>>>>> suggestion about what should be included (or any exclusions)
> >> in
> >> >> the
> >> >> >>>>>>> release.
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> Currently, we have around 158 patches submitted to trunk.
> >> Based
> >> >> on
> >> >> >>>>>>> the
> >> >> >>>>>>>> scope, we will include (exclude) more if required.
> >> >> >>>>>>>>> Version Upgrade
> >> >> >>>>>>>>> Java 1.7 (OOZIE-2036 <
> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> >> >> >>>>>>>>> OpenJPA (OOZIE-2488 <
> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> >> >> >>>>>>>>> Curator  (OOZIE-2231 <
> >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
> >> >> >>>>>>>>> ),
> >> >> >>>>>>>>> HttpClient  (OOZIE-2538 <
> >> >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> >> >> >>>>>>>>> Pig (OOZIE-2602 <
> >> >> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
> >> >> >>>>>>>> ).
> >> >> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database
> >> >> schema
> >> >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
> >> >> >>>>>>>>> Bug Fixes.
> >> >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above
> >> points
> >> >> or
> >> >> >>>>>>> more
> >> >> >>>>>>>> for the scope of the release. We can also, take some work
> >> which
> >> >> is
> >> >> >>>>>>> already
> >> >> >>>>>>>> done and review is pending for them (depending on the
> >> >> availability
> >> >> >>>>>>> of the
> >> >> >>>>>>>> reviewers).
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> Thanks,
> >> >> >>>>>>>>> Abhishek
> >> >> >>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>
> >> >> >>>>
> >> >> >>>
> >> >> >>>
> >> >> >
> >> >>
> >> >>
> >> >
> >>
> >
> >
>

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

Abhishek, next time you do a bulk change, please remember to tell it not to
send out an email for every change :)

On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rk...@cloudera.com> wrote:

> Rohini, that sounds good to me.  Especially because we're on "master"
> branch but "trunk" version in JIRA and it gets a little confusing.  Instead
> of deleting "trunk", I think we're better off just renaming it to the
> release number.  Otherwise, we'll have to re-label any current "trunk"
> fields.  We can use 4.4.0 for now, but I imagine that the next release
> might end up being 5.0.0 (with Oozie on Yarn); we can always rename it
> later.
>
> I'm not sure if there's a specific rule here, but I suppose we should
> probably have a vote in the private list to make things official as this is
> a release-management-related change.  Can you send an email?
>
> Abhishek, thanks for putting this all together.  I think Rohini's
> suggestion of updating the JIRAs is the right next step.  You're probably
> best off using the bulk change feature instead of doing each JIRA
> individually.  I should have time to do a bunch of reviews this week, I've
> actually already started.  I should have some time to continue doing more
> reviews.  As a general thing, we've been pretty slow about getting these
> done :(
>
> - Robert
>
> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <
> rohini.aditya@gmail.com> wrote:
>
>> I also did mark some of the enhancement jiras that Satish was working on
>> and which might take time to review to 4.4.0
>>
>> Regards,
>> Rohini
>>
>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
>> rohini.aditya@gmail.com>
>> wrote:
>>
>> > Abhishek,
>> >     Can you mark all jiras already committed to trunk as 4.3.0 and those
>> > planned in the scope document as 4.3.0? This will help filter on those
>> > and review instead of referring to a google doc. I can pick up 10-15
>> > jiras for review end of this week. We should try to get majority of
>> patches
>> > in the next 10 days and branch by Aug 12th. You can then use the 3rd
>> week
>> > for any pending critical issues and rolling the release candidate.
>> >
>> > Robert,
>> >   I am thinking, that we should get rid of the trunk version and use
>> > release numbers when committing from the beginning like all other
>> projects
>> > do. Unnecessary step to change from trunk to release version for every
>> > release. If folks are in agreement, then will delete the trunk version
>> > altogether. Does that require a mail to private?
>> >
>> > Regards,
>> > Rohini
>> >
>> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <ba...@gmail.com>
>> > wrote:
>> >
>> >> Hi All,
>> >>
>> >> As we prepared a Oozie 4.3.0 scope document <
>> >>
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
>> >
>> >> and listed all the things which we wanted to include in the release.
>> In the
>> >> last few days, I spent some time and reviewed some of the patches and
>> >> provided my feedback. I will continue my efforts to review more
>> patches.
>> >> These patches needs to be reviewed by committers and get committed. I
>> >> request other contributors also, review some of the patches and provide
>> >> their feedback.
>> >>
>> >> I am thinking, may be, we can decide upon some fixed (approximate)
>> >> release date and start working accordingly. Because, If we wait for
>> all the
>> >> listed items to be reviewed and committed, then it might take a lot
>> more
>> >> time. I am thinking we should target third week of August to complete
>> and
>> >> release.
>> >>
>> >> Feel free to provide your feedback and suggestions.
>> >>
>> >> Thanks,
>> >> Abhishek
>> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <ba...@gmail.com>
>> >> wrote:
>> >> >
>> >> > Hi,
>> >> >
>> >> > I looked into the available patches and prepared the list. I have
>> put,
>> >> scope content which we have discussed till now, into a word document.
>> (For
>> >> the better formatting and easy to read and comment).
>> >> >
>> >> > I have noted the additional Jira into the list. Added two more Jira
>> >> related to spark (Scala version and Spark 2.0 property).
>> >> >
>> >> > Please take a look at the document (link below) and provide your
>> >> comments.
>> >> >
>> >> > If this looks good, we can start the release process.
>> >> >
>> >> > Scope Document:
>> >>
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
>> >> >
>> >> > Thanks,
>> >> > Abhishek
>> >> >
>> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rk...@cloudera.com>
>> >> wrote:
>> >> >>
>> >> >> That list looks good to me.  The only one I'm not sure of is
>> OOZIE-2306
>> >> >> (Oozie Health Check).  It's a new feature and it might make sense to
>> >> let it
>> >> >> "bake" a while first?  I'm also not sure it's quite ready yet.
>> >> >>
>> >> >> I'd also like to add a few others to the list:
>> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other
>> fixes)
>> >> >>
>> >> >> There's a number of other Patch Available JIRAs not on your list
>> >> (including
>> >> >> the ones I mentioned above).  While we don't need to include
>> everything
>> >> >> here (and we probably shouldn't nor do we have the review
>> bandwidth),
>> >> did
>> >> >> you consider each of them when creating the list?  I currently see
>> 97
>> >> >> JIRAs: https://s.apache.org/oozie-patch-available
>> >> >> I think we should include all Spark Action fixes/improvements, given
>> >> how
>> >> >> popular it seems to be.
>> >> >>
>> >> >> thanks
>> >> >> - Robert
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>> Hi All,
>> >> >>>
>> >> >>> Below is the list of issues, picked from currently available
>> patches.
>> >> I
>> >> >>> could not decide up any 'open' issue, If there is anything, please
>> >> reply
>> >> >>> here.
>> >> >>>
>> >> >>> We can 'drop/add' more if required based on the feedback. Important
>> >> is,
>> >> >>> how many of these we will be able to review and commit.
>> >> >>>
>> >> >>> I will starting looking into these issues and provide feedback.
>> >> >>>
>> >>
>> ---------------------------------------------------------------------------
>> >> >>> OOZIE-1978      Forkjoin validation code is ridiculously slow in
>> some
>> >> cases
>> >> >>> OOZIE-2508      Documentation change for Coord action rerun
>> >> [OOZIE-1735]
>> >> >>> OOZIE-2583      oozie throws EL Exception when reference variable
>> name
>> >> >>> containing dot
>> >> >>> OOZIE-2582      Populating external child Ids for action failures
>> >> >>> OOZIE-2517      Add support for startCreatedTime and endCreatedTime
>> >> >>> filters for coord and bundles
>> >> >>> OOZIE-2512      ShareLibservice returns incorrect path for jar
>> >> >>> OOZIE-2572      SLA DURATION miss not shown when job is running for
>> >> longer
>> >> >>> than expected time
>> >> >>> OOZIE-2457      Oozie log parsing regex consume more than 90% cpu
>> >> >>> OOZIE-2574      Oozie to support replication-enabled mysql urls
>> >> >>> OOZIE-2506      Add logs into RecoverService for logging
>> information
>> >> about
>> >> >>> queued commnads
>> >> >>> OOZIE-2573      dataset url which contains spaces can not be
>> handled
>> >> >>> rightly
>> >> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
>> mapred-site
>> >> >>> into spark launcher
>> >> >>> OOZIE-2491      oozie acl cannot specify group,it does`t work
>> >> >>> OOZIE-2539      Incorrect property key is used for 'hive log4j
>> >> >>> configuration file for execution mode'
>> >> >>> OOZIE-2555      Oozie SSL enable setup does not return port for
>> admin
>> >> >>> -servers
>> >> >>> OOZIE-2515      Duplicate information for "Changing
>> endtime/pausetime
>> >> of a
>> >> >>> Bundle Job" in CommandLineTool wiki
>> >> >>> OOZIE-2498      Oozie CallerId configuration for downstream
>> components
>> >> >>> OOZIE-2516      Update web service documentation for jobs API
>> >> >>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie doc
>> >> does not
>> >> >>> work
>> >> >>> OOZIE-2513      log.scan.duration should not be used for error and
>> >> audit
>> >> >>> logs
>> >> >>> OOZIE-2461      Workflow, Coordinator and Bundle job querying
>> should
>> >> have
>> >> >>> last modified filter
>> >> >>> OOZIE-2306      Oozie health check
>> >> >>> OOZIE-2518      Oozie is creating many files and directories in
>> /tmp
>> >> per
>> >> >>> day
>> >> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>> >> >>> OOZIE-2473      Connection pool for SMTP connection
>> >> >>> OOZIE-2440      Exponential re-try policy for workflow action
>> >> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure
>> >> uploaded
>> >> >>> jars are world readable
>> >> >>> OOZIE-2390      Rerun with failed option removing completed output
>> >> data
>> >> >>> OOZIE-1810      Workflow cannot get into Failed state when kill
>> >> control
>> >> >>> node cannot resolve variable in message
>> >> >>> OOZIE-2244      Oozie should mask passwords in the logs when
>> logging
>> >> >>> command arguments
>> >> >>>
>> >>
>> ---------------------------------------------------------------------------
>> >> >>> Thanks,
>> >> >>> Abhishek
>> >> >>>
>> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <bafna.iitr@gmail.com
>> >
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> Hi All,
>> >> >>>>
>> >> >>>> I looked into the commits (approx. 160), all looked good to me and
>> >> >>> should be included into the release. If there are things, which
>> needs
>> >> to be
>> >> >>> excluded, provide your comments.
>> >> >>>>
>> >> >>>> We still need to decide on the patch which are in progress status.
>> >> >>>>
>> >> >>>> On the upgrade side:
>> >> >>>> Java version          1.6 to 1.7
>> >> >>>> OpenJPA version       2.2.2 to 2.4.1
>> >> >>>> Curator version       2.5.0 to 2.11.0
>> >> >>>> Pig version           0.12.1 to 0.16.0
>> >> >>>> Hive version          0.13.1 to 1.2.0
>> >> >>>> HttpClient version    4.3 to 4.3.6
>> >> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>> >> >>>>
>> >> >>>> Please provide your feedback on the above upgrades.
>> >> >>>>
>> >> >>>> Once we decide on the scope of the release. We will start working
>> on
>> >> the
>> >> >>> completing/testing the release.
>> >> >>>>
>> >> >>>> Thanks,
>> >> >>>> Abhishek
>> >> >>>>
>> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com
>> >> >>> <ma...@gmail.com>> wrote:
>> >> >>>>>
>> >> >>>>> Okay. We will make the Java version to 1.7.
>> >> >>>>>
>> >> >>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient,
>> Pig
>> >> etc.
>> >> >>>>>
>> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done internally as
>> >> well.
>> >> >>>>>
>> >> >>>>> As per the comments on the OOZIE-2231 <
>> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version
>> >> also
>> >> >>> can be upgraded to 2.11.
>> >> >>>>>
>> >> >>>>> Thanks,
>> >> >>>>> Abhishek
>> >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <
>> rkanter@cloudera.com
>> >> >>> <ma...@cloudera.com>> wrote:
>> >> >>>>>>
>> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for over
>> a
>> >> year
>> >> >>> now
>> >> >>>>>> with no problems.  We did have to comment out a few lines in
>> >> >>> MiniHCatServer
>> >> >>>>>> and MiniHS2 that were no longer needed:
>> >> >>>>>>
>> >> >>>>>> diff --git
>> >> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >> >>>>>> index 8699ff8..d694977 100644
>> >> >>>>>> ---
>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >> >>>>>> +++
>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>> >> >>>>>>   public static void resetDefaultDBCreation() throws Exception {
>> >> >>>>>>       // Need to do this, else default db will not be created
>> for
>> >> >>> local
>> >> >>>>>> metastores.
>> >> >>>>>>       // TestHiveMain will fail with
>> >> >>>>>> InvalidObjectException(message:There is no database named
>> default)
>> >> >>>>>> -        Field declaredField =
>> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> >> >>>>>> -        declaredField.setAccessible(true);
>> >> >>>>>> -        declaredField.set(null, false);
>> >> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and
>> later
>> >> >>>>>> +        //Field declaredField =
>> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> >> >>>>>> +        //declaredField.setAccessible(true);
>> >> >>>>>> +        //declaredField.set(null, false);
>> >> >>>>>>   }
>> >> >>>>>>
>> >> >>>>>>   public static void resetHiveConfStaticVariables() throws
>> >> Exception
>> >> >>> {
>> >> >>>>>> diff --git
>> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >> >>>>>> index c9dfd56..505dc55 100644
>> >> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
>> AbstractHiveService
>> >> {
>> >> >>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>> >> getHost());
>> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
>> >> >>> getBinaryPort());
>> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>> >> >>>>>> getHttpPort());
>> >> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> >> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>> >> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> >> >>>>>>
>> >> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
>> >> >>>>>>   fs.mkdirs(scratchDir);
>> >> >>>>>>
>> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>> >> >>>>>> vranganathan@hortonworks.com <mailto:
>> vranganathan@hortonworks.com
>> >> >>
>> >> >>> wrote:
>> >> >>>>>>
>> >> >>>>>>> Let us explore changing the default hive version also  to 1.x.
>> >>  0.13
>> >> >>> is
>> >> >>>>>>> old
>> >> >>>>>>>
>> >> >>>>>>> Thanks
>> >> >>>>>>>
>> >> >>>>>>> Venkat
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com
>> >> <mailto:
>> >> >>> rkanter@cloudera.com>> wrote:
>> >> >>>>>>>
>> >> >>>>>>>  I don't think we have any guidelines defined anywhere about
>> when
>> >> >>> it's
>> >> >>>>>>> okay
>> >> >>>>>>>  to change the minimum Java version, but perhaps we need to
>> wait
>> >> >>> until
>> >> >>>>>>> the
>> >> >>>>>>>  next major release (5.0)?  That said, Hadoop did this in a
>> minor
>> >> >>>>>>> release
>> >> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3
>> if
>> >> >>>>>>> nobody has
>> >> >>>>>>>  an objections; we should just make sure to call that out in
>> our
>> >> >>> release
>> >> >>>>>>>  notes.
>> >> >>>>>>>
>> >> >>>>>>>  Otherwise, nothing comes to mind for anything we should leave
>> out
>> >> >>> other
>> >> >>>>>>>  than the Oozie on Yarn stuff, which is not currently in trunk
>> >> >>> anyway.
>> >> >>>>>>>
>> >> >>>>>>>  I know there's a number of Spark Action JIRAs; we should make
>> >> sure
>> >> >>> to
>> >> >>>>>>>  include all of these and any pending ones.
>> >> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more time to
>> >> bake
>> >> >>>>>>> that we
>> >> >>>>>>>  should hold off on including in the release?
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>  thanks
>> >> >>>>>>>  - Robert
>> >> >>>>>>>
>> >> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>> >> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
>> >> >>>>>>>  wrote:
>> >> >>>>>>>
>> >> >>>>>>>> Hi All,
>> >> >>>>>>>>
>> >> >>>>>>>> Any suggestions or comments on the scope for the release?
>> >> >>>>>>>>
>> >> >>>>>>>> Thanks,
>> >> >>>>>>>> Abhishek
>> >> >>>>>>>>
>> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>> >> bafna.iitr@gmail.com
>> >> >>> <ma...@gmail.com>>
>> >> >>>>>>>> wrote:
>> >> >>>>>>>>>
>> >> >>>>>>>>> Hi All,
>> >> >>>>>>>>>
>> >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This
>> >> thread
>> >> >>>>>>> is
>> >> >>>>>>>> for discussion about the scope of the release. Please provide
>> >> your
>> >> >>>>>>>> suggestion about what should be included (or any exclusions)
>> in
>> >> the
>> >> >>>>>>> release.
>> >> >>>>>>>>>
>> >> >>>>>>>>> Currently, we have around 158 patches submitted to trunk.
>> Based
>> >> on
>> >> >>>>>>> the
>> >> >>>>>>>> scope, we will include (exclude) more if required.
>> >> >>>>>>>>> Version Upgrade
>> >> >>>>>>>>> Java 1.7 (OOZIE-2036 <
>> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>> >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>> >> >>>>>>>>> OpenJPA (OOZIE-2488 <
>> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>> >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>> >> >>>>>>>>> Curator  (OOZIE-2231 <
>> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
>> >> >>>>>>>>> ),
>> >> >>>>>>>>> HttpClient  (OOZIE-2538 <
>> >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>> >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>> >> >>>>>>>>> Pig (OOZIE-2602 <
>> >> https://issues.apache.org/jira/browse/OOZIE-2602 <
>> >> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
>> >> >>>>>>>> ).
>> >> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database
>> >> schema
>> >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
>> >> >>>>>>>>> Bug Fixes.
>> >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above
>> points
>> >> or
>> >> >>>>>>> more
>> >> >>>>>>>> for the scope of the release. We can also, take some work
>> which
>> >> is
>> >> >>>>>>> already
>> >> >>>>>>>> done and review is pending for them (depending on the
>> >> availability
>> >> >>>>>>> of the
>> >> >>>>>>>> reviewers).
>> >> >>>>>>>>>
>> >> >>>>>>>>> Thanks,
>> >> >>>>>>>>> Abhishek
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>
>> >> >>>>
>> >> >>>
>> >> >>>
>> >> >
>> >>
>> >>
>> >
>>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

Rohini, that sounds good to me.  Especially because we're on "master"
branch but "trunk" version in JIRA and it gets a little confusing.  Instead
of deleting "trunk", I think we're better off just renaming it to the
release number.  Otherwise, we'll have to re-label any current "trunk"
fields.  We can use 4.4.0 for now, but I imagine that the next release
might end up being 5.0.0 (with Oozie on Yarn); we can always rename it
later.

I'm not sure if there's a specific rule here, but I suppose we should
probably have a vote in the private list to make things official as this is
a release-management-related change.  Can you send an email?

Abhishek, thanks for putting this all together.  I think Rohini's
suggestion of updating the JIRAs is the right next step.  You're probably
best off using the bulk change feature instead of doing each JIRA
individually.  I should have time to do a bunch of reviews this week, I've
actually already started.  I should have some time to continue doing more
reviews.  As a general thing, we've been pretty slow about getting these
done :(

- Robert

On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <ro...@gmail.com>
wrote:

> I also did mark some of the enhancement jiras that Satish was working on
> and which might take time to review to 4.4.0
>
> Regards,
> Rohini
>
> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> rohini.aditya@gmail.com>
> wrote:
>
> > Abhishek,
> >     Can you mark all jiras already committed to trunk as 4.3.0 and those
> > planned in the scope document as 4.3.0? This will help filter on those
> > and review instead of referring to a google doc. I can pick up 10-15
> > jiras for review end of this week. We should try to get majority of
> patches
> > in the next 10 days and branch by Aug 12th. You can then use the 3rd week
> > for any pending critical issues and rolling the release candidate.
> >
> > Robert,
> >   I am thinking, that we should get rid of the trunk version and use
> > release numbers when committing from the beginning like all other
> projects
> > do. Unnecessary step to change from trunk to release version for every
> > release. If folks are in agreement, then will delete the trunk version
> > altogether. Does that require a mail to private?
> >
> > Regards,
> > Rohini
> >
> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <ba...@gmail.com>
> > wrote:
> >
> >> Hi All,
> >>
> >> As we prepared a Oozie 4.3.0 scope document <
> >>
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >
> >> and listed all the things which we wanted to include in the release. In
> the
> >> last few days, I spent some time and reviewed some of the patches and
> >> provided my feedback. I will continue my efforts to review more patches.
> >> These patches needs to be reviewed by committers and get committed. I
> >> request other contributors also, review some of the patches and provide
> >> their feedback.
> >>
> >> I am thinking, may be, we can decide upon some fixed (approximate)
> >> release date and start working accordingly. Because, If we wait for all
> the
> >> listed items to be reviewed and committed, then it might take a lot more
> >> time. I am thinking we should target third week of August to complete
> and
> >> release.
> >>
> >> Feel free to provide your feedback and suggestions.
> >>
> >> Thanks,
> >> Abhishek
> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <ba...@gmail.com>
> >> wrote:
> >> >
> >> > Hi,
> >> >
> >> > I looked into the available patches and prepared the list. I have put,
> >> scope content which we have discussed till now, into a word document.
> (For
> >> the better formatting and easy to read and comment).
> >> >
> >> > I have noted the additional Jira into the list. Added two more Jira
> >> related to spark (Scala version and Spark 2.0 property).
> >> >
> >> > Please take a look at the document (link below) and provide your
> >> comments.
> >> >
> >> > If this looks good, we can start the release process.
> >> >
> >> > Scope Document:
> >>
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
> >> >
> >> > Thanks,
> >> > Abhishek
> >> >
> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rk...@cloudera.com>
> >> wrote:
> >> >>
> >> >> That list looks good to me.  The only one I'm not sure of is
> OOZIE-2306
> >> >> (Oozie Health Check).  It's a new feature and it might make sense to
> >> let it
> >> >> "bake" a while first?  I'm also not sure it's quite ready yet.
> >> >>
> >> >> I'd also like to add a few others to the list:
> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes)
> >> >>
> >> >> There's a number of other Patch Available JIRAs not on your list
> >> (including
> >> >> the ones I mentioned above).  While we don't need to include
> everything
> >> >> here (and we probably shouldn't nor do we have the review bandwidth),
> >> did
> >> >> you consider each of them when creating the list?  I currently see 97
> >> >> JIRAs: https://s.apache.org/oozie-patch-available
> >> >> I think we should include all Spark Action fixes/improvements, given
> >> how
> >> >> popular it seems to be.
> >> >>
> >> >> thanks
> >> >> - Robert
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <
> bafna.iitr@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Hi All,
> >> >>>
> >> >>> Below is the list of issues, picked from currently available
> patches.
> >> I
> >> >>> could not decide up any 'open' issue, If there is anything, please
> >> reply
> >> >>> here.
> >> >>>
> >> >>> We can 'drop/add' more if required based on the feedback. Important
> >> is,
> >> >>> how many of these we will be able to review and commit.
> >> >>>
> >> >>> I will starting looking into these issues and provide feedback.
> >> >>>
> >>
> ---------------------------------------------------------------------------
> >> >>> OOZIE-1978      Forkjoin validation code is ridiculously slow in
> some
> >> cases
> >> >>> OOZIE-2508      Documentation change for Coord action rerun
> >> [OOZIE-1735]
> >> >>> OOZIE-2583      oozie throws EL Exception when reference variable
> name
> >> >>> containing dot
> >> >>> OOZIE-2582      Populating external child Ids for action failures
> >> >>> OOZIE-2517      Add support for startCreatedTime and endCreatedTime
> >> >>> filters for coord and bundles
> >> >>> OOZIE-2512      ShareLibservice returns incorrect path for jar
> >> >>> OOZIE-2572      SLA DURATION miss not shown when job is running for
> >> longer
> >> >>> than expected time
> >> >>> OOZIE-2457      Oozie log parsing regex consume more than 90% cpu
> >> >>> OOZIE-2574      Oozie to support replication-enabled mysql urls
> >> >>> OOZIE-2506      Add logs into RecoverService for logging information
> >> about
> >> >>> queued commnads
> >> >>> OOZIE-2573      dataset url which contains spaces can not be handled
> >> >>> rightly
> >> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and
> mapred-site
> >> >>> into spark launcher
> >> >>> OOZIE-2491      oozie acl cannot specify group,it does`t work
> >> >>> OOZIE-2539      Incorrect property key is used for 'hive log4j
> >> >>> configuration file for execution mode'
> >> >>> OOZIE-2555      Oozie SSL enable setup does not return port for
> admin
> >> >>> -servers
> >> >>> OOZIE-2515      Duplicate information for "Changing
> endtime/pausetime
> >> of a
> >> >>> Bundle Job" in CommandLineTool wiki
> >> >>> OOZIE-2498      Oozie CallerId configuration for downstream
> components
> >> >>> OOZIE-2516      Update web service documentation for jobs API
> >> >>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie doc
> >> does not
> >> >>> work
> >> >>> OOZIE-2513      log.scan.duration should not be used for error and
> >> audit
> >> >>> logs
> >> >>> OOZIE-2461      Workflow, Coordinator and Bundle job querying should
> >> have
> >> >>> last modified filter
> >> >>> OOZIE-2306      Oozie health check
> >> >>> OOZIE-2518      Oozie is creating many files and directories in /tmp
> >> per
> >> >>> day
> >> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> >> >>> OOZIE-2473      Connection pool for SMTP connection
> >> >>> OOZIE-2440      Exponential re-try policy for workflow action
> >> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure
> >> uploaded
> >> >>> jars are world readable
> >> >>> OOZIE-2390      Rerun with failed option removing completed output
> >> data
> >> >>> OOZIE-1810      Workflow cannot get into Failed state when kill
> >> control
> >> >>> node cannot resolve variable in message
> >> >>> OOZIE-2244      Oozie should mask passwords in the logs when logging
> >> >>> command arguments
> >> >>>
> >>
> ---------------------------------------------------------------------------
> >> >>> Thanks,
> >> >>> Abhishek
> >> >>>
> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <ba...@gmail.com>
> >> >>> wrote:
> >> >>>>
> >> >>>> Hi All,
> >> >>>>
> >> >>>> I looked into the commits (approx. 160), all looked good to me and
> >> >>> should be included into the release. If there are things, which
> needs
> >> to be
> >> >>> excluded, provide your comments.
> >> >>>>
> >> >>>> We still need to decide on the patch which are in progress status.
> >> >>>>
> >> >>>> On the upgrade side:
> >> >>>> Java version          1.6 to 1.7
> >> >>>> OpenJPA version       2.2.2 to 2.4.1
> >> >>>> Curator version       2.5.0 to 2.11.0
> >> >>>> Pig version           0.12.1 to 0.16.0
> >> >>>> Hive version          0.13.1 to 1.2.0
> >> >>>> HttpClient version    4.3 to 4.3.6
> >> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> >> >>>>
> >> >>>> Please provide your feedback on the above upgrades.
> >> >>>>
> >> >>>> Once we decide on the scope of the release. We will start working
> on
> >> the
> >> >>> completing/testing the release.
> >> >>>>
> >> >>>> Thanks,
> >> >>>> Abhishek
> >> >>>>
> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <bafna.iitr@gmail.com
> >> >>> <ma...@gmail.com>> wrote:
> >> >>>>>
> >> >>>>> Okay. We will make the Java version to 1.7.
> >> >>>>>
> >> >>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient,
> Pig
> >> etc.
> >> >>>>>
> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done internally as
> >> well.
> >> >>>>>
> >> >>>>> As per the comments on the OOZIE-2231 <
> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version
> >> also
> >> >>> can be upgraded to 2.11.
> >> >>>>>
> >> >>>>> Thanks,
> >> >>>>> Abhishek
> >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rkanter@cloudera.com
> >> >>> <ma...@cloudera.com>> wrote:
> >> >>>>>>
> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for over a
> >> year
> >> >>> now
> >> >>>>>> with no problems.  We did have to comment out a few lines in
> >> >>> MiniHCatServer
> >> >>>>>> and MiniHS2 that were no longer needed:
> >> >>>>>>
> >> >>>>>> diff --git
> >> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >>>>>> index 8699ff8..d694977 100644
> >> >>>>>> ---
> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >>>>>> +++
> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> >> >>>>>>   public static void resetDefaultDBCreation() throws Exception {
> >> >>>>>>       // Need to do this, else default db will not be created for
> >> >>> local
> >> >>>>>> metastores.
> >> >>>>>>       // TestHiveMain will fail with
> >> >>>>>> InvalidObjectException(message:There is no database named
> default)
> >> >>>>>> -        Field declaredField =
> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >> >>>>>> -        declaredField.setAccessible(true);
> >> >>>>>> -        declaredField.set(null, false);
> >> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and
> later
> >> >>>>>> +        //Field declaredField =
> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >> >>>>>> +        //declaredField.setAccessible(true);
> >> >>>>>> +        //declaredField.set(null, false);
> >> >>>>>>   }
> >> >>>>>>
> >> >>>>>>   public static void resetHiveConfStaticVariables() throws
> >> Exception
> >> >>> {
> >> >>>>>> diff --git
> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >>>>>> index c9dfd56..505dc55 100644
> >> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends
> AbstractHiveService
> >> {
> >> >>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> >> getHost());
> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> >> >>> getBinaryPort());
> >> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >> >>>>>> getHttpPort());
> >> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
> >> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >> >>>>>>
> >> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
> >> >>>>>>   fs.mkdirs(scratchDir);
> >> >>>>>>
> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> >> >>>>>> vranganathan@hortonworks.com <mailto:
> vranganathan@hortonworks.com
> >> >>
> >> >>> wrote:
> >> >>>>>>
> >> >>>>>>> Let us explore changing the default hive version also  to 1.x.
> >>  0.13
> >> >>> is
> >> >>>>>>> old
> >> >>>>>>>
> >> >>>>>>> Thanks
> >> >>>>>>>
> >> >>>>>>> Venkat
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com
> >> <mailto:
> >> >>> rkanter@cloudera.com>> wrote:
> >> >>>>>>>
> >> >>>>>>>  I don't think we have any guidelines defined anywhere about
> when
> >> >>> it's
> >> >>>>>>> okay
> >> >>>>>>>  to change the minimum Java version, but perhaps we need to wait
> >> >>> until
> >> >>>>>>> the
> >> >>>>>>>  next major release (5.0)?  That said, Hadoop did this in a
> minor
> >> >>>>>>> release
> >> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3
> if
> >> >>>>>>> nobody has
> >> >>>>>>>  an objections; we should just make sure to call that out in our
> >> >>> release
> >> >>>>>>>  notes.
> >> >>>>>>>
> >> >>>>>>>  Otherwise, nothing comes to mind for anything we should leave
> out
> >> >>> other
> >> >>>>>>>  than the Oozie on Yarn stuff, which is not currently in trunk
> >> >>> anyway.
> >> >>>>>>>
> >> >>>>>>>  I know there's a number of Spark Action JIRAs; we should make
> >> sure
> >> >>> to
> >> >>>>>>>  include all of these and any pending ones.
> >> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more time to
> >> bake
> >> >>>>>>> that we
> >> >>>>>>>  should hold off on including in the release?
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>  thanks
> >> >>>>>>>  - Robert
> >> >>>>>>>
> >> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> >> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
> >> >>>>>>>  wrote:
> >> >>>>>>>
> >> >>>>>>>> Hi All,
> >> >>>>>>>>
> >> >>>>>>>> Any suggestions or comments on the scope for the release?
> >> >>>>>>>>
> >> >>>>>>>> Thanks,
> >> >>>>>>>> Abhishek
> >> >>>>>>>>
> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> >> bafna.iitr@gmail.com
> >> >>> <ma...@gmail.com>>
> >> >>>>>>>> wrote:
> >> >>>>>>>>>
> >> >>>>>>>>> Hi All,
> >> >>>>>>>>>
> >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This
> >> thread
> >> >>>>>>> is
> >> >>>>>>>> for discussion about the scope of the release. Please provide
> >> your
> >> >>>>>>>> suggestion about what should be included (or any exclusions) in
> >> the
> >> >>>>>>> release.
> >> >>>>>>>>>
> >> >>>>>>>>> Currently, we have around 158 patches submitted to trunk.
> Based
> >> on
> >> >>>>>>> the
> >> >>>>>>>> scope, we will include (exclude) more if required.
> >> >>>>>>>>> Version Upgrade
> >> >>>>>>>>> Java 1.7 (OOZIE-2036 <
> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> >> >>>>>>>>> OpenJPA (OOZIE-2488 <
> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> >> >>>>>>>>> Curator  (OOZIE-2231 <
> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
> >> >>>>>>>>> ),
> >> >>>>>>>>> HttpClient  (OOZIE-2538 <
> >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> >> >>>>>>>>> Pig (OOZIE-2602 <
> >> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
> >> >>>>>>>> ).
> >> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database
> >> schema
> >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
> >> >>>>>>>>> Bug Fixes.
> >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above
> points
> >> or
> >> >>>>>>> more
> >> >>>>>>>> for the scope of the release. We can also, take some work which
> >> is
> >> >>>>>>> already
> >> >>>>>>>> done and review is pending for them (depending on the
> >> availability
> >> >>>>>>> of the
> >> >>>>>>>> reviewers).
> >> >>>>>>>>>
> >> >>>>>>>>> Thanks,
> >> >>>>>>>>> Abhishek
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>>
> >> >
> >>
> >>
> >
>

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy <ro...@gmail.com>]

I also did mark some of the enhancement jiras that Satish was working on
and which might take time to review to 4.4.0

Regards,
Rohini

On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <ro...@gmail.com>
wrote:

> Abhishek,
>     Can you mark all jiras already committed to trunk as 4.3.0 and those
> planned in the scope document as 4.3.0? This will help filter on those
> and review instead of referring to a google doc. I can pick up 10-15
> jiras for review end of this week. We should try to get majority of patches
> in the next 10 days and branch by Aug 12th. You can then use the 3rd week
> for any pending critical issues and rolling the release candidate.
>
> Robert,
>   I am thinking, that we should get rid of the trunk version and use
> release numbers when committing from the beginning like all other projects
> do. Unnecessary step to change from trunk to release version for every
> release. If folks are in agreement, then will delete the trunk version
> altogether. Does that require a mail to private?
>
> Regards,
> Rohini
>
> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> As we prepared a Oozie 4.3.0 scope document <
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu>
>> and listed all the things which we wanted to include in the release. In the
>> last few days, I spent some time and reviewed some of the patches and
>> provided my feedback. I will continue my efforts to review more patches.
>> These patches needs to be reviewed by committers and get committed. I
>> request other contributors also, review some of the patches and provide
>> their feedback.
>>
>> I am thinking, may be, we can decide upon some fixed (approximate)
>> release date and start working accordingly. Because, If we wait for all the
>> listed items to be reviewed and committed, then it might take a lot more
>> time. I am thinking we should target third week of August to complete and
>> release.
>>
>> Feel free to provide your feedback and suggestions.
>>
>> Thanks,
>> Abhishek
>> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > I looked into the available patches and prepared the list. I have put,
>> scope content which we have discussed till now, into a word document. (For
>> the better formatting and easy to read and comment).
>> >
>> > I have noted the additional Jira into the list. Added two more Jira
>> related to spark (Scala version and Spark 2.0 property).
>> >
>> > Please take a look at the document (link below) and provide your
>> comments.
>> >
>> > If this looks good, we can start the release process.
>> >
>> > Scope Document:
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
>> >
>> > Thanks,
>> > Abhishek
>> >
>> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>> >>
>> >> That list looks good to me.  The only one I'm not sure of is OOZIE-2306
>> >> (Oozie Health Check).  It's a new feature and it might make sense to
>> let it
>> >> "bake" a while first?  I'm also not sure it's quite ready yet.
>> >>
>> >> I'd also like to add a few others to the list:
>> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>> >> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes)
>> >>
>> >> There's a number of other Patch Available JIRAs not on your list
>> (including
>> >> the ones I mentioned above).  While we don't need to include everything
>> >> here (and we probably shouldn't nor do we have the review bandwidth),
>> did
>> >> you consider each of them when creating the list?  I currently see 97
>> >> JIRAs: https://s.apache.org/oozie-patch-available
>> >> I think we should include all Spark Action fixes/improvements, given
>> how
>> >> popular it seems to be.
>> >>
>> >> thanks
>> >> - Robert
>> >>
>> >>
>> >>
>> >>
>> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <ba...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi All,
>> >>>
>> >>> Below is the list of issues, picked from currently available patches.
>> I
>> >>> could not decide up any 'open' issue, If there is anything, please
>> reply
>> >>> here.
>> >>>
>> >>> We can 'drop/add' more if required based on the feedback. Important
>> is,
>> >>> how many of these we will be able to review and commit.
>> >>>
>> >>> I will starting looking into these issues and provide feedback.
>> >>>
>> ---------------------------------------------------------------------------
>> >>> OOZIE-1978      Forkjoin validation code is ridiculously slow in some
>> cases
>> >>> OOZIE-2508      Documentation change for Coord action rerun
>> [OOZIE-1735]
>> >>> OOZIE-2583      oozie throws EL Exception when reference variable name
>> >>> containing dot
>> >>> OOZIE-2582      Populating external child Ids for action failures
>> >>> OOZIE-2517      Add support for startCreatedTime and endCreatedTime
>> >>> filters for coord and bundles
>> >>> OOZIE-2512      ShareLibservice returns incorrect path for jar
>> >>> OOZIE-2572      SLA DURATION miss not shown when job is running for
>> longer
>> >>> than expected time
>> >>> OOZIE-2457      Oozie log parsing regex consume more than 90% cpu
>> >>> OOZIE-2574      Oozie to support replication-enabled mysql urls
>> >>> OOZIE-2506      Add logs into RecoverService for logging information
>> about
>> >>> queued commnads
>> >>> OOZIE-2573      dataset url which contains spaces can not be handled
>> >>> rightly
>> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and mapred-site
>> >>> into spark launcher
>> >>> OOZIE-2491      oozie acl cannot specify group,it does`t work
>> >>> OOZIE-2539      Incorrect property key is used for 'hive log4j
>> >>> configuration file for execution mode'
>> >>> OOZIE-2555      Oozie SSL enable setup does not return port for admin
>> >>> -servers
>> >>> OOZIE-2515      Duplicate information for "Changing endtime/pausetime
>> of a
>> >>> Bundle Job" in CommandLineTool wiki
>> >>> OOZIE-2498      Oozie CallerId configuration for downstream components
>> >>> OOZIE-2516      Update web service documentation for jobs API
>> >>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie doc
>> does not
>> >>> work
>> >>> OOZIE-2513      log.scan.duration should not be used for error and
>> audit
>> >>> logs
>> >>> OOZIE-2461      Workflow, Coordinator and Bundle job querying should
>> have
>> >>> last modified filter
>> >>> OOZIE-2306      Oozie health check
>> >>> OOZIE-2518      Oozie is creating many files and directories in /tmp
>> per
>> >>> day
>> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>> >>> OOZIE-2473      Connection pool for SMTP connection
>> >>> OOZIE-2440      Exponential re-try policy for workflow action
>> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure
>> uploaded
>> >>> jars are world readable
>> >>> OOZIE-2390      Rerun with failed option removing completed output
>> data
>> >>> OOZIE-1810      Workflow cannot get into Failed state when kill
>> control
>> >>> node cannot resolve variable in message
>> >>> OOZIE-2244      Oozie should mask passwords in the logs when logging
>> >>> command arguments
>> >>>
>> ---------------------------------------------------------------------------
>> >>> Thanks,
>> >>> Abhishek
>> >>>
>> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <ba...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> Hi All,
>> >>>>
>> >>>> I looked into the commits (approx. 160), all looked good to me and
>> >>> should be included into the release. If there are things, which needs
>> to be
>> >>> excluded, provide your comments.
>> >>>>
>> >>>> We still need to decide on the patch which are in progress status.
>> >>>>
>> >>>> On the upgrade side:
>> >>>> Java version          1.6 to 1.7
>> >>>> OpenJPA version       2.2.2 to 2.4.1
>> >>>> Curator version       2.5.0 to 2.11.0
>> >>>> Pig version           0.12.1 to 0.16.0
>> >>>> Hive version          0.13.1 to 1.2.0
>> >>>> HttpClient version    4.3 to 4.3.6
>> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>> >>>>
>> >>>> Please provide your feedback on the above upgrades.
>> >>>>
>> >>>> Once we decide on the scope of the release. We will start working on
>> the
>> >>> completing/testing the release.
>> >>>>
>> >>>> Thanks,
>> >>>> Abhishek
>> >>>>
>> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <bafna.iitr@gmail.com
>> >>> <ma...@gmail.com>> wrote:
>> >>>>>
>> >>>>> Okay. We will make the Java version to 1.7.
>> >>>>>
>> >>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig
>> etc.
>> >>>>>
>> >>>>> HttpClient, Pig (0.16) version upgrade we have done internally as
>> well.
>> >>>>>
>> >>>>> As per the comments on the OOZIE-2231 <
>> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version
>> also
>> >>> can be upgraded to 2.11.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Abhishek
>> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rkanter@cloudera.com
>> >>> <ma...@cloudera.com>> wrote:
>> >>>>>>
>> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for over a
>> year
>> >>> now
>> >>>>>> with no problems.  We did have to comment out a few lines in
>> >>> MiniHCatServer
>> >>>>>> and MiniHS2 that were no longer needed:
>> >>>>>>
>> >>>>>> diff --git
>> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >>>>>> index 8699ff8..d694977 100644
>> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>> >>>>>>   public static void resetDefaultDBCreation() throws Exception {
>> >>>>>>       // Need to do this, else default db will not be created for
>> >>> local
>> >>>>>> metastores.
>> >>>>>>       // TestHiveMain will fail with
>> >>>>>> InvalidObjectException(message:There is no database named default)
>> >>>>>> -        Field declaredField =
>> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> >>>>>> -        declaredField.setAccessible(true);
>> >>>>>> -        declaredField.set(null, false);
>> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>> >>>>>> +        //Field declaredField =
>> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> >>>>>> +        //declaredField.setAccessible(true);
>> >>>>>> +        //declaredField.set(null, false);
>> >>>>>>   }
>> >>>>>>
>> >>>>>>   public static void resetHiveConfStaticVariables() throws
>> Exception
>> >>> {
>> >>>>>> diff --git
>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >>>>>> index c9dfd56..505dc55 100644
>> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService
>> {
>> >>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
>> getHost());
>> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
>> >>> getBinaryPort());
>> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>> >>>>>> getHttpPort());
>> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> >>>>>>
>> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
>> >>>>>>   fs.mkdirs(scratchDir);
>> >>>>>>
>> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>> >>>>>> vranganathan@hortonworks.com <mailto:vranganathan@hortonworks.com
>> >>
>> >>> wrote:
>> >>>>>>
>> >>>>>>> Let us explore changing the default hive version also  to 1.x.
>>  0.13
>> >>> is
>> >>>>>>> old
>> >>>>>>>
>> >>>>>>> Thanks
>> >>>>>>>
>> >>>>>>> Venkat
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com
>> <mailto:
>> >>> rkanter@cloudera.com>> wrote:
>> >>>>>>>
>> >>>>>>>  I don't think we have any guidelines defined anywhere about when
>> >>> it's
>> >>>>>>> okay
>> >>>>>>>  to change the minimum Java version, but perhaps we need to wait
>> >>> until
>> >>>>>>> the
>> >>>>>>>  next major release (5.0)?  That said, Hadoop did this in a minor
>> >>>>>>> release
>> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
>> >>>>>>> nobody has
>> >>>>>>>  an objections; we should just make sure to call that out in our
>> >>> release
>> >>>>>>>  notes.
>> >>>>>>>
>> >>>>>>>  Otherwise, nothing comes to mind for anything we should leave out
>> >>> other
>> >>>>>>>  than the Oozie on Yarn stuff, which is not currently in trunk
>> >>> anyway.
>> >>>>>>>
>> >>>>>>>  I know there's a number of Spark Action JIRAs; we should make
>> sure
>> >>> to
>> >>>>>>>  include all of these and any pending ones.
>> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more time to
>> bake
>> >>>>>>> that we
>> >>>>>>>  should hold off on including in the release?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>  thanks
>> >>>>>>>  - Robert
>> >>>>>>>
>> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
>> >>>>>>>  wrote:
>> >>>>>>>
>> >>>>>>>> Hi All,
>> >>>>>>>>
>> >>>>>>>> Any suggestions or comments on the scope for the release?
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> Abhishek
>> >>>>>>>>
>> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com
>> >>> <ma...@gmail.com>>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Hi All,
>> >>>>>>>>>
>> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This
>> thread
>> >>>>>>> is
>> >>>>>>>> for discussion about the scope of the release. Please provide
>> your
>> >>>>>>>> suggestion about what should be included (or any exclusions) in
>> the
>> >>>>>>> release.
>> >>>>>>>>>
>> >>>>>>>>> Currently, we have around 158 patches submitted to trunk. Based
>> on
>> >>>>>>> the
>> >>>>>>>> scope, we will include (exclude) more if required.
>> >>>>>>>>> Version Upgrade
>> >>>>>>>>> Java 1.7 (OOZIE-2036 <
>> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>> >>>>>>>>> OpenJPA (OOZIE-2488 <
>> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>> >>>>>>>>> Curator  (OOZIE-2231 <
>> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
>> >>>>>>>>> ),
>> >>>>>>>>> HttpClient  (OOZIE-2538 <
>> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>> >>>>>>>>> Pig (OOZIE-2602 <
>> https://issues.apache.org/jira/browse/OOZIE-2602 <
>> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
>> >>>>>>>> ).
>> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database
>> schema
>> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
>> >>>>>>>>> Bug Fixes.
>> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above points
>> or
>> >>>>>>> more
>> >>>>>>>> for the scope of the release. We can also, take some work which
>> is
>> >>>>>>> already
>> >>>>>>>> done and review is pending for them (depending on the
>> availability
>> >>>>>>> of the
>> >>>>>>>> reviewers).
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>> Abhishek
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >
>>
>>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy <ro...@gmail.com>]

Abhishek,
    Can you mark all jiras already committed to trunk as 4.3.0 and those
planned in the scope document as 4.3.0? This will help filter on those and
review instead of referring to a google doc. I can pick up 10-15 jiras for
review end of this week. We should try to get majority of patches in the
next 10 days and branch by Aug 12th. You can then use the 3rd week for any
pending critical issues and rolling the release candidate.

Robert,
  I am thinking, that we should get rid of the trunk version and use
release numbers when committing from the beginning like all other projects
do. Unnecessary step to change from trunk to release version for every
release. If folks are in agreement, then will delete the trunk version
altogether. Does that require a mail to private?

Regards,
Rohini

On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <ba...@gmail.com>
wrote:

> Hi All,
>
> As we prepared a Oozie 4.3.0 scope document <
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu>
> and listed all the things which we wanted to include in the release. In the
> last few days, I spent some time and reviewed some of the patches and
> provided my feedback. I will continue my efforts to review more patches.
> These patches needs to be reviewed by committers and get committed. I
> request other contributors also, review some of the patches and provide
> their feedback.
>
> I am thinking, may be, we can decide upon some fixed (approximate) release
> date and start working accordingly. Because, If we wait for all the listed
> items to be reviewed and committed, then it might take a lot more time. I
> am thinking we should target third week of August to complete and release.
>
> Feel free to provide your feedback and suggestions.
>
> Thanks,
> Abhishek
> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> >
> > Hi,
> >
> > I looked into the available patches and prepared the list. I have put,
> scope content which we have discussed till now, into a word document. (For
> the better formatting and easy to read and comment).
> >
> > I have noted the additional Jira into the list. Added two more Jira
> related to spark (Scala version and Spark 2.0 property).
> >
> > Please take a look at the document (link below) and provide your
> comments.
> >
> > If this looks good, we can start the release process.
> >
> > Scope Document:
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
> >
> > Thanks,
> > Abhishek
> >
> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rk...@cloudera.com>
> wrote:
> >>
> >> That list looks good to me.  The only one I'm not sure of is OOZIE-2306
> >> (Oozie Health Check).  It's a new feature and it might make sense to
> let it
> >> "bake" a while first?  I'm also not sure it's quite ready yet.
> >>
> >> I'd also like to add a few others to the list:
> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> >> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes)
> >>
> >> There's a number of other Patch Available JIRAs not on your list
> (including
> >> the ones I mentioned above).  While we don't need to include everything
> >> here (and we probably shouldn't nor do we have the review bandwidth),
> did
> >> you consider each of them when creating the list?  I currently see 97
> >> JIRAs: https://s.apache.org/oozie-patch-available
> >> I think we should include all Spark Action fixes/improvements, given how
> >> popular it seems to be.
> >>
> >> thanks
> >> - Robert
> >>
> >>
> >>
> >>
> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <ba...@gmail.com>
> >> wrote:
> >>
> >>> Hi All,
> >>>
> >>> Below is the list of issues, picked from currently available patches. I
> >>> could not decide up any 'open' issue, If there is anything, please
> reply
> >>> here.
> >>>
> >>> We can 'drop/add' more if required based on the feedback. Important is,
> >>> how many of these we will be able to review and commit.
> >>>
> >>> I will starting looking into these issues and provide feedback.
> >>>
> ---------------------------------------------------------------------------
> >>> OOZIE-1978      Forkjoin validation code is ridiculously slow in some
> cases
> >>> OOZIE-2508      Documentation change for Coord action rerun
> [OOZIE-1735]
> >>> OOZIE-2583      oozie throws EL Exception when reference variable name
> >>> containing dot
> >>> OOZIE-2582      Populating external child Ids for action failures
> >>> OOZIE-2517      Add support for startCreatedTime and endCreatedTime
> >>> filters for coord and bundles
> >>> OOZIE-2512      ShareLibservice returns incorrect path for jar
> >>> OOZIE-2572      SLA DURATION miss not shown when job is running for
> longer
> >>> than expected time
> >>> OOZIE-2457      Oozie log parsing regex consume more than 90% cpu
> >>> OOZIE-2574      Oozie to support replication-enabled mysql urls
> >>> OOZIE-2506      Add logs into RecoverService for logging information
> about
> >>> queued commnads
> >>> OOZIE-2573      dataset url which contains spaces can not be handled
> >>> rightly
> >>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and mapred-site
> >>> into spark launcher
> >>> OOZIE-2491      oozie acl cannot specify group,it does`t work
> >>> OOZIE-2539      Incorrect property key is used for 'hive log4j
> >>> configuration file for execution mode'
> >>> OOZIE-2555      Oozie SSL enable setup does not return port for admin
> >>> -servers
> >>> OOZIE-2515      Duplicate information for "Changing endtime/pausetime
> of a
> >>> Bundle Job" in CommandLineTool wiki
> >>> OOZIE-2498      Oozie CallerId configuration for downstream components
> >>> OOZIE-2516      Update web service documentation for jobs API
> >>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie doc does
> not
> >>> work
> >>> OOZIE-2513      log.scan.duration should not be used for error and
> audit
> >>> logs
> >>> OOZIE-2461      Workflow, Coordinator and Bundle job querying should
> have
> >>> last modified filter
> >>> OOZIE-2306      Oozie health check
> >>> OOZIE-2518      Oozie is creating many files and directories in /tmp
> per
> >>> day
> >>> OOZIE-2409      hive2 action with hive 1.2.1 failed
> >>> OOZIE-2473      Connection pool for SMTP connection
> >>> OOZIE-2440      Exponential re-try policy for workflow action
> >>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure uploaded
> >>> jars are world readable
> >>> OOZIE-2390      Rerun with failed option removing completed output data
> >>> OOZIE-1810      Workflow cannot get into Failed state when kill control
> >>> node cannot resolve variable in message
> >>> OOZIE-2244      Oozie should mask passwords in the logs when logging
> >>> command arguments
> >>>
> ---------------------------------------------------------------------------
> >>> Thanks,
> >>> Abhishek
> >>>
> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <ba...@gmail.com>
> >>> wrote:
> >>>>
> >>>> Hi All,
> >>>>
> >>>> I looked into the commits (approx. 160), all looked good to me and
> >>> should be included into the release. If there are things, which needs
> to be
> >>> excluded, provide your comments.
> >>>>
> >>>> We still need to decide on the patch which are in progress status.
> >>>>
> >>>> On the upgrade side:
> >>>> Java version          1.6 to 1.7
> >>>> OpenJPA version       2.2.2 to 2.4.1
> >>>> Curator version       2.5.0 to 2.11.0
> >>>> Pig version           0.12.1 to 0.16.0
> >>>> Hive version          0.13.1 to 1.2.0
> >>>> HttpClient version    4.3 to 4.3.6
> >>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
> >>>>
> >>>> Please provide your feedback on the above upgrades.
> >>>>
> >>>> Once we decide on the scope of the release. We will start working on
> the
> >>> completing/testing the release.
> >>>>
> >>>> Thanks,
> >>>> Abhishek
> >>>>
> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <bafna.iitr@gmail.com
> >>> <ma...@gmail.com>> wrote:
> >>>>>
> >>>>> Okay. We will make the Java version to 1.7.
> >>>>>
> >>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig
> etc.
> >>>>>
> >>>>> HttpClient, Pig (0.16) version upgrade we have done internally as
> well.
> >>>>>
> >>>>> As per the comments on the OOZIE-2231 <
> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version
> also
> >>> can be upgraded to 2.11.
> >>>>>
> >>>>> Thanks,
> >>>>> Abhishek
> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rkanter@cloudera.com
> >>> <ma...@cloudera.com>> wrote:
> >>>>>>
> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for over a
> year
> >>> now
> >>>>>> with no problems.  We did have to comment out a few lines in
> >>> MiniHCatServer
> >>>>>> and MiniHS2 that were no longer needed:
> >>>>>>
> >>>>>> diff --git
> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>> index 8699ff8..d694977 100644
> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> >>>>>>   public static void resetDefaultDBCreation() throws Exception {
> >>>>>>       // Need to do this, else default db will not be created for
> >>> local
> >>>>>> metastores.
> >>>>>>       // TestHiveMain will fail with
> >>>>>> InvalidObjectException(message:There is no database named default)
> >>>>>> -        Field declaredField =
> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >>>>>> -        declaredField.setAccessible(true);
> >>>>>> -        declaredField.set(null, false);
> >>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
> >>>>>> +        //Field declaredField =
> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >>>>>> +        //declaredField.setAccessible(true);
> >>>>>> +        //declaredField.set(null, false);
> >>>>>>   }
> >>>>>>
> >>>>>>   public static void resetHiveConfStaticVariables() throws Exception
> >>> {
> >>>>>> diff --git
> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>> index c9dfd56..505dc55 100644
> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService {
> >>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST,
> getHost());
> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> >>> getBinaryPort());
> >>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >>>>>> getHttpPort());
> >>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
> >>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >>>>>>
> >>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
> >>>>>>   fs.mkdirs(scratchDir);
> >>>>>>
> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> >>>>>> vranganathan@hortonworks.com <ma...@hortonworks.com>>
> >>> wrote:
> >>>>>>
> >>>>>>> Let us explore changing the default hive version also  to 1.x.
>  0.13
> >>> is
> >>>>>>> old
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>>
> >>>>>>> Venkat
> >>>>>>>
> >>>>>>>
> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com
> <mailto:
> >>> rkanter@cloudera.com>> wrote:
> >>>>>>>
> >>>>>>>  I don't think we have any guidelines defined anywhere about when
> >>> it's
> >>>>>>> okay
> >>>>>>>  to change the minimum Java version, but perhaps we need to wait
> >>> until
> >>>>>>> the
> >>>>>>>  next major release (5.0)?  That said, Hadoop did this in a minor
> >>>>>>> release
> >>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
> >>>>>>> nobody has
> >>>>>>>  an objections; we should just make sure to call that out in our
> >>> release
> >>>>>>>  notes.
> >>>>>>>
> >>>>>>>  Otherwise, nothing comes to mind for anything we should leave out
> >>> other
> >>>>>>>  than the Oozie on Yarn stuff, which is not currently in trunk
> >>> anyway.
> >>>>>>>
> >>>>>>>  I know there's a number of Spark Action JIRAs; we should make sure
> >>> to
> >>>>>>>  include all of these and any pending ones.
> >>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more time to
> bake
> >>>>>>> that we
> >>>>>>>  should hold off on including in the release?
> >>>>>>>
> >>>>>>>
> >>>>>>>  thanks
> >>>>>>>  - Robert
> >>>>>>>
> >>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> >>> bafna.iitr@gmail.com <ma...@gmail.com>>
> >>>>>>>  wrote:
> >>>>>>>
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> Any suggestions or comments on the scope for the release?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Abhishek
> >>>>>>>>
> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <
> bafna.iitr@gmail.com
> >>> <ma...@gmail.com>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi All,
> >>>>>>>>>
> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This
> thread
> >>>>>>> is
> >>>>>>>> for discussion about the scope of the release. Please provide your
> >>>>>>>> suggestion about what should be included (or any exclusions) in
> the
> >>>>>>> release.
> >>>>>>>>>
> >>>>>>>>> Currently, we have around 158 patches submitted to trunk. Based
> on
> >>>>>>> the
> >>>>>>>> scope, we will include (exclude) more if required.
> >>>>>>>>> Version Upgrade
> >>>>>>>>> Java 1.7 (OOZIE-2036 <
> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> >>>>>>>>> OpenJPA (OOZIE-2488 <
> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> >>>>>>>>> Curator  (OOZIE-2231 <
> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> >>> https://issues.apache.org/jira/browse/OOZIE-2231>
> >>>>>>>>> ),
> >>>>>>>>> HttpClient  (OOZIE-2538 <
> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> >>>>>>>>> Pig (OOZIE-2602 <
> https://issues.apache.org/jira/browse/OOZIE-2602 <
> >>> https://issues.apache.org/jira/browse/OOZIE-2602>
> >>>>>>>> ).
> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database schema
> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
> >>>>>>>>> Bug Fixes.
> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above points
> or
> >>>>>>> more
> >>>>>>>> for the scope of the release. We can also, take some work which is
> >>>>>>> already
> >>>>>>>> done and review is pending for them (depending on the availability
> >>>>>>> of the
> >>>>>>>> reviewers).
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Abhishek
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

As we prepared a Oozie 4.3.0 scope document <https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu> and listed all the things which we wanted to include in the release. In the last few days, I spent some time and reviewed some of the patches and provided my feedback. I will continue my efforts to review more patches. These patches needs to be reviewed by committers and get committed. I request other contributors also, review some of the patches and provide their feedback.

I am thinking, may be, we can decide upon some fixed (approximate) release date and start working accordingly. Because, If we wait for all the listed items to be reviewed and committed, then it might take a lot more time. I am thinking we should target third week of August to complete and release.

Feel free to provide your feedback and suggestions.

Thanks,
Abhishek
> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hi,
> 
> I looked into the available patches and prepared the list. I have put, scope content which we have discussed till now, into a word document. (For the better formatting and easy to read and comment).
> 
> I have noted the additional Jira into the list. Added two more Jira related to spark (Scala version and Spark 2.0 property).
> 
> Please take a look at the document (link below) and provide your comments.
> 
> If this looks good, we can start the release process.
> 
> Scope Document: https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
> 
> Thanks,
> Abhishek 
> 
>> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rk...@cloudera.com> wrote:
>> 
>> That list looks good to me.  The only one I'm not sure of is OOZIE-2306
>> (Oozie Health Check).  It's a new feature and it might make sense to let it
>> "bake" a while first?  I'm also not sure it's quite ready yet.
>> 
>> I'd also like to add a few others to the list:
>> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes)
>> 
>> There's a number of other Patch Available JIRAs not on your list (including
>> the ones I mentioned above).  While we don't need to include everything
>> here (and we probably shouldn't nor do we have the review bandwidth), did
>> you consider each of them when creating the list?  I currently see 97
>> JIRAs: https://s.apache.org/oozie-patch-available
>> I think we should include all Spark Action fixes/improvements, given how
>> popular it seems to be.
>> 
>> thanks
>> - Robert
>> 
>> 
>> 
>> 
>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>> 
>>> Hi All,
>>> 
>>> Below is the list of issues, picked from currently available patches. I
>>> could not decide up any 'open' issue, If there is anything, please reply
>>> here.
>>> 
>>> We can 'drop/add' more if required based on the feedback. Important is,
>>> how many of these we will be able to review and commit.
>>> 
>>> I will starting looking into these issues and provide feedback.
>>> ---------------------------------------------------------------------------
>>> OOZIE-1978      Forkjoin validation code is ridiculously slow in some cases
>>> OOZIE-2508      Documentation change for Coord action rerun [OOZIE-1735]
>>> OOZIE-2583      oozie throws EL Exception when reference variable name
>>> containing dot
>>> OOZIE-2582      Populating external child Ids for action failures
>>> OOZIE-2517      Add support for startCreatedTime and endCreatedTime
>>> filters for coord and bundles
>>> OOZIE-2512      ShareLibservice returns incorrect path for jar
>>> OOZIE-2572      SLA DURATION miss not shown when job is running for longer
>>> than expected time
>>> OOZIE-2457      Oozie log parsing regex consume more than 90% cpu
>>> OOZIE-2574      Oozie to support replication-enabled mysql urls
>>> OOZIE-2506      Add logs into RecoverService for logging information about
>>> queued commnads
>>> OOZIE-2573      dataset url which contains spaces can not be handled
>>> rightly
>>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and mapred-site
>>> into spark launcher
>>> OOZIE-2491      oozie acl cannot specify group,it does`t work
>>> OOZIE-2539      Incorrect property key is used for 'hive log4j
>>> configuration file for execution mode'
>>> OOZIE-2555      Oozie SSL enable setup does not return port for admin
>>> -servers
>>> OOZIE-2515      Duplicate information for "Changing endtime/pausetime of a
>>> Bundle Job" in CommandLineTool wiki
>>> OOZIE-2498      Oozie CallerId configuration for downstream components
>>> OOZIE-2516      Update web service documentation for jobs API
>>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie doc does not
>>> work
>>> OOZIE-2513      log.scan.duration should not be used for error and audit
>>> logs
>>> OOZIE-2461      Workflow, Coordinator and Bundle job querying should have
>>> last modified filter
>>> OOZIE-2306      Oozie health check
>>> OOZIE-2518      Oozie is creating many files and directories in /tmp per
>>> day
>>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>>> OOZIE-2473      Connection pool for SMTP connection
>>> OOZIE-2440      Exponential re-try policy for workflow action
>>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure uploaded
>>> jars are world readable
>>> OOZIE-2390      Rerun with failed option removing completed output data
>>> OOZIE-1810      Workflow cannot get into Failed state when kill control
>>> node cannot resolve variable in message
>>> OOZIE-2244      Oozie should mask passwords in the logs when logging
>>> command arguments
>>> ---------------------------------------------------------------------------
>>> Thanks,
>>> Abhishek
>>> 
>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <ba...@gmail.com>
>>> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> I looked into the commits (approx. 160), all looked good to me and
>>> should be included into the release. If there are things, which needs to be
>>> excluded, provide your comments.
>>>> 
>>>> We still need to decide on the patch which are in progress status.
>>>> 
>>>> On the upgrade side:
>>>> Java version          1.6 to 1.7
>>>> OpenJPA version       2.2.2 to 2.4.1
>>>> Curator version       2.5.0 to 2.11.0
>>>> Pig version           0.12.1 to 0.16.0
>>>> Hive version          0.13.1 to 1.2.0
>>>> HttpClient version    4.3 to 4.3.6
>>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>>> 
>>>> Please provide your feedback on the above upgrades.
>>>> 
>>>> Once we decide on the scope of the release. We will start working on the
>>> completing/testing the release.
>>>> 
>>>> Thanks,
>>>> Abhishek
>>>> 
>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <bafna.iitr@gmail.com
>>> <ma...@gmail.com>> wrote:
>>>>> 
>>>>> Okay. We will make the Java version to 1.7.
>>>>> 
>>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig etc.
>>>>> 
>>>>> HttpClient, Pig (0.16) version upgrade we have done internally as well.
>>>>> 
>>>>> As per the comments on the OOZIE-2231 <
>>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version also
>>> can be upgraded to 2.11.
>>>>> 
>>>>> Thanks,
>>>>> Abhishek
>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rkanter@cloudera.com
>>> <ma...@cloudera.com>> wrote:
>>>>>> 
>>>>>> We've been using a version of Hive/HCat based on 1.1.0 for over a year
>>> now
>>>>>> with no problems.  We did have to comment out a few lines in
>>> MiniHCatServer
>>>>>> and MiniHS2 that were no longer needed:
>>>>>> 
>>>>>> diff --git
>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>> index 8699ff8..d694977 100644
>>>>>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>>   public static void resetDefaultDBCreation() throws Exception {
>>>>>>       // Need to do this, else default db will not be created for
>>> local
>>>>>> metastores.
>>>>>>       // TestHiveMain will fail with
>>>>>> InvalidObjectException(message:There is no database named default)
>>>>>> -        Field declaredField =
>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>> -        declaredField.setAccessible(true);
>>>>>> -        declaredField.set(null, false);
>>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>>>>>> +        //Field declaredField =
>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>>> +        //declaredField.setAccessible(true);
>>>>>> +        //declaredField.set(null, false);
>>>>>>   }
>>>>>> 
>>>>>>   public static void resetHiveConfStaticVariables() throws Exception
>>> {
>>>>>> diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>> index c9dfd56..505dc55 100644
>>>>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService {
>>>>>>   hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost());
>>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
>>> getBinaryPort());
>>>>>>   hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>>> getHttpPort());
>>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>>> 
>>>>>>   Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>>   fs.mkdirs(scratchDir);
>>>>>> 
>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>>> vranganathan@hortonworks.com <ma...@hortonworks.com>>
>>> wrote:
>>>>>> 
>>>>>>> Let us explore changing the default hive version also  to 1.x.   0.13
>>> is
>>>>>>> old
>>>>>>> 
>>>>>>> Thanks
>>>>>>> 
>>>>>>> Venkat
>>>>>>> 
>>>>>>> 
>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com <mailto:
>>> rkanter@cloudera.com>> wrote:
>>>>>>> 
>>>>>>>  I don't think we have any guidelines defined anywhere about when
>>> it's
>>>>>>> okay
>>>>>>>  to change the minimum Java version, but perhaps we need to wait
>>> until
>>>>>>> the
>>>>>>>  next major release (5.0)?  That said, Hadoop did this in a minor
>>>>>>> release
>>>>>>>  and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
>>>>>>> nobody has
>>>>>>>  an objections; we should just make sure to call that out in our
>>> release
>>>>>>>  notes.
>>>>>>> 
>>>>>>>  Otherwise, nothing comes to mind for anything we should leave out
>>> other
>>>>>>>  than the Oozie on Yarn stuff, which is not currently in trunk
>>> anyway.
>>>>>>> 
>>>>>>>  I know there's a number of Spark Action JIRAs; we should make sure
>>> to
>>>>>>>  include all of these and any pending ones.
>>>>>>>  @Rohini, @Puru, @Satish, anything you think needs more time to bake
>>>>>>> that we
>>>>>>>  should hold off on including in the release?
>>>>>>> 
>>>>>>> 
>>>>>>>  thanks
>>>>>>>  - Robert
>>>>>>> 
>>>>>>>  On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>>  wrote:
>>>>>>> 
>>>>>>>> Hi All,
>>>>>>>> 
>>>>>>>> Any suggestions or comments on the scope for the release?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Abhishek
>>>>>>>> 
>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <bafna.iitr@gmail.com
>>> <ma...@gmail.com>>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi All,
>>>>>>>>> 
>>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This thread
>>>>>>> is
>>>>>>>> for discussion about the scope of the release. Please provide your
>>>>>>>> suggestion about what should be included (or any exclusions) in the
>>>>>>> release.
>>>>>>>>> 
>>>>>>>>> Currently, we have around 158 patches submitted to trunk. Based on
>>>>>>> the
>>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>>> Version Upgrade
>>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>>> Curator  (OOZIE-2231 <
>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>>> ),
>>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>>> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602 <
>>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>>> ).
>>>>>>>>> New Features/Improvements: Add default JT and NN, Database schema
>>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
>>>>>>>>> Bug Fixes.
>>>>>>>>> Provide your opinion on inclusion/exclusion of the above points or
>>>>>>> more
>>>>>>>> for the scope of the release. We can also, take some work which is
>>>>>>> already
>>>>>>>> done and review is pending for them (depending on the availability
>>>>>>> of the
>>>>>>>> reviewers).
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Abhishek
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi,

I looked into the available patches and prepared the list. I have put, scope content which we have discussed till now, into a word document. (For the better formatting and easy to read and comment).

I have noted the additional Jira into the list. Added two more Jira related to spark (Scala version and Spark 2.0 property).

Please take a look at the document (link below) and provide your comments.

If this looks good, we can start the release process.

Scope Document: https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#

Thanks,
Abhishek 

> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rk...@cloudera.com> wrote:
> 
> That list looks good to me.  The only one I'm not sure of is OOZIE-2306
> (Oozie Health Check).  It's a new feature and it might make sense to let it
> "bake" a while first?  I'm also not sure it's quite ready yet.
> 
> I'd also like to add a few others to the list:
> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes)
> 
> There's a number of other Patch Available JIRAs not on your list (including
> the ones I mentioned above).  While we don't need to include everything
> here (and we probably shouldn't nor do we have the review bandwidth), did
> you consider each of them when creating the list?  I currently see 97
> JIRAs: https://s.apache.org/oozie-patch-available
> I think we should include all Spark Action fixes/improvements, given how
> popular it seems to be.
> 
> thanks
> - Robert
> 
> 
> 
> 
> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> 
>> Hi All,
>> 
>> Below is the list of issues, picked from currently available patches. I
>> could not decide up any 'open' issue, If there is anything, please reply
>> here.
>> 
>> We can 'drop/add' more if required based on the feedback. Important is,
>> how many of these we will be able to review and commit.
>> 
>> I will starting looking into these issues and provide feedback.
>> ---------------------------------------------------------------------------
>> OOZIE-1978      Forkjoin validation code is ridiculously slow in some cases
>> OOZIE-2508      Documentation change for Coord action rerun [OOZIE-1735]
>> OOZIE-2583      oozie throws EL Exception when reference variable name
>> containing dot
>> OOZIE-2582      Populating external child Ids for action failures
>> OOZIE-2517      Add support for startCreatedTime and endCreatedTime
>> filters for coord and bundles
>> OOZIE-2512      ShareLibservice returns incorrect path for jar
>> OOZIE-2572      SLA DURATION miss not shown when job is running for longer
>> than expected time
>> OOZIE-2457      Oozie log parsing regex consume more than 90% cpu
>> OOZIE-2574      Oozie to support replication-enabled mysql urls
>> OOZIE-2506      Add logs into RecoverService for logging information about
>> queued commnads
>> OOZIE-2573      dataset url which contains spaces can not be handled
>> rightly
>> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and mapred-site
>> into spark launcher
>> OOZIE-2491      oozie acl cannot specify group,it does`t work
>> OOZIE-2539      Incorrect property key is used for 'hive log4j
>> configuration file for execution mode'
>> OOZIE-2555      Oozie SSL enable setup does not return port for admin
>> -servers
>> OOZIE-2515      Duplicate information for "Changing endtime/pausetime of a
>> Bundle Job" in CommandLineTool wiki
>> OOZIE-2498      Oozie CallerId configuration for downstream components
>> OOZIE-2516      Update web service documentation for jobs API
>> OOZIE-2500      -DtestJarSimple option mentioned in minioozie doc does not
>> work
>> OOZIE-2513      log.scan.duration should not be used for error and audit
>> logs
>> OOZIE-2461      Workflow, Coordinator and Bundle job querying should have
>> last modified filter
>> OOZIE-2306      Oozie health check
>> OOZIE-2518      Oozie is creating many files and directories in /tmp per
>> day
>> OOZIE-2409      hive2 action with hive 1.2.1 failed
>> OOZIE-2473      Connection pool for SMTP connection
>> OOZIE-2440      Exponential re-try policy for workflow action
>> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure uploaded
>> jars are world readable
>> OOZIE-2390      Rerun with failed option removing completed output data
>> OOZIE-1810      Workflow cannot get into Failed state when kill control
>> node cannot resolve variable in message
>> OOZIE-2244      Oozie should mask passwords in the logs when logging
>> command arguments
>> ---------------------------------------------------------------------------
>> Thanks,
>> Abhishek
>> 
>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <ba...@gmail.com>
>> wrote:
>>> 
>>> Hi All,
>>> 
>>> I looked into the commits (approx. 160), all looked good to me and
>> should be included into the release. If there are things, which needs to be
>> excluded, provide your comments.
>>> 
>>> We still need to decide on the patch which are in progress status.
>>> 
>>> On the upgrade side:
>>> Java version          1.6 to 1.7
>>> OpenJPA version       2.2.2 to 2.4.1
>>> Curator version       2.5.0 to 2.11.0
>>> Pig version           0.12.1 to 0.16.0
>>> Hive version          0.13.1 to 1.2.0
>>> HttpClient version    4.3 to 4.3.6
>>> commons-httpclient version    3.1 to 4.3.6 (discontinued)
>>> 
>>> Please provide your feedback on the above upgrades.
>>> 
>>> Once we decide on the scope of the release. We will start working on the
>> completing/testing the release.
>>> 
>>> Thanks,
>>> Abhishek
>>> 
>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <bafna.iitr@gmail.com
>> <ma...@gmail.com>> wrote:
>>>> 
>>>> Okay. We will make the Java version to 1.7.
>>>> 
>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig etc.
>>>> 
>>>> HttpClient, Pig (0.16) version upgrade we have done internally as well.
>>>> 
>>>> As per the comments on the OOZIE-2231 <
>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version also
>> can be upgraded to 2.11.
>>>> 
>>>> Thanks,
>>>> Abhishek
>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rkanter@cloudera.com
>> <ma...@cloudera.com>> wrote:
>>>>> 
>>>>> We've been using a version of Hive/HCat based on 1.1.0 for over a year
>> now
>>>>> with no problems.  We did have to comment out a few lines in
>> MiniHCatServer
>>>>> and MiniHS2 that were no longer needed:
>>>>> 
>>>>> diff --git
>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>> index 8699ff8..d694977 100644
>>>>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>>>    public static void resetDefaultDBCreation() throws Exception {
>>>>>        // Need to do this, else default db will not be created for
>> local
>>>>> metastores.
>>>>>        // TestHiveMain will fail with
>>>>> InvalidObjectException(message:There is no database named default)
>>>>> -        Field declaredField =
>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>> -        declaredField.setAccessible(true);
>>>>> -        declaredField.set(null, false);
>>>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>>>>> +        //Field declaredField =
>>>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>>>> +        //declaredField.setAccessible(true);
>>>>> +        //declaredField.set(null, false);
>>>>>    }
>>>>> 
>>>>>    public static void resetHiveConfStaticVariables() throws Exception
>> {
>>>>> diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>> index c9dfd56..505dc55 100644
>>>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService {
>>>>>    hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost());
>>>>>    hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
>> getBinaryPort());
>>>>>    hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>>>> getHttpPort());
>>>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>>>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>>>> 
>>>>>    Path scratchDir = new Path(baseDfsDir, "scratch");
>>>>>    fs.mkdirs(scratchDir);
>>>>> 
>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>>>> vranganathan@hortonworks.com <ma...@hortonworks.com>>
>> wrote:
>>>>> 
>>>>>> Let us explore changing the default hive version also  to 1.x.   0.13
>> is
>>>>>> old
>>>>>> 
>>>>>> Thanks
>>>>>> 
>>>>>> Venkat
>>>>>> 
>>>>>> 
>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com <mailto:
>> rkanter@cloudera.com>> wrote:
>>>>>> 
>>>>>>   I don't think we have any guidelines defined anywhere about when
>> it's
>>>>>> okay
>>>>>>   to change the minimum Java version, but perhaps we need to wait
>> until
>>>>>> the
>>>>>>   next major release (5.0)?  That said, Hadoop did this in a minor
>>>>>> release
>>>>>>   and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
>>>>>> nobody has
>>>>>>   an objections; we should just make sure to call that out in our
>> release
>>>>>>   notes.
>>>>>> 
>>>>>>   Otherwise, nothing comes to mind for anything we should leave out
>> other
>>>>>>   than the Oozie on Yarn stuff, which is not currently in trunk
>> anyway.
>>>>>> 
>>>>>>   I know there's a number of Spark Action JIRAs; we should make sure
>> to
>>>>>>   include all of these and any pending ones.
>>>>>>   @Rohini, @Puru, @Satish, anything you think needs more time to bake
>>>>>> that we
>>>>>>   should hold off on including in the release?
>>>>>> 
>>>>>> 
>>>>>>   thanks
>>>>>>   - Robert
>>>>>> 
>>>>>>   On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
>> bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>>>   wrote:
>>>>>> 
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> Any suggestions or comments on the scope for the release?
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Abhishek
>>>>>>> 
>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <bafna.iitr@gmail.com
>> <ma...@gmail.com>>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hi All,
>>>>>>>> 
>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This thread
>>>>>> is
>>>>>>> for discussion about the scope of the release. Please provide your
>>>>>>> suggestion about what should be included (or any exclusions) in the
>>>>>> release.
>>>>>>>> 
>>>>>>>> Currently, we have around 158 patches submitted to trunk. Based on
>>>>>> the
>>>>>>> scope, we will include (exclude) more if required.
>>>>>>>> Version Upgrade
>>>>>>>> Java 1.7 (OOZIE-2036 <
>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
>> https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>>>> OpenJPA (OOZIE-2488 <
>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
>> https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>>>> Curator  (OOZIE-2231 <
>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
>> https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>>>> ),
>>>>>>>> HttpClient  (OOZIE-2538 <
>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
>> https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>>>> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602 <
>> https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>>>> ).
>>>>>>>> New Features/Improvements: Add default JT and NN, Database schema
>>>>>>> service check, instrumental metrics, Tomcat upgrade etc.
>>>>>>>> Bug Fixes.
>>>>>>>> Provide your opinion on inclusion/exclusion of the above points or
>>>>>> more
>>>>>>> for the scope of the release. We can also, take some work which is
>>>>>> already
>>>>>>> done and review is pending for them (depending on the availability
>>>>>> of the
>>>>>>> reviewers).
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Abhishek
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>> 
>> 
>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

That list looks good to me.  The only one I'm not sure of is OOZIE-2306
(Oozie Health Check).  It's a new feature and it might make sense to let it
"bake" a while first?  I'm also not sure it's quite ready yet.

I'd also like to add a few others to the list:
- OOZIE-2273 (MiniOozie does not work outside of Oozie)
- The subtasks under OOZIE-2036 (Drop support for Java 1.6)
--- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
--- OOZIE-1793 (Improve find bugs reporting for Oozie)
--- OOZIE-2552 (Update ActiveMQ version for security and other fixes)

There's a number of other Patch Available JIRAs not on your list (including
the ones I mentioned above).  While we don't need to include everything
here (and we probably shouldn't nor do we have the review bandwidth), did
you consider each of them when creating the list?  I currently see 97
JIRAs: https://s.apache.org/oozie-patch-available
I think we should include all Spark Action fixes/improvements, given how
popular it seems to be.

thanks
- Robert




On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <ba...@gmail.com>
wrote:

> Hi All,
>
> Below is the list of issues, picked from currently available patches. I
> could not decide up any 'open' issue, If there is anything, please reply
> here.
>
> We can 'drop/add' more if required based on the feedback. Important is,
> how many of these we will be able to review and commit.
>
> I will starting looking into these issues and provide feedback.
> ---------------------------------------------------------------------------
> OOZIE-1978      Forkjoin validation code is ridiculously slow in some cases
> OOZIE-2508      Documentation change for Coord action rerun [OOZIE-1735]
> OOZIE-2583      oozie throws EL Exception when reference variable name
> containing dot
> OOZIE-2582      Populating external child Ids for action failures
> OOZIE-2517      Add support for startCreatedTime and endCreatedTime
> filters for coord and bundles
> OOZIE-2512      ShareLibservice returns incorrect path for jar
> OOZIE-2572      SLA DURATION miss not shown when job is running for longer
> than expected time
> OOZIE-2457      Oozie log parsing regex consume more than 90% cpu
> OOZIE-2574      Oozie to support replication-enabled mysql urls
> OOZIE-2506      Add logs into RecoverService for logging information about
> queued commnads
> OOZIE-2573      dataset url which contains spaces can not be handled
> rightly
> OOZIE-2569      Adding yarn-site, core-site, hdfs-site and mapred-site
> into spark launcher
> OOZIE-2491      oozie acl cannot specify group,it does`t work
> OOZIE-2539      Incorrect property key is used for 'hive log4j
> configuration file for execution mode'
> OOZIE-2555      Oozie SSL enable setup does not return port for admin
> -servers
> OOZIE-2515      Duplicate information for "Changing endtime/pausetime of a
> Bundle Job" in CommandLineTool wiki
> OOZIE-2498      Oozie CallerId configuration for downstream components
> OOZIE-2516      Update web service documentation for jobs API
> OOZIE-2500      -DtestJarSimple option mentioned in minioozie doc does not
> work
> OOZIE-2513      log.scan.duration should not be used for error and audit
> logs
> OOZIE-2461      Workflow, Coordinator and Bundle job querying should have
> last modified filter
> OOZIE-2306      Oozie health check
> OOZIE-2518      Oozie is creating many files and directories in /tmp per
> day
> OOZIE-2409      hive2 action with hive 1.2.1 failed
> OOZIE-2473      Connection pool for SMTP connection
> OOZIE-2440      Exponential re-try policy for workflow action
> OOZIE-2458      'oozie-setup.sh sharelib create' should ensure uploaded
> jars are world readable
> OOZIE-2390      Rerun with failed option removing completed output data
> OOZIE-1810      Workflow cannot get into Failed state when kill control
> node cannot resolve variable in message
> OOZIE-2244      Oozie should mask passwords in the logs when logging
> command arguments
> ---------------------------------------------------------------------------
> Thanks,
> Abhishek
>
> > On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > I looked into the commits (approx. 160), all looked good to me and
> should be included into the release. If there are things, which needs to be
> excluded, provide your comments.
> >
> > We still need to decide on the patch which are in progress status.
> >
> > On the upgrade side:
> > Java version          1.6 to 1.7
> > OpenJPA version       2.2.2 to 2.4.1
> > Curator version       2.5.0 to 2.11.0
> > Pig version           0.12.1 to 0.16.0
> > Hive version          0.13.1 to 1.2.0
> > HttpClient version    4.3 to 4.3.6
> > commons-httpclient version    3.1 to 4.3.6 (discontinued)
> >
> > Please provide your feedback on the above upgrades.
> >
> > Once we decide on the scope of the release. We will start working on the
> completing/testing the release.
> >
> > Thanks,
> > Abhishek
> >
> >> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <bafna.iitr@gmail.com
> <ma...@gmail.com>> wrote:
> >>
> >> Okay. We will make the Java version to 1.7.
> >>
> >> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig etc.
> >>
> >> HttpClient, Pig (0.16) version upgrade we have done internally as well.
> >>
> >> As per the comments on the OOZIE-2231 <
> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version also
> can be upgraded to 2.11.
> >>
> >> Thanks,
> >> Abhishek
> >>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rkanter@cloudera.com
> <ma...@cloudera.com>> wrote:
> >>>
> >>> We've been using a version of Hive/HCat based on 1.1.0 for over a year
> now
> >>> with no problems.  We did have to comment out a few lines in
> MiniHCatServer
> >>> and MiniHS2 that were no longer needed:
> >>>
> >>> diff --git
> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>> index 8699ff8..d694977 100644
> >>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> >>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
> >>>     public static void resetDefaultDBCreation() throws Exception {
> >>>         // Need to do this, else default db will not be created for
> local
> >>> metastores.
> >>>         // TestHiveMain will fail with
> >>> InvalidObjectException(message:There is no database named default)
> >>> -        Field declaredField =
> >>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >>> -        declaredField.setAccessible(true);
> >>> -        declaredField.set(null, false);
> >>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
> >>> +        //Field declaredField =
> >>> HMSHandler.class.getDeclaredField("createDefaultDB");
> >>> +        //declaredField.setAccessible(true);
> >>> +        //declaredField.set(null, false);
> >>>     }
> >>>
> >>>     public static void resetHiveConfStaticVariables() throws Exception
> {
> >>> diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>> index c9dfd56..505dc55 100644
> >>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> >>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService {
> >>>     hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost());
> >>>     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT,
> getBinaryPort());
> >>>     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> >>> getHttpPort());
> >>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
> >>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> >>>
> >>>     Path scratchDir = new Path(baseDfsDir, "scratch");
> >>>     fs.mkdirs(scratchDir);
> >>>
> >>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> >>> vranganathan@hortonworks.com <ma...@hortonworks.com>>
> wrote:
> >>>
> >>>> Let us explore changing the default hive version also  to 1.x.   0.13
> is
> >>>> old
> >>>>
> >>>> Thanks
> >>>>
> >>>> Venkat
> >>>>
> >>>>
> >>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com <mailto:
> rkanter@cloudera.com>> wrote:
> >>>>
> >>>>    I don't think we have any guidelines defined anywhere about when
> it's
> >>>> okay
> >>>>    to change the minimum Java version, but perhaps we need to wait
> until
> >>>> the
> >>>>    next major release (5.0)?  That said, Hadoop did this in a minor
> >>>> release
> >>>>    and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
> >>>> nobody has
> >>>>    an objections; we should just make sure to call that out in our
> release
> >>>>    notes.
> >>>>
> >>>>    Otherwise, nothing comes to mind for anything we should leave out
> other
> >>>>    than the Oozie on Yarn stuff, which is not currently in trunk
> anyway.
> >>>>
> >>>>    I know there's a number of Spark Action JIRAs; we should make sure
> to
> >>>>    include all of these and any pending ones.
> >>>>    @Rohini, @Puru, @Satish, anything you think needs more time to bake
> >>>> that we
> >>>>    should hold off on including in the release?
> >>>>
> >>>>
> >>>>    thanks
> >>>>    - Robert
> >>>>
> >>>>    On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <
> bafna.iitr@gmail.com <ma...@gmail.com>>
> >>>>    wrote:
> >>>>
> >>>>> Hi All,
> >>>>>
> >>>>> Any suggestions or comments on the scope for the release?
> >>>>>
> >>>>> Thanks,
> >>>>> Abhishek
> >>>>>
> >>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <bafna.iitr@gmail.com
> <ma...@gmail.com>>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi All,
> >>>>>>
> >>>>>> Oozie is starting the release process for Oozie 4.3.0. This thread
> >>>> is
> >>>>> for discussion about the scope of the release. Please provide your
> >>>>> suggestion about what should be included (or any exclusions) in the
> >>>> release.
> >>>>>>
> >>>>>> Currently, we have around 158 patches submitted to trunk. Based on
> >>>> the
> >>>>> scope, we will include (exclude) more if required.
> >>>>>> Version Upgrade
> >>>>>> Java 1.7 (OOZIE-2036 <
> >>>> https://issues.apache.org/jira/browse/OOZIE-2036 <
> https://issues.apache.org/jira/browse/OOZIE-2036>>)
> >>>>>> OpenJPA (OOZIE-2488 <
> >>>> https://issues.apache.org/jira/browse/OOZIE-2488 <
> https://issues.apache.org/jira/browse/OOZIE-2488>>),
> >>>>>> Curator  (OOZIE-2231 <
> >>>> https://issues.apache.org/jira/browse/OOZIE-2231 <
> https://issues.apache.org/jira/browse/OOZIE-2231>
> >>>>>> ),
> >>>>>> HttpClient  (OOZIE-2538 <
> >>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <
> https://issues.apache.org/jira/browse/OOZIE-2538>>),
> >>>>>> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602 <
> https://issues.apache.org/jira/browse/OOZIE-2602>
> >>>>> ).
> >>>>>> New Features/Improvements: Add default JT and NN, Database schema
> >>>>> service check, instrumental metrics, Tomcat upgrade etc.
> >>>>>> Bug Fixes.
> >>>>>> Provide your opinion on inclusion/exclusion of the above points or
> >>>> more
> >>>>> for the scope of the release. We can also, take some work which is
> >>>> already
> >>>>> done and review is pending for them (depending on the availability
> >>>> of the
> >>>>> reviewers).
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Abhishek
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>
> >
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

Below is the list of issues, picked from currently available patches. I could not decide up any 'open' issue, If there is anything, please reply here.

We can 'drop/add' more if required based on the feedback. Important is, how many of these we will be able to review and commit.

I will starting looking into these issues and provide feedback.
---------------------------------------------------------------------------
OOZIE-1978	Forkjoin validation code is ridiculously slow in some cases
OOZIE-2508	Documentation change for Coord action rerun [OOZIE-1735]
OOZIE-2583	oozie throws EL Exception when reference variable name containing dot
OOZIE-2582	Populating external child Ids for action failures
OOZIE-2517	Add support for startCreatedTime and endCreatedTime filters for coord and bundles
OOZIE-2512	ShareLibservice returns incorrect path for jar
OOZIE-2572	SLA DURATION miss not shown when job is running for longer than expected time
OOZIE-2457	Oozie log parsing regex consume more than 90% cpu
OOZIE-2574	Oozie to support replication-enabled mysql urls
OOZIE-2506	Add logs into RecoverService for logging information about queued commnads
OOZIE-2573	dataset url which contains spaces can not be handled rightly
OOZIE-2569	Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher
OOZIE-2491	oozie acl cannot specify group,it does`t work
OOZIE-2539	Incorrect property key is used for 'hive log4j configuration file for execution mode'
OOZIE-2555	Oozie SSL enable setup does not return port for admin -servers
OOZIE-2515	Duplicate information for "Changing endtime/pausetime of a Bundle Job" in CommandLineTool wiki
OOZIE-2498	Oozie CallerId configuration for downstream components
OOZIE-2516	Update web service documentation for jobs API
OOZIE-2500	-DtestJarSimple option mentioned in minioozie doc does not work
OOZIE-2513	log.scan.duration should not be used for error and audit logs
OOZIE-2461	Workflow, Coordinator and Bundle job querying should have last modified filter
OOZIE-2306	Oozie health check
OOZIE-2518	Oozie is creating many files and directories in /tmp per day
OOZIE-2409	hive2 action with hive 1.2.1 failed
OOZIE-2473	Connection pool for SMTP connection
OOZIE-2440	Exponential re-try policy for workflow action
OOZIE-2458	'oozie-setup.sh sharelib create' should ensure uploaded jars are world readable
OOZIE-2390	Rerun with failed option removing completed output data
OOZIE-1810	Workflow cannot get into Failed state when kill control node cannot resolve variable in message
OOZIE-2244	Oozie should mask passwords in the logs when logging command arguments
---------------------------------------------------------------------------
Thanks,
Abhishek

> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hi All,
> 
> I looked into the commits (approx. 160), all looked good to me and should be included into the release. If there are things, which needs to be excluded, provide your comments.
> 
> We still need to decide on the patch which are in progress status.
> 
> On the upgrade side:
> Java version 		1.6 to 1.7
> OpenJPA version 	2.2.2 to 2.4.1
> Curator version 	2.5.0 to 2.11.0
> Pig version 		0.12.1 to 0.16.0
> Hive version 		0.13.1 to 1.2.0
> HttpClient version 	4.3 to 4.3.6
> commons-httpclient version 	3.1 to 4.3.6 (discontinued)
> 
> Please provide your feedback on the above upgrades. 
> 
> Once we decide on the scope of the release. We will start working on the completing/testing the release.
> 
> Thanks,
> Abhishek
> 
>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <bafna.iitr@gmail.com <ma...@gmail.com>> wrote:
>> 
>> Okay. We will make the Java version to 1.7.
>> 
>> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig etc.
>> 
>> HttpClient, Pig (0.16) version upgrade we have done internally as well.
>> 
>> As per the comments on the OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231>, curator version also can be upgraded to 2.11.
>> 
>> Thanks,
>> Abhishek
>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rkanter@cloudera.com <ma...@cloudera.com>> wrote:
>>> 
>>> We've been using a version of Hive/HCat based on 1.1.0 for over a year now
>>> with no problems.  We did have to comment out a few lines in MiniHCatServer
>>> and MiniHS2 that were no longer needed:
>>> 
>>> diff --git a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>> index 8699ff8..d694977 100644
>>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>>     public static void resetDefaultDBCreation() throws Exception {
>>>         // Need to do this, else default db will not be created for local
>>> metastores.
>>>         // TestHiveMain will fail with
>>> InvalidObjectException(message:There is no database named default)
>>> -        Field declaredField =
>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>> -        declaredField.setAccessible(true);
>>> -        declaredField.set(null, false);
>>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>>> +        //Field declaredField =
>>> HMSHandler.class.getDeclaredField("createDefaultDB");
>>> +        //declaredField.setAccessible(true);
>>> +        //declaredField.set(null, false);
>>>     }
>>> 
>>>     public static void resetHiveConfStaticVariables() throws Exception {
>>> diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>> index c9dfd56..505dc55 100644
>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService {
>>>     hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost());
>>>     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, getBinaryPort());
>>>     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>>> getHttpPort());
>>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>>> 
>>>     Path scratchDir = new Path(baseDfsDir, "scratch");
>>>     fs.mkdirs(scratchDir);
>>> 
>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>>> vranganathan@hortonworks.com <ma...@hortonworks.com>> wrote:
>>> 
>>>> Let us explore changing the default hive version also  to 1.x.   0.13 is
>>>> old
>>>> 
>>>> Thanks
>>>> 
>>>> Venkat
>>>> 
>>>> 
>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com <ma...@cloudera.com>> wrote:
>>>> 
>>>>    I don't think we have any guidelines defined anywhere about when it's
>>>> okay
>>>>    to change the minimum Java version, but perhaps we need to wait until
>>>> the
>>>>    next major release (5.0)?  That said, Hadoop did this in a minor
>>>> release
>>>>    and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
>>>> nobody has
>>>>    an objections; we should just make sure to call that out in our release
>>>>    notes.
>>>> 
>>>>    Otherwise, nothing comes to mind for anything we should leave out other
>>>>    than the Oozie on Yarn stuff, which is not currently in trunk anyway.
>>>> 
>>>>    I know there's a number of Spark Action JIRAs; we should make sure to
>>>>    include all of these and any pending ones.
>>>>    @Rohini, @Puru, @Satish, anything you think needs more time to bake
>>>> that we
>>>>    should hold off on including in the release?
>>>> 
>>>> 
>>>>    thanks
>>>>    - Robert
>>>> 
>>>>    On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>    wrote:
>>>> 
>>>>> Hi All,
>>>>> 
>>>>> Any suggestions or comments on the scope for the release?
>>>>> 
>>>>> Thanks,
>>>>> Abhishek
>>>>> 
>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <bafna.iitr@gmail.com <ma...@gmail.com>>
>>>>> wrote:
>>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> Oozie is starting the release process for Oozie 4.3.0. This thread
>>>> is
>>>>> for discussion about the scope of the release. Please provide your
>>>>> suggestion about what should be included (or any exclusions) in the
>>>> release.
>>>>>> 
>>>>>> Currently, we have around 158 patches submitted to trunk. Based on
>>>> the
>>>>> scope, we will include (exclude) more if required.
>>>>>> Version Upgrade
>>>>>> Java 1.7 (OOZIE-2036 <
>>>> https://issues.apache.org/jira/browse/OOZIE-2036 <https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>>> OpenJPA (OOZIE-2488 <
>>>> https://issues.apache.org/jira/browse/OOZIE-2488 <https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>>> Curator  (OOZIE-2231 <
>>>> https://issues.apache.org/jira/browse/OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>>> ),
>>>>>> HttpClient  (OOZIE-2538 <
>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 <https://issues.apache.org/jira/browse/OOZIE-2538>>),
>>>>>> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602>
>>>>> ).
>>>>>> New Features/Improvements: Add default JT and NN, Database schema
>>>>> service check, instrumental metrics, Tomcat upgrade etc.
>>>>>> Bug Fixes.
>>>>>> Provide your opinion on inclusion/exclusion of the above points or
>>>> more
>>>>> for the scope of the release. We can also, take some work which is
>>>> already
>>>>> done and review is pending for them (depending on the availability
>>>> of the
>>>>> reviewers).
>>>>>> 
>>>>>> Thanks,
>>>>>> Abhishek
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

I looked into the commits (approx. 160), all looked good to me and should be included into the release. If there are things, which needs to be excluded, provide your comments.

We still need to decide on the patch which are in progress status.

On the upgrade side:
Java version 		1.6 to 1.7
OpenJPA version 	2.2.2 to 2.4.1
Curator version 	2.5.0 to 2.11.0
Pig version 		0.12.1 to 0.16.0
Hive version 		0.13.1 to 1.2.0
HttpClient version 	4.3 to 4.3.6
commons-httpclient version 	3.1 to 4.3.6 (discontinued)

Please provide your feedback on the above upgrades. 

Once we decide on the scope of the release. We will start working on the completing/testing the release.

Thanks,
Abhishek

> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Okay. We will make the Java version to 1.7.
> 
> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig etc.
> 
> HttpClient, Pig (0.16) version upgrade we have done internally as well.
> 
> As per the comments on the OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231>, curator version also can be upgraded to 2.11.
> 
> Thanks,
> Abhishek
>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rkanter@cloudera.com <ma...@cloudera.com>> wrote:
>> 
>> We've been using a version of Hive/HCat based on 1.1.0 for over a year now
>> with no problems.  We did have to comment out a few lines in MiniHCatServer
>> and MiniHS2 that were no longer needed:
>> 
>> diff --git a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> index 8699ff8..d694977 100644
>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
>> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>>     public static void resetDefaultDBCreation() throws Exception {
>>         // Need to do this, else default db will not be created for local
>> metastores.
>>         // TestHiveMain will fail with
>> InvalidObjectException(message:There is no database named default)
>> -        Field declaredField =
>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> -        declaredField.setAccessible(true);
>> -        declaredField.set(null, false);
>> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>> +        //Field declaredField =
>> HMSHandler.class.getDeclaredField("createDefaultDB");
>> +        //declaredField.setAccessible(true);
>> +        //declaredField.set(null, false);
>>     }
>> 
>>     public static void resetHiveConfStaticVariables() throws Exception {
>> diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> index c9dfd56..505dc55 100644
>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService {
>>     hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost());
>>     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, getBinaryPort());
>>     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
>> getHttpPort());
>> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
>> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
>> 
>>     Path scratchDir = new Path(baseDfsDir, "scratch");
>>     fs.mkdirs(scratchDir);
>> 
>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
>> vranganathan@hortonworks.com <ma...@hortonworks.com>> wrote:
>> 
>>> Let us explore changing the default hive version also  to 1.x.   0.13 is
>>> old
>>> 
>>> Thanks
>>> 
>>> Venkat
>>> 
>>> 
>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkanter@cloudera.com <ma...@cloudera.com>> wrote:
>>> 
>>>    I don't think we have any guidelines defined anywhere about when it's
>>> okay
>>>    to change the minimum Java version, but perhaps we need to wait until
>>> the
>>>    next major release (5.0)?  That said, Hadoop did this in a minor
>>> release
>>>    and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
>>> nobody has
>>>    an objections; we should just make sure to call that out in our release
>>>    notes.
>>> 
>>>    Otherwise, nothing comes to mind for anything we should leave out other
>>>    than the Oozie on Yarn stuff, which is not currently in trunk anyway.
>>> 
>>>    I know there's a number of Spark Action JIRAs; we should make sure to
>>>    include all of these and any pending ones.
>>>    @Rohini, @Puru, @Satish, anything you think needs more time to bake
>>> that we
>>>    should hold off on including in the release?
>>> 
>>> 
>>>    thanks
>>>    - Robert
>>> 
>>>    On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <bafna.iitr@gmail.com <ma...@gmail.com>>
>>>    wrote:
>>> 
>>>> Hi All,
>>>> 
>>>> Any suggestions or comments on the scope for the release?
>>>> 
>>>> Thanks,
>>>> Abhishek
>>>> 
>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <bafna.iitr@gmail.com <ma...@gmail.com>>
>>>> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> Oozie is starting the release process for Oozie 4.3.0. This thread
>>> is
>>>> for discussion about the scope of the release. Please provide your
>>>> suggestion about what should be included (or any exclusions) in the
>>> release.
>>>>> 
>>>>> Currently, we have around 158 patches submitted to trunk. Based on
>>> the
>>>> scope, we will include (exclude) more if required.
>>>>> Version Upgrade
>>>>> Java 1.7 (OOZIE-2036 <
>>> https://issues.apache.org/jira/browse/OOZIE-2036 <https://issues.apache.org/jira/browse/OOZIE-2036>>)
>>>>> OpenJPA (OOZIE-2488 <
>>> https://issues.apache.org/jira/browse/OOZIE-2488 <https://issues.apache.org/jira/browse/OOZIE-2488>>),
>>>>> Curator  (OOZIE-2231 <
>>> https://issues.apache.org/jira/browse/OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231>
>>>>> ),
>>>>> HttpClient  (OOZIE-2538 <
>>>> https://issues.apache.org/jira/browse/OOZIE-2538>),
>>>>> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602
>>>> ).
>>>>> New Features/Improvements: Add default JT and NN, Database schema
>>>> service check, instrumental metrics, Tomcat upgrade etc.
>>>>> Bug Fixes.
>>>>> Provide your opinion on inclusion/exclusion of the above points or
>>> more
>>>> for the scope of the release. We can also, take some work which is
>>> already
>>>> done and review is pending for them (depending on the availability
>>> of the
>>>> reviewers).
>>>>> 
>>>>> Thanks,
>>>>> Abhishek
>>>> 
>>>> 
>>> 
>>> 
>>> 
> 

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Okay. We will make the Java version to 1.7.

There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig etc.

HttpClient, Pig (0.16) version upgrade we have done internally as well.

As per the comments on the OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231>, curator version also can be upgraded to 2.11.

Thanks,
Abhishek
> On Jul 16, 2016, at 2:06 AM, Robert Kanter <rk...@cloudera.com> wrote:
> 
> We've been using a version of Hive/HCat based on 1.1.0 for over a year now
> with no problems.  We did have to comment out a few lines in MiniHCatServer
> and MiniHS2 that were no longer needed:
> 
> diff --git a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> index 8699ff8..d694977 100644
> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
> @@ -150,9 +150,10 @@ public class MiniHCatServer {
>     public static void resetDefaultDBCreation() throws Exception {
>         // Need to do this, else default db will not be created for local
> metastores.
>         // TestHiveMain will fail with
> InvalidObjectException(message:There is no database named default)
> -        Field declaredField =
> HMSHandler.class.getDeclaredField("createDefaultDB");
> -        declaredField.setAccessible(true);
> -        declaredField.set(null, false);
> +        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
> +        //Field declaredField =
> HMSHandler.class.getDeclaredField("createDefaultDB");
> +        //declaredField.setAccessible(true);
> +        //declaredField.set(null, false);
>     }
> 
>     public static void resetHiveConfStaticVariables() throws Exception {
> diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> index c9dfd56..505dc55 100644
> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService {
>     hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost());
>     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, getBinaryPort());
>     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
> getHttpPort());
> -    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> +    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
> +    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();
> 
>     Path scratchDir = new Path(baseDfsDir, "scratch");
>     fs.mkdirs(scratchDir);
> 
> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
> vranganathan@hortonworks.com> wrote:
> 
>> Let us explore changing the default hive version also  to 1.x.   0.13 is
>> old
>> 
>> Thanks
>> 
>> Venkat
>> 
>> 
>> On 7/15/16, 11:31 AM, "Robert Kanter" <rk...@cloudera.com> wrote:
>> 
>>    I don't think we have any guidelines defined anywhere about when it's
>> okay
>>    to change the minimum Java version, but perhaps we need to wait until
>> the
>>    next major release (5.0)?  That said, Hadoop did this in a minor
>> release
>>    and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
>> nobody has
>>    an objections; we should just make sure to call that out in our release
>>    notes.
>> 
>>    Otherwise, nothing comes to mind for anything we should leave out other
>>    than the Oozie on Yarn stuff, which is not currently in trunk anyway.
>> 
>>    I know there's a number of Spark Action JIRAs; we should make sure to
>>    include all of these and any pending ones.
>>    @Rohini, @Puru, @Satish, anything you think needs more time to bake
>> that we
>>    should hold off on including in the release?
>> 
>> 
>>    thanks
>>    - Robert
>> 
>>    On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <ba...@gmail.com>
>>    wrote:
>> 
>>> Hi All,
>>> 
>>> Any suggestions or comments on the scope for the release?
>>> 
>>> Thanks,
>>> Abhishek
>>> 
>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <ba...@gmail.com>
>>> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> Oozie is starting the release process for Oozie 4.3.0. This thread
>> is
>>> for discussion about the scope of the release. Please provide your
>>> suggestion about what should be included (or any exclusions) in the
>> release.
>>>> 
>>>> Currently, we have around 158 patches submitted to trunk. Based on
>> the
>>> scope, we will include (exclude) more if required.
>>>> Version Upgrade
>>>> Java 1.7 (OOZIE-2036 <
>> https://issues.apache.org/jira/browse/OOZIE-2036>)
>>>> OpenJPA (OOZIE-2488 <
>> https://issues.apache.org/jira/browse/OOZIE-2488>),
>>>> Curator  (OOZIE-2231 <
>> https://issues.apache.org/jira/browse/OOZIE-2231
>>>> ),
>>>> HttpClient  (OOZIE-2538 <
>>> https://issues.apache.org/jira/browse/OOZIE-2538>),
>>>> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602
>>> ).
>>>> New Features/Improvements: Add default JT and NN, Database schema
>>> service check, instrumental metrics, Tomcat upgrade etc.
>>>> Bug Fixes.
>>>> Provide your opinion on inclusion/exclusion of the above points or
>> more
>>> for the scope of the release. We can also, take some work which is
>> already
>>> done and review is pending for them (depending on the availability
>> of the
>>> reviewers).
>>>> 
>>>> Thanks,
>>>> Abhishek
>>> 
>>> 
>> 
>> 
>> 

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

We've been using a version of Hive/HCat based on 1.1.0 for over a year now
with no problems.  We did have to comment out a few lines in MiniHCatServer
and MiniHS2 that were no longer needed:

diff --git a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
index 8699ff8..d694977 100644
--- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
+++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java
@@ -150,9 +150,10 @@ public class MiniHCatServer {
     public static void resetDefaultDBCreation() throws Exception {
         // Need to do this, else default db will not be created for local
metastores.
         // TestHiveMain will fail with
InvalidObjectException(message:There is no database named default)
-        Field declaredField =
HMSHandler.class.getDeclaredField("createDefaultDB");
-        declaredField.setAccessible(true);
-        declaredField.set(null, false);
+        // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
+        //Field declaredField =
HMSHandler.class.getDeclaredField("createDefaultDB");
+        //declaredField.setAccessible(true);
+        //declaredField.set(null, false);
     }

     public static void resetHiveConfStaticVariables() throws Exception {
diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
index c9dfd56..505dc55 100644
--- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
+++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java
@@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService {
     hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost());
     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, getBinaryPort());
     hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT,
getHttpPort());
-    HiveMetaStore.HMSHandler.resetDefaultDBFlag();
+    // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later
+    //HiveMetaStore.HMSHandler.resetDefaultDBFlag();

     Path scratchDir = new Path(baseDfsDir, "scratch");
     fs.mkdirs(scratchDir);

On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan <
vranganathan@hortonworks.com> wrote:

> Let us explore changing the default hive version also  to 1.x.   0.13 is
> old
>
> Thanks
>
> Venkat
>
>
> On 7/15/16, 11:31 AM, "Robert Kanter" <rk...@cloudera.com> wrote:
>
>     I don't think we have any guidelines defined anywhere about when it's
> okay
>     to change the minimum Java version, but perhaps we need to wait until
> the
>     next major release (5.0)?  That said, Hadoop did this in a minor
> release
>     and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if
> nobody has
>     an objections; we should just make sure to call that out in our release
>     notes.
>
>     Otherwise, nothing comes to mind for anything we should leave out other
>     than the Oozie on Yarn stuff, which is not currently in trunk anyway.
>
>     I know there's a number of Spark Action JIRAs; we should make sure to
>     include all of these and any pending ones.
>     @Rohini, @Puru, @Satish, anything you think needs more time to bake
> that we
>     should hold off on including in the release?
>
>
>     thanks
>     - Robert
>
>     On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <ba...@gmail.com>
>     wrote:
>
>     > Hi All,
>     >
>     > Any suggestions or comments on the scope for the release?
>     >
>     > Thanks,
>     > Abhishek
>     >
>     > > On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <ba...@gmail.com>
>     > wrote:
>     > >
>     > > Hi All,
>     > >
>     > > Oozie is starting the release process for Oozie 4.3.0. This thread
> is
>     > for discussion about the scope of the release. Please provide your
>     > suggestion about what should be included (or any exclusions) in the
> release.
>     > >
>     > > Currently, we have around 158 patches submitted to trunk. Based on
> the
>     > scope, we will include (exclude) more if required.
>     > > Version Upgrade
>     > > Java 1.7 (OOZIE-2036 <
> https://issues.apache.org/jira/browse/OOZIE-2036>)
>     > > OpenJPA (OOZIE-2488 <
> https://issues.apache.org/jira/browse/OOZIE-2488>),
>     > > Curator  (OOZIE-2231 <
> https://issues.apache.org/jira/browse/OOZIE-2231
>     > >),
>     > > HttpClient  (OOZIE-2538 <
>     > https://issues.apache.org/jira/browse/OOZIE-2538>),
>     > > Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602
> >).
>     > > New Features/Improvements: Add default JT and NN, Database schema
>     > service check, instrumental metrics, Tomcat upgrade etc.
>     > > Bug Fixes.
>     > > Provide your opinion on inclusion/exclusion of the above points or
> more
>     > for the scope of the release. We can also, take some work which is
> already
>     > done and review is pending for them (depending on the availability
> of the
>     > reviewers).
>     > >
>     > > Thanks,
>     > > Abhishek
>     >
>     >
>
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Venkat Ranganathan <vr...@hortonworks.com>]

Let us explore changing the default hive version also  to 1.x.   0.13 is old

Thanks

Venkat


On 7/15/16, 11:31 AM, "Robert Kanter" <rk...@cloudera.com> wrote:

    I don't think we have any guidelines defined anywhere about when it's okay
    to change the minimum Java version, but perhaps we need to wait until the
    next major release (5.0)?  That said, Hadoop did this in a minor release
    and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if nobody has
    an objections; we should just make sure to call that out in our release
    notes.
    
    Otherwise, nothing comes to mind for anything we should leave out other
    than the Oozie on Yarn stuff, which is not currently in trunk anyway.
    
    I know there's a number of Spark Action JIRAs; we should make sure to
    include all of these and any pending ones.
    @Rohini, @Puru, @Satish, anything you think needs more time to bake that we
    should hold off on including in the release?
    
    
    thanks
    - Robert
    
    On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <ba...@gmail.com>
    wrote:
    
    > Hi All,
    >
    > Any suggestions or comments on the scope for the release?
    >
    > Thanks,
    > Abhishek
    >
    > > On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <ba...@gmail.com>
    > wrote:
    > >
    > > Hi All,
    > >
    > > Oozie is starting the release process for Oozie 4.3.0. This thread is
    > for discussion about the scope of the release. Please provide your
    > suggestion about what should be included (or any exclusions) in the release.
    > >
    > > Currently, we have around 158 patches submitted to trunk. Based on the
    > scope, we will include (exclude) more if required.
    > > Version Upgrade
    > > Java 1.7 (OOZIE-2036 <https://issues.apache.org/jira/browse/OOZIE-2036>)
    > > OpenJPA (OOZIE-2488 <https://issues.apache.org/jira/browse/OOZIE-2488>),
    > > Curator  (OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231
    > >),
    > > HttpClient  (OOZIE-2538 <
    > https://issues.apache.org/jira/browse/OOZIE-2538>),
    > > Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602>).
    > > New Features/Improvements: Add default JT and NN, Database schema
    > service check, instrumental metrics, Tomcat upgrade etc.
    > > Bug Fixes.
    > > Provide your opinion on inclusion/exclusion of the above points or more
    > for the scope of the release. We can also, take some work which is already
    > done and review is pending for them (depending on the availability of the
    > reviewers).
    > >
    > > Thanks,
    > > Abhishek
    >
    >
    

Re: [SCOPE] Oozie 4.3.0 Release

[Purshotam Shah <pu...@yahoo-inc.com.INVALID>]

+1 for the approach. I think there are some critical JIRA for 4.3 and not all of them are committed. I will mark them for 4.3 release.
Thanks,
 

    On Friday, July 15, 2016 12:06 PM, Rohini Palaniswamy <ro...@yahoo-inc.com> wrote:
 

 It should be fine changing java version in 4.3.  Pig to 0.16 should also be fine. Those two I can vouch with confidence. We were running with Java 7 for about two years and now have been running with Java 8 for more than a year. So jdk upgrade does not need baking time. Openjpa and curator I am fine with deferring. For httpclient, shouldn't we upgrade if it is a security issue?

We need to create version 4.3 and mark all closed ones with trunk as 4.3. Anything in patch available that we are going to incorporate mark for 4.3 and rest leave it as it is. That will make it easy to scrub and scope the release. If anyone specifically comes back asking for things not marked for 4.3, then we can pull them into 4.3. For the ones in Patch Available, we can divide the review work and get all in within 2 weeks.

Regards,Rohini
      From: Robert Kanter <rk...@cloudera.com>
 To: "dev@oozie.apache.org" <de...@oozie.apache.org>; Rohini Palaniswamy <ro...@yahoo-inc.com>; Purshotam Shah <pu...@yahoo-inc.com>; Satish Subhashrao Saley <sa...@yahoo-inc.com> 
 Sent: Friday, July 15, 2016 11:31 AM
 Subject: Re: [SCOPE] Oozie 4.3.0 Release
  
I don't think we have any guidelines defined anywhere about when it's okay to change the minimum Java version, but perhaps we need to wait until the next major release (5.0)?  That said, Hadoop did this in a minor release and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if nobody has an objections; we should just make sure to call that out in our release notes.  
Otherwise, nothing comes to mind for anything we should leave out other than the Oozie on Yarn stuff, which is not currently in trunk anyway.  
I know there's a number of Spark Action JIRAs; we should make sure to include all of these and any pending ones.  @Rohini, @Puru, @Satish, anything you think needs more time to bake that we should hold off on including in the release?  

thanks- Robert
On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <ba...@gmail.com> wrote:

Hi All,

Any suggestions or comments on the scope for the release?

Thanks,
Abhishek

> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <ba...@gmail.com> wrote:
>
> Hi All,
>
> Oozie is starting the release process for Oozie 4.3.0. This thread is for discussion about the scope of the release. Please provide your suggestion about what should be included (or any exclusions) in the release.
>
> Currently, we have around 158 patches submitted to trunk. Based on the scope, we will include (exclude) more if required.
> Version Upgrade
> Java 1.7 (OOZIE-2036 <https://issues.apache.org/jira/browse/OOZIE-2036>)
> OpenJPA (OOZIE-2488 <https://issues.apache.org/jira/browse/OOZIE-2488>),
> Curator  (OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231>),
> HttpClient  (OOZIE-2538 <https://issues.apache.org/jira/browse/OOZIE-2538>),
> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602>).
> New Features/Improvements: Add default JT and NN, Database schema service check, instrumental metrics, Tomcat upgrade etc.
> Bug Fixes.
> Provide your opinion on inclusion/exclusion of the above points or more for the scope of the release. We can also, take some work which is already done and review is pending for them (depending on the availability of the reviewers).
>
> Thanks,
> Abhishek





   

  

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter <rk...@cloudera.com>]

I don't think we have any guidelines defined anywhere about when it's okay
to change the minimum Java version, but perhaps we need to wait until the
next major release (5.0)?  That said, Hadoop did this in a minor release
and Java 6 was EoL in 2013.  So I'm fine with doing it in 4.3 if nobody has
an objections; we should just make sure to call that out in our release
notes.

Otherwise, nothing comes to mind for anything we should leave out other
than the Oozie on Yarn stuff, which is not currently in trunk anyway.

I know there's a number of Spark Action JIRAs; we should make sure to
include all of these and any pending ones.
@Rohini, @Puru, @Satish, anything you think needs more time to bake that we
should hold off on including in the release?


thanks
- Robert

On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <ba...@gmail.com>
wrote:

> Hi All,
>
> Any suggestions or comments on the scope for the release?
>
> Thanks,
> Abhishek
>
> > On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <ba...@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > Oozie is starting the release process for Oozie 4.3.0. This thread is
> for discussion about the scope of the release. Please provide your
> suggestion about what should be included (or any exclusions) in the release.
> >
> > Currently, we have around 158 patches submitted to trunk. Based on the
> scope, we will include (exclude) more if required.
> > Version Upgrade
> > Java 1.7 (OOZIE-2036 <https://issues.apache.org/jira/browse/OOZIE-2036>)
> > OpenJPA (OOZIE-2488 <https://issues.apache.org/jira/browse/OOZIE-2488>),
> > Curator  (OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231
> >),
> > HttpClient  (OOZIE-2538 <
> https://issues.apache.org/jira/browse/OOZIE-2538>),
> > Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602>).
> > New Features/Improvements: Add default JT and NN, Database schema
> service check, instrumental metrics, Tomcat upgrade etc.
> > Bug Fixes.
> > Provide your opinion on inclusion/exclusion of the above points or more
> for the scope of the release. We can also, take some work which is already
> done and review is pending for them (depending on the availability of the
> reviewers).
> >
> > Thanks,
> > Abhishek
>
>

Re: [SCOPE] Oozie 4.3.0 Release

[Abhishek Bafna <ba...@gmail.com>]

Hi All,

Any suggestions or comments on the scope for the release?

Thanks,
Abhishek

> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <ba...@gmail.com> wrote:
> 
> Hi All,
> 
> Oozie is starting the release process for Oozie 4.3.0. This thread is for discussion about the scope of the release. Please provide your suggestion about what should be included (or any exclusions) in the release.
> 
> Currently, we have around 158 patches submitted to trunk. Based on the scope, we will include (exclude) more if required.
> Version Upgrade 
> Java 1.7 (OOZIE-2036 <https://issues.apache.org/jira/browse/OOZIE-2036>)
> OpenJPA (OOZIE-2488 <https://issues.apache.org/jira/browse/OOZIE-2488>), 
> Curator  (OOZIE-2231 <https://issues.apache.org/jira/browse/OOZIE-2231>), 
> HttpClient  (OOZIE-2538 <https://issues.apache.org/jira/browse/OOZIE-2538>), 
> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602>).
> New Features/Improvements: Add default JT and NN, Database schema service check, instrumental metrics, Tomcat upgrade etc.
> Bug Fixes.
> Provide your opinion on inclusion/exclusion of the above points or more for the scope of the release. We can also, take some work which is already done and review is pending for them (depending on the availability of the reviewers).
> 
> Thanks,
> Abhishek