You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@asterixdb.apache.org by Ian Maxon <im...@uci.edu> on 2016/03/30 21:36:45 UTC

The Great Merge

Hi all,
I went ahead preliminarily merged the Hyracks and AsterixDB repositories
into one. Unfortunately this can't be reviewed in Gerrit so you all can
check it out here:
https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks

You will likely have to do some ugly rebasing for whatever changes you
might have open once this gets done, since it moves asterixdb down one
folder and swaps out pom.xml in the repository root. Hyracks is in a
similar situation, though you would want to reapply your change to the
AsterixDB repo from Hyracks (which is a bit odd). If you would like to see
how this affects your branch please do try fetching the branch I linked
above and testing it out on a copy of your topic branch.

I'm still making sure all of the tests pass but nothing's failed so far.
Unless anyone has objections I think we should push this change either this
week or early next week.

Let me know what you all think.

Thanks,
- Ian

Re: [WARNING] Don't push anything into the hyracks git repository

Posted by Mike Carey <dt...@gmail.com>.
PHEW!  Stopped me just in time.  :-)

On 4/7/16 10:24 AM, Till Westmann wrote:
> Jut a forward for people (like myself) that don’t always see which 
> messages are important :)
>
> Cheers,
> Till
>
> Forwarded message:
>
>> From: Ian Maxon <im...@uci.edu>
>> To: dev@asterixdb.incubator.apache.org
>> Subject: Re: The Great Merge
>> Date: Thu, 7 Apr 2016 09:12:40 -0700
>>
>> Also, please don't push any new changes to Hyracks. You will need to 
>> merge
>> these into your asterix change or a new change if it is a hyracks change
>> with no asterixdb content. I have disabled the jenkins job that verifies
>> hyracks patches on Gerrit so hopefully nothing gets through.
>>
>> On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <im...@uci.edu> wrote:
>>
>>> Alright, everything is pushed. Please post here or start another 
>>> thread if
>>> you start to experience issues.
>>>
>>> On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <im...@uci.edu> wrote:
>>>
>>>> Just as a reminder, I'll be merging this tommorow since it seems like
>>>> everything should go fairly smoothly.
>>>>
>>>> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu> 
>>>> wrote:
>>>>
>>>>> http://landbeforetime.wikia.com/wiki/Great_Valley
>>>>>
>>>>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu> 
>>>>> wrote:
>>>>>
>>>>>> No, we are living in the GREAT valley :)
>>>>>> Steven
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com> 
>>>>>> wrote:
>>>>>>
>>>>>>> Sounds like things are GOOD! Excellent.  (So not to be feared like
>>>>> the
>>>>>>> event that the name of this one keeps reminding me of:
>>>>>>> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>>>>>>>
>>>>>>>
>>>>>>> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>>>>>>>
>>>>>>>> It seems that I might be the only one concerned here, but it seems
>>>>> like
>>>>>>>> there should be others, so I am continuing this thread.
>>>>>>>>
>>>>>>>> I modified the perl REGEX from Chris' summer solution, and it 
>>>>>>>> works!
>>>>>>>>
>>>>>>>> Once Ian has merged master:
>>>>>>>>
>>>>>>>> 1. On your local branch, find the *parent* of the first commit you
>>>>> want
>>>>>>>> to
>>>>>>>> migrate onto the new master, e.g.
>>>>>>>> de6e0da24c26037967eb9a937d2c77c6c43e8761
>>>>>>>>
>>>>>>>> 2. Run this magic command:
>>>>>>>>
>>>>>>>>     git format-patch --stdout
>>>>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>>>>>>>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>>>>>>>>
>>>>>>>> 3. Now fetch master, and create a new local branch from it:
>>>>>>>>
>>>>>>>>     git switch master; git pull; git checkout -B newbranch
>>>>>>>>
>>>>>>>> 4. Apply your tweaked patch:
>>>>>>>>
>>>>>>>>     git am /tmp/my.patch
>>>>>>>>
>>>>>>>>
>>>>>>>> This recognized ALL of my file moves/renames and applied them
>>>>> correctly.
>>>>>>>> It
>>>>>>>> leaves only two issues:
>>>>>>>> 1) Something similar will probably need to be done for Hyracks
>>>>> changes
>>>>>>>> 2) My pom changes didn't apply. This isn't so bad since there are
>>>>> only a
>>>>>>>> few pom files total.
>>>>>>>>
>>>>>>>>
>>>>>>>> I hope this helps,
>>>>>>>> Steven
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
>>>>> wrote:
>>>>>>>>
>>>>>>>> Here is Chris's original solution to give context. I think 
>>>>>>>> changing
>>>>> the
>>>>>>>>> REGEX might be enough to re-use the solution:
>>>>>>>>>
>>>>>>>>> 1. On your local branch, find the *parent* of the first commit 
>>>>>>>>> you
>>>>> want
>>>>>>>>> to
>>>>>>>>> migrate onto the new master. If you were fully up-to-date 
>>>>>>>>> before the
>>>>>>>>> repackaging commits went in, this will be Till's
>>>>>>>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that
>>>>> here.
>>>>>>>>>
>>>>>>>>> 2. Run this magic command:
>>>>>>>>>
>>>>>>>>>     git format-patch --stdout
>>>>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>>>>>>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>>>>>>>>
>>>>>>>>> 3. Now fetch the new master, and create a new local branch 
>>>>>>>>> from it:
>>>>>>>>>
>>>>>>>>>     git switch master; git pull; git checkout -B newbranch
>>>>>>>>>
>>>>>>>>> 4. Apply your tweaked patch:
>>>>>>>>>
>>>>>>>>>     git am /tmp/my.patch
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Steven
>>>>>>>>>
>>>>>>>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I've tried doing this now on my branch.
>>>>>>>>>> As I feared, all of the files that are renamed/moved become
>>>>> conflicts
>>>>>>>>>> (just a few hundred conflicts in my case 😑).
>>>>>>>>>> I'm wondering if we could use a similar technique for what we 
>>>>>>>>>> did
>>>>>>>>>> during
>>>>>>>>>> the summer (for the apache change) to get around this.
>>>>>>>>>>
>>>>>>>>>> Steven
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I’m not sure I completely understand what you are saying. Is 
>>>>>>>>>> this a
>>>>>>>>>>> temporary state that will get cleaned up later or is this
>>>>> supposed to
>>>>>>>>>>> stay this way (having "-fullstack" in the names)?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Till
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure if it was necessary to rename it, but the original
>>>>> issue
>>>>>>>>>>> is
>>>>>>>>>>>
>>>>>>>>>>>> that the hyracks repo itself has a folder named hyracks, that
>>>>>>>>>>>> contains
>>>>>>>>>>>> hyracks. I thought this might confuse git if I did 
>>>>>>>>>>>> something like
>>>>>>>>>>>> make a
>>>>>>>>>>>> new temporary folder, move everything into that, and then 
>>>>>>>>>>>> rename
>>>>> it
>>>>>>>>>>>> to
>>>>>>>>>>>> 'hyracks'.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann 
>>>>>>>>>>>> <tillw@apache.org
>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Interesting!
>>>>>>>>>>>>
>>>>>>>>>>>>> One thing I’m wondering about is why you’ve added 
>>>>>>>>>>>>> "-fullstack"
>>>>> to
>>>>>>>>>>>>> the
>>>>>>>>>>>>> artifactId and the hyracks module.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Till
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've gone ahead and tried merging my topic branch with this
>>>>> change,
>>>>>>>>>>>>> and it
>>>>>>>>>>>>>
>>>>>>>>>>>>> turned out surprisingly well. I really didn't have many 
>>>>>>>>>>>>> issues.
>>>>> I'll
>>>>>>>>>>>>>> summarize the process:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) Merge the change from asterixdb with your topic branch
>>>>> checked
>>>>>>>>>>>>>> out, so
>>>>>>>>>>>>>> just 'git merge hyracks-merge2'.
>>>>>>>>>>>>>> The only real conflict should be the pom, if you altered 
>>>>>>>>>>>>>> that.
>>>>> I
>>>>>>>>>>>>>> found it
>>>>>>>>>>>>>> easiest to just replicate my changes and take the upstream,
>>>>> rather
>>>>>>>>>>>>>> than
>>>>>>>>>>>>>> trying anything funny, since usually pom changes are not 
>>>>>>>>>>>>>> major.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git 
>>>>>>>>>>>>>> remote add
>>>>>>>>>>>>>> hyracks-local file:///home/...')
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git 
>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>> hyracks-local/imaxon/hdfs')
>>>>>>>>>>>>>> This also worked pretty well, the only extra hiccup besides
>>>>> the pom
>>>>>>>>>>>>>> was
>>>>>>>>>>>>>> files I had created. Those appeared at the top level again
>>>>> after
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> merge.
>>>>>>>>>>>>>> But, all you have to do is move them back down one folder 
>>>>>>>>>>>>>> into
>>>>>>>>>>>>>> hyracks-fullstack.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That's about it really. I went ahead and pushed this up to
>>>>> github
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>> well
>>>>>>>>>>>>>> so if anyone would like to take a look at the process or 
>>>>>>>>>>>>>> check
>>>>> out
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> branch to see what happened (at least for me), the branch is
>>>>> here:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks 
>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -Ian
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu>
>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Chris found an issue with the way git histories were being
>>>>> handled
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> way I merged things, so I have revised the proposed branch:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2 
>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Basically I was trying to fit everything into one commit,
>>>>> because
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> thought at first that I could submit it to Gerrit that way.
>>>>>>>>>>>>>>> However
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to
>>>>> treat
>>>>>>>>>>>>>>> every new
>>>>>>>>>>>>>>> commit from Hyracks as a new change. Splitting the 
>>>>>>>>>>>>>>> commits of
>>>>> the
>>>>>>>>>>>>>>> repository merge fixes the issue.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> @Till, I think that creating a textual patch would just be
>>>>> more
>>>>>>>>>>>>>>> work. If
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> were to do it that way I would try fetching the Gerrit 
>>>>>>>>>>>>>>> patch,
>>>>> and
>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>> cherry-picking it onto a new branch that has the
>>>>> hyracks+asterix
>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>> the head.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
>>>>> tillw@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To get existing patches in, could we just create a textual
>>>>> patch
>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> from gerrit), apply that with the necessary -p option to 
>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>> checkout of the merged repositories and submit a new 
>>>>>>>>>>>>>>>> review
>>>>> to
>>>>>>>>>>>>>>>> gerrit?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Till
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I went ahead preliminarily merged the Hyracks and 
>>>>>>>>>>>>>>>> AsterixDB
>>>>>>>>>>>>>>>>> repositories
>>>>>>>>>>>>>>>>> into one. Unfortunately this can't be reviewed in 
>>>>>>>>>>>>>>>>> Gerrit so
>>>>> you
>>>>>>>>>>>>>>>>> all can
>>>>>>>>>>>>>>>>> check it out here:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks 
>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
>>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> might have open once this gets done, since it moves
>>>>> asterixdb
>>>>>>>>>>>>>>>>> down
>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>> folder and swaps out pom.xml in the repository root.
>>>>> Hyracks is
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> similar situation, though you would want to reapply your
>>>>> change
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you
>>>>> would
>>>>>>>>>>>>>>>>> like to
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> how this affects your branch please do try fetching the
>>>>> branch I
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> linked
>>>>>>>>>>>>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
>>>>>>>>>>>>>>>>> failed so
>>>>>>>>>>>>>>>>> far.
>>>>>>>>>>>>>>>>> Unless anyone has objections I think we should push this
>>>>> change
>>>>>>>>>>>>>>>>> either
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> week or early next week.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Let me know what you all think.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> - Ian
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>

[WARNING] Don't push anything into the hyracks git repository (was: The Great Merge)

Posted by Till Westmann <ti...@apache.org>.
Jut a forward for people (like myself) that don’t always see which 
messages are important :)

Cheers,
Till

Forwarded message:

> From: Ian Maxon <im...@uci.edu>
> To: dev@asterixdb.incubator.apache.org
> Subject: Re: The Great Merge
> Date: Thu, 7 Apr 2016 09:12:40 -0700
>
> Also, please don't push any new changes to Hyracks. You will need to 
> merge
> these into your asterix change or a new change if it is a hyracks 
> change
> with no asterixdb content. I have disabled the jenkins job that 
> verifies
> hyracks patches on Gerrit so hopefully nothing gets through.
>
> On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <im...@uci.edu> wrote:
>
>> Alright, everything is pushed. Please post here or start another 
>> thread if
>> you start to experience issues.
>>
>> On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <im...@uci.edu> wrote:
>>
>>> Just as a reminder, I'll be merging this tommorow since it seems 
>>> like
>>> everything should go fairly smoothly.
>>>
>>> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu> 
>>> wrote:
>>>
>>>> http://landbeforetime.wikia.com/wiki/Great_Valley
>>>>
>>>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu> 
>>>> wrote:
>>>>
>>>>> No, we are living in the GREAT valley :)
>>>>> Steven
>>>>>
>>>>> On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com> 
>>>>> wrote:
>>>>>
>>>>>> Sounds like things are GOOD!  Excellent.  (So not to be feared 
>>>>>> like
>>>> the
>>>>>> event that the name of this one keeps reminding me of:
>>>>>> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>>>>>>
>>>>>>
>>>>>> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>>>>>>
>>>>>>> It seems that I might be the only one concerned here, but it 
>>>>>>> seems
>>>> like
>>>>>>> there should be others, so I am continuing this thread.
>>>>>>>
>>>>>>> I modified the perl REGEX from Chris' summer solution, and it 
>>>>>>> works!
>>>>>>>
>>>>>>> Once Ian has merged master:
>>>>>>>
>>>>>>> 1. On your local branch, find the *parent* of the first commit 
>>>>>>> you
>>>> want
>>>>>>> to
>>>>>>> migrate onto the new master, e.g.
>>>>>>> de6e0da24c26037967eb9a937d2c77c6c43e8761
>>>>>>>
>>>>>>> 2. Run this magic command:
>>>>>>>
>>>>>>>     git format-patch --stdout
>>>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>>>>>>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>>>>>>>
>>>>>>> 3. Now fetch master, and create a new local branch from it:
>>>>>>>
>>>>>>>     git switch master; git pull; git checkout -B newbranch
>>>>>>>
>>>>>>> 4. Apply your tweaked patch:
>>>>>>>
>>>>>>>     git am /tmp/my.patch
>>>>>>>
>>>>>>>
>>>>>>> This recognized ALL of my file moves/renames and applied them
>>>> correctly.
>>>>>>> It
>>>>>>> leaves only two issues:
>>>>>>> 1) Something similar will probably need to be done for Hyracks
>>>> changes
>>>>>>> 2) My pom changes didn't apply. This isn't so bad since there 
>>>>>>> are
>>>> only a
>>>>>>> few pom files total.
>>>>>>>
>>>>>>>
>>>>>>> I hope this helps,
>>>>>>> Steven
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs 
>>>>>>> <sj...@ucr.edu>
>>>> wrote:
>>>>>>>
>>>>>>> Here is Chris's original solution to give context. I think 
>>>>>>> changing
>>>> the
>>>>>>>> REGEX might be enough to re-use the solution:
>>>>>>>>
>>>>>>>> 1. On your local branch, find the *parent* of the first commit 
>>>>>>>> you
>>>> want
>>>>>>>> to
>>>>>>>> migrate onto the new master. If you were fully up-to-date 
>>>>>>>> before the
>>>>>>>> repackaging commits went in, this will be Till's
>>>>>>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use 
>>>>>>>> that
>>>> here.
>>>>>>>>
>>>>>>>> 2. Run this magic command:
>>>>>>>>
>>>>>>>>     git format-patch --stdout
>>>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>>>>>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>>>>>>>
>>>>>>>> 3. Now fetch the new master, and create a new local branch from 
>>>>>>>> it:
>>>>>>>>
>>>>>>>>     git switch master; git pull; git checkout -B newbranch
>>>>>>>>
>>>>>>>> 4. Apply your tweaked patch:
>>>>>>>>
>>>>>>>>     git am /tmp/my.patch
>>>>>>>>
>>>>>>>>
>>>>>>>> Steven
>>>>>>>>
>>>>>>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs 
>>>>>>>> <sj...@ucr.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> I've tried doing this now on my branch.
>>>>>>>>> As I feared, all of the files that are renamed/moved become
>>>> conflicts
>>>>>>>>> (just a few hundred conflicts in my case 😑).
>>>>>>>>> I'm wondering if we could use a similar technique for what we 
>>>>>>>>> did
>>>>>>>>> during
>>>>>>>>> the summer (for the apache change) to get around this.
>>>>>>>>>
>>>>>>>>> Steven
>>>>>>>>>
>>>>>>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann 
>>>>>>>>> <ti...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I’m not sure I completely understand what you are saying. Is 
>>>>>>>>> this a
>>>>>>>>>> temporary state that will get cleaned up later or is this
>>>> supposed to
>>>>>>>>>> stay this way (having "-fullstack" in the names)?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Till
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>>>>>>>>
>>>>>>>>>> I'm not sure if it was necessary to rename it, but the 
>>>>>>>>>> original
>>>> issue
>>>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>>> that the hyracks repo itself has a folder named hyracks, 
>>>>>>>>>>> that
>>>>>>>>>>> contains
>>>>>>>>>>> hyracks. I thought this might confuse git if I did something 
>>>>>>>>>>> like
>>>>>>>>>>> make a
>>>>>>>>>>> new temporary folder, move everything into that, and then 
>>>>>>>>>>> rename
>>>> it
>>>>>>>>>>> to
>>>>>>>>>>> 'hyracks'.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann 
>>>>>>>>>>> <tillw@apache.org
>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Interesting!
>>>>>>>>>>>
>>>>>>>>>>>> One thing I’m wondering about is why you’ve added 
>>>>>>>>>>>> "-fullstack"
>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>> artifactId and the hyracks module.
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Till
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I've gone ahead and tried merging my topic branch with this
>>>> change,
>>>>>>>>>>>> and it
>>>>>>>>>>>>
>>>>>>>>>>>> turned out surprisingly well. I really didn't have many 
>>>>>>>>>>>> issues.
>>>> I'll
>>>>>>>>>>>>> summarize the process:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) Merge the change from asterixdb with your topic branch
>>>> checked
>>>>>>>>>>>>> out, so
>>>>>>>>>>>>> just 'git merge hyracks-merge2'.
>>>>>>>>>>>>> The only real conflict should be the pom, if you altered 
>>>>>>>>>>>>> that.
>>>> I
>>>>>>>>>>>>> found it
>>>>>>>>>>>>> easiest to just replicate my changes and take the 
>>>>>>>>>>>>> upstream,
>>>> rather
>>>>>>>>>>>>> than
>>>>>>>>>>>>> trying anything funny, since usually pom changes are not 
>>>>>>>>>>>>> major.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git 
>>>>>>>>>>>>> remote add
>>>>>>>>>>>>> hyracks-local file:///home/...')
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git 
>>>>>>>>>>>>> merge
>>>>>>>>>>>>> hyracks-local/imaxon/hdfs')
>>>>>>>>>>>>> This also worked pretty well, the only extra hiccup 
>>>>>>>>>>>>> besides
>>>> the pom
>>>>>>>>>>>>> was
>>>>>>>>>>>>> files I had created. Those appeared at the top level again
>>>> after
>>>>>>>>>>>>> the
>>>>>>>>>>>>> merge.
>>>>>>>>>>>>> But, all you have to do is move them back down one folder 
>>>>>>>>>>>>> into
>>>>>>>>>>>>> hyracks-fullstack.
>>>>>>>>>>>>>
>>>>>>>>>>>>> That's about it really. I went ahead and pushed this up to
>>>> github
>>>>>>>>>>>>> as
>>>>>>>>>>>>> well
>>>>>>>>>>>>> so if anyone would like to take a look at the process or 
>>>>>>>>>>>>> check
>>>> out
>>>>>>>>>>>>> the
>>>>>>>>>>>>> branch to see what happened (at least for me), the branch 
>>>>>>>>>>>>> is
>>>> here:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -Ian
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon 
>>>>>>>>>>>>> <im...@uci.edu>
>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Chris found an issue with the way git histories were being
>>>> handled
>>>>>>>>>>>>> in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>
>>>>>>>>>>>>> way I merged things, so I have revised the proposed 
>>>>>>>>>>>>> branch:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Basically I was trying to fit everything into one commit,
>>>> because
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> thought at first that I could submit it to Gerrit that 
>>>>>>>>>>>>>> way.
>>>>>>>>>>>>>> However
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to
>>>> treat
>>>>>>>>>>>>>> every new
>>>>>>>>>>>>>> commit from Hyracks as a new change. Splitting the 
>>>>>>>>>>>>>> commits of
>>>> the
>>>>>>>>>>>>>> repository merge fixes the issue.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> @Till, I think that creating a textual patch would just 
>>>>>>>>>>>>>> be
>>>> more
>>>>>>>>>>>>>> work. If
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> were to do it that way I would try fetching the Gerrit 
>>>>>>>>>>>>>> patch,
>>>> and
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>> cherry-picking it onto a new branch that has the
>>>> hyracks+asterix
>>>>>>>>>>>>>> master
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>> the head.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
>>>> tillw@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To get existing patches in, could we just create a 
>>>>>>>>>>>>>> textual
>>>> patch
>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> from gerrit), apply that with the necessary -p option to 
>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>> checkout of the merged repositories and submit a new 
>>>>>>>>>>>>>>> review
>>>> to
>>>>>>>>>>>>>>> gerrit?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Till
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I went ahead preliminarily merged the Hyracks and 
>>>>>>>>>>>>>>> AsterixDB
>>>>>>>>>>>>>>>> repositories
>>>>>>>>>>>>>>>> into one. Unfortunately this can't be reviewed in 
>>>>>>>>>>>>>>>> Gerrit so
>>>> you
>>>>>>>>>>>>>>>> all can
>>>>>>>>>>>>>>>> check it out here:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You will likely have to do some ugly rebasing for 
>>>>>>>>>>>>>>> whatever
>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> might have open once this gets done, since it moves
>>>> asterixdb
>>>>>>>>>>>>>>>> down
>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>> folder and swaps out pom.xml in the repository root.
>>>> Hyracks is
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> similar situation, though you would want to reapply 
>>>>>>>>>>>>>>>> your
>>>> change
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If 
>>>>>>>>>>>>>>>> you
>>>> would
>>>>>>>>>>>>>>>> like to
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> how this affects your branch please do try fetching the
>>>> branch I
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> linked
>>>>>>>>>>>>>>>> above and testing it out on a copy of your topic 
>>>>>>>>>>>>>>>> branch.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm still making sure all of the tests pass but 
>>>>>>>>>>>>>>>> nothing's
>>>>>>>>>>>>>>>> failed so
>>>>>>>>>>>>>>>> far.
>>>>>>>>>>>>>>>> Unless anyone has objections I think we should push 
>>>>>>>>>>>>>>>> this
>>>> change
>>>>>>>>>>>>>>>> either
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> week or early next week.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Let me know what you all think.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> - Ian
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>

Re: Re: The Great Merge

Posted by 李文海 <lw...@whu.edu.cn>.
Cool, really great.


> -----Original Messages-----
> From: "Ian Maxon" <im...@uci.edu>
> Sent Time: Friday, April 8, 2016
> To: dev@asterixdb.incubator.apache.org
> Cc: 
> Subject: Re: The Great Merge
> 
> Say if you had an AsterixDB change in a topic branch that had both
> AsterixDB and Hyracks components, it'd be like:
> 
> git checkout (topic branch)
> git fetch origin
> git merge origin/master
> (resolve conflicts)
> git commit
> git remote add hyracks-local file://(absolute path to your hyracks repo)
> git fetch hyracks-local
> git merge hyracks-local/(name of your hyracks topic branch)
> (resolve conflicts)
> 
> 
> On Thu, Apr 7, 2016 at 7:31 PM, Taewoo Kim <wa...@gmail.com> wrote:
> 
> > @Ian:
> >
> > Could you show the process (the explicit command, if possible) step by
> > step? I know you did. But, I have a hard time to apply your merge. Thank
> > you. Steven's patch should work, also. But it didn't work on my branch. It
> > looks like I miss something.
> >
> >
> >
> > Best,
> > Taewoo
> >
> > On Thu, Apr 7, 2016 at 9:12 AM, Ian Maxon <im...@uci.edu> wrote:
> >
> > > Also, please don't push any new changes to Hyracks. You will need to
> > merge
> > > these into your asterix change or a new change if it is a hyracks change
> > > with no asterixdb content. I have disabled the jenkins job that verifies
> > > hyracks patches on Gerrit so hopefully nothing gets through.
> > >
> > > On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <im...@uci.edu> wrote:
> > >
> > > > Alright, everything is pushed. Please post here or start another thread
> > > if
> > > > you start to experience issues.
> > > >
> > > > On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <im...@uci.edu> wrote:
> > > >
> > > >> Just as a reminder, I'll be merging this tommorow since it seems like
> > > >> everything should go fairly smoothly.
> > > >>
> > > >> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu>
> > wrote:
> > > >>
> > > >>> http://landbeforetime.wikia.com/wiki/Great_Valley
> > > >>>
> > > >>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu>
> > > wrote:
> > > >>>
> > > >>> > No, we are living in the GREAT valley :)
> > > >>> > Steven
> > > >>> >
> > > >>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com>
> > > wrote:
> > > >>> >
> > > >>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared
> > like
> > > >>> the
> > > >>> >> event that the name of this one keeps reminding me of:
> > > >>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
> > > >>> >>
> > > >>> >>
> > > >>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
> > > >>> >>
> > > >>> >>> It seems that I might be the only one concerned here, but it
> > seems
> > > >>> like
> > > >>> >>> there should be others, so I am continuing this thread.
> > > >>> >>>
> > > >>> >>> I modified the perl REGEX from Chris' summer solution, and it
> > > works!
> > > >>> >>>
> > > >>> >>> Once Ian has merged master:
> > > >>> >>>
> > > >>> >>> 1. On your local branch, find the *parent* of the first commit
> > you
> > > >>> want
> > > >>> >>> to
> > > >>> >>> migrate onto the new master, e.g.
> > > >>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
> > > >>> >>>
> > > >>> >>> 2. Run this magic command:
> > > >>> >>>
> > > >>> >>>     git format-patch --stdout
> > > >>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> > > >>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> > > >>> >>>
> > > >>> >>> 3. Now fetch master, and create a new local branch from it:
> > > >>> >>>
> > > >>> >>>     git switch master; git pull; git checkout -B newbranch
> > > >>> >>>
> > > >>> >>> 4. Apply your tweaked patch:
> > > >>> >>>
> > > >>> >>>     git am /tmp/my.patch
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> This recognized ALL of my file moves/renames and applied them
> > > >>> correctly.
> > > >>> >>> It
> > > >>> >>> leaves only two issues:
> > > >>> >>> 1) Something similar will probably need to be done for Hyracks
> > > >>> changes
> > > >>> >>> 2) My pom changes didn't apply. This isn't so bad since there are
> > > >>> only a
> > > >>> >>> few pom files total.
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> I hope this helps,
> > > >>> >>> Steven
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sjaco002@ucr.edu
> > >
> > > >>> wrote:
> > > >>> >>>
> > > >>> >>> Here is Chris's original solution to give context. I think
> > changing
> > > >>> the
> > > >>> >>>> REGEX might be enough to re-use the solution:
> > > >>> >>>>
> > > >>> >>>> 1. On your local branch, find the *parent* of the first commit
> > you
> > > >>> want
> > > >>> >>>> to
> > > >>> >>>> migrate onto the new master. If you were fully up-to-date before
> > > the
> > > >>> >>>> repackaging commits went in, this will be Till's
> > > >>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use
> > that
> > > >>> here.
> > > >>> >>>>
> > > >>> >>>> 2. Run this magic command:
> > > >>> >>>>
> > > >>> >>>>     git format-patch --stdout
> > > >>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> > > >>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> > > >>> >>>>
> > > >>> >>>> 3. Now fetch the new master, and create a new local branch from
> > > it:
> > > >>> >>>>
> > > >>> >>>>     git switch master; git pull; git checkout -B newbranch
> > > >>> >>>>
> > > >>> >>>> 4. Apply your tweaked patch:
> > > >>> >>>>
> > > >>> >>>>     git am /tmp/my.patch
> > > >>> >>>>
> > > >>> >>>>
> > > >>> >>>> Steven
> > > >>> >>>>
> > > >>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <
> > sjaco002@ucr.edu>
> > > >>> >>>> wrote:
> > > >>> >>>>
> > > >>> >>>> I've tried doing this now on my branch.
> > > >>> >>>>> As I feared, all of the files that are renamed/moved become
> > > >>> conflicts
> > > >>> >>>>> (just a few hundred conflicts in my case 😑).
> > > >>> >>>>> I'm wondering if we could use a similar technique for what we
> > did
> > > >>> >>>>> during
> > > >>> >>>>> the summer (for the apache change) to get around this.
> > > >>> >>>>>
> > > >>> >>>>> Steven
> > > >>> >>>>>
> > > >>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <
> > tillw@apache.org>
> > > >>> >>>>> wrote:
> > > >>> >>>>>
> > > >>> >>>>> I’m not sure I completely understand what you are saying. Is
> > > this a
> > > >>> >>>>>> temporary state that will get cleaned up later or is this
> > > >>> supposed to
> > > >>> >>>>>> stay this way (having "-fullstack" in the names)?
> > > >>> >>>>>>
> > > >>> >>>>>> Thanks,
> > > >>> >>>>>> Till
> > > >>> >>>>>>
> > > >>> >>>>>>
> > > >>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> > > >>> >>>>>>
> > > >>> >>>>>> I'm not sure if it was necessary to rename it, but the
> > original
> > > >>> issue
> > > >>> >>>>>> is
> > > >>> >>>>>>
> > > >>> >>>>>>> that the hyracks repo itself has a folder named hyracks, that
> > > >>> >>>>>>> contains
> > > >>> >>>>>>> hyracks. I thought this might confuse git if I did something
> > > like
> > > >>> >>>>>>> make a
> > > >>> >>>>>>> new temporary folder, move everything into that, and then
> > > rename
> > > >>> it
> > > >>> >>>>>>> to
> > > >>> >>>>>>> 'hyracks'.
> > > >>> >>>>>>>
> > > >>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <
> > > tillw@apache.org
> > > >>> >
> > > >>> >>>>>>> wrote:
> > > >>> >>>>>>>
> > > >>> >>>>>>> Interesting!
> > > >>> >>>>>>>
> > > >>> >>>>>>>> One thing I’m wondering about is why you’ve added
> > "-fullstack"
> > > >>> to
> > > >>> >>>>>>>> the
> > > >>> >>>>>>>> artifactId and the hyracks module.
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> Cheers,
> > > >>> >>>>>>>> Till
> > > >>> >>>>>>>>
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> I've gone ahead and tried merging my topic branch with this
> > > >>> change,
> > > >>> >>>>>>>> and it
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> turned out surprisingly well. I really didn't have many
> > > issues.
> > > >>> I'll
> > > >>> >>>>>>>>> summarize the process:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 1) Merge the change from asterixdb with your topic branch
> > > >>> checked
> > > >>> >>>>>>>>> out, so
> > > >>> >>>>>>>>> just 'git merge hyracks-merge2'.
> > > >>> >>>>>>>>> The only real conflict should be the pom, if you altered
> > > that.
> > > >>> I
> > > >>> >>>>>>>>> found it
> > > >>> >>>>>>>>> easiest to just replicate my changes and take the upstream,
> > > >>> rather
> > > >>> >>>>>>>>> than
> > > >>> >>>>>>>>> trying anything funny, since usually pom changes are not
> > > major.
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote
> > > add
> > > >>> >>>>>>>>> hyracks-local file:///home/...')
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git
> > > merge
> > > >>> >>>>>>>>> hyracks-local/imaxon/hdfs')
> > > >>> >>>>>>>>> This also worked pretty well, the only extra hiccup besides
> > > >>> the pom
> > > >>> >>>>>>>>> was
> > > >>> >>>>>>>>> files I had created. Those appeared at the top level again
> > > >>> after
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>> merge.
> > > >>> >>>>>>>>> But, all you have to do is move them back down one folder
> > > into
> > > >>> >>>>>>>>> hyracks-fullstack.
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> That's about it really. I went ahead and pushed this up to
> > > >>> github
> > > >>> >>>>>>>>> as
> > > >>> >>>>>>>>> well
> > > >>> >>>>>>>>> so if anyone would like to take a look at the process or
> > > check
> > > >>> out
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>> branch to see what happened (at least for me), the branch
> > is
> > > >>> here:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>>
> > >
> > https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> Thanks,
> > > >>> >>>>>>>>> -Ian
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <imaxon@uci.edu
> > >
> > > >>> wrote:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> Chris found an issue with the way git histories were being
> > > >>> handled
> > > >>> >>>>>>>>> in
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> way I merged things, so I have revised the proposed branch:
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>>
> > > https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> Basically I was trying to fit everything into one commit,
> > > >>> because
> > > >>> >>>>>>>>>> I
> > > >>> >>>>>>>>>> thought at first that I could submit it to Gerrit that
> > way.
> > > >>> >>>>>>>>>> However
> > > >>> >>>>>>>>>> that
> > > >>> >>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to
> > > >>> treat
> > > >>> >>>>>>>>>> every new
> > > >>> >>>>>>>>>> commit from Hyracks as a new change. Splitting the commits
> > > of
> > > >>> the
> > > >>> >>>>>>>>>> repository merge fixes the issue.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> @Till, I think that creating a textual patch would just be
> > > >>> more
> > > >>> >>>>>>>>>> work. If
> > > >>> >>>>>>>>>> I
> > > >>> >>>>>>>>>> were to do it that way I would try fetching the Gerrit
> > > patch,
> > > >>> and
> > > >>> >>>>>>>>>> then
> > > >>> >>>>>>>>>> cherry-picking it onto a new branch that has the
> > > >>> hyracks+asterix
> > > >>> >>>>>>>>>> master
> > > >>> >>>>>>>>>> as
> > > >>> >>>>>>>>>> the head.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
> > > >>> tillw@apache.org>
> > > >>> >>>>>>>>>> wrote:
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> To get existing patches in, could we just create a textual
> > > >>> patch
> > > >>> >>>>>>>>>> (e.g.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> from gerrit), apply that with the necessary -p option to a
> > > new
> > > >>> >>>>>>>>>>> local
> > > >>> >>>>>>>>>>> checkout of the merged repositories and submit a new
> > review
> > > >>> to
> > > >>> >>>>>>>>>>> gerrit?
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> Thanks,
> > > >>> >>>>>>>>>>> Till
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> Hi all,
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and
> > AsterixDB
> > > >>> >>>>>>>>>>>> repositories
> > > >>> >>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit
> > > so
> > > >>> you
> > > >>> >>>>>>>>>>>> all can
> > > >>> >>>>>>>>>>>> check it out here:
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>>
> > >
> > https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> You will likely have to do some ugly rebasing for
> > whatever
> > > >>> >>>>>>>>>>>> changes
> > > >>> >>>>>>>>>>>> you
> > > >>> >>>>>>>>>>>> might have open once this gets done, since it moves
> > > >>> asterixdb
> > > >>> >>>>>>>>>>>> down
> > > >>> >>>>>>>>>>>> one
> > > >>> >>>>>>>>>>>> folder and swaps out pom.xml in the repository root.
> > > >>> Hyracks is
> > > >>> >>>>>>>>>>>> in
> > > >>> >>>>>>>>>>>> a
> > > >>> >>>>>>>>>>>> similar situation, though you would want to reapply your
> > > >>> change
> > > >>> >>>>>>>>>>>> to
> > > >>> >>>>>>>>>>>> the
> > > >>> >>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you
> > > >>> would
> > > >>> >>>>>>>>>>>> like to
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> see
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>> how this affects your branch please do try fetching the
> > > >>> branch I
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>> linked
> > > >>> >>>>>>>>>>>> above and testing it out on a copy of your topic branch.
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> I'm still making sure all of the tests pass but
> > nothing's
> > > >>> >>>>>>>>>>>> failed so
> > > >>> >>>>>>>>>>>> far.
> > > >>> >>>>>>>>>>>> Unless anyone has objections I think we should push this
> > > >>> change
> > > >>> >>>>>>>>>>>> either
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> this
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>> week or early next week.
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>> Let me know what you all think.
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> Thanks,
> > > >>> >>>>>>>>>>>> - Ian
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >

Re: The Great Merge

Posted by Taewoo Kim <wa...@gmail.com>.
Thanks. I will try.

Best,
Taewoo

On Thu, Apr 7, 2016 at 7:58 PM, Ian Maxon <im...@uci.edu> wrote:

> Say if you had an AsterixDB change in a topic branch that had both
> AsterixDB and Hyracks components, it'd be like:
>
> git checkout (topic branch)
> git fetch origin
> git merge origin/master
> (resolve conflicts)
> git commit
> git remote add hyracks-local file://(absolute path to your hyracks repo)
> git fetch hyracks-local
> git merge hyracks-local/(name of your hyracks topic branch)
> (resolve conflicts)
>
>
> On Thu, Apr 7, 2016 at 7:31 PM, Taewoo Kim <wa...@gmail.com> wrote:
>
> > @Ian:
> >
> > Could you show the process (the explicit command, if possible) step by
> > step? I know you did. But, I have a hard time to apply your merge. Thank
> > you. Steven's patch should work, also. But it didn't work on my branch.
> It
> > looks like I miss something.
> >
> >
> >
> > Best,
> > Taewoo
> >
> > On Thu, Apr 7, 2016 at 9:12 AM, Ian Maxon <im...@uci.edu> wrote:
> >
> > > Also, please don't push any new changes to Hyracks. You will need to
> > merge
> > > these into your asterix change or a new change if it is a hyracks
> change
> > > with no asterixdb content. I have disabled the jenkins job that
> verifies
> > > hyracks patches on Gerrit so hopefully nothing gets through.
> > >
> > > On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <im...@uci.edu> wrote:
> > >
> > > > Alright, everything is pushed. Please post here or start another
> thread
> > > if
> > > > you start to experience issues.
> > > >
> > > > On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <im...@uci.edu> wrote:
> > > >
> > > >> Just as a reminder, I'll be merging this tommorow since it seems
> like
> > > >> everything should go fairly smoothly.
> > > >>
> > > >> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu>
> > wrote:
> > > >>
> > > >>> http://landbeforetime.wikia.com/wiki/Great_Valley
> > > >>>
> > > >>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu>
> > > wrote:
> > > >>>
> > > >>> > No, we are living in the GREAT valley :)
> > > >>> > Steven
> > > >>> >
> > > >>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com>
> > > wrote:
> > > >>> >
> > > >>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared
> > like
> > > >>> the
> > > >>> >> event that the name of this one keeps reminding me of:
> > > >>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
> > > >>> >>
> > > >>> >>
> > > >>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
> > > >>> >>
> > > >>> >>> It seems that I might be the only one concerned here, but it
> > seems
> > > >>> like
> > > >>> >>> there should be others, so I am continuing this thread.
> > > >>> >>>
> > > >>> >>> I modified the perl REGEX from Chris' summer solution, and it
> > > works!
> > > >>> >>>
> > > >>> >>> Once Ian has merged master:
> > > >>> >>>
> > > >>> >>> 1. On your local branch, find the *parent* of the first commit
> > you
> > > >>> want
> > > >>> >>> to
> > > >>> >>> migrate onto the new master, e.g.
> > > >>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
> > > >>> >>>
> > > >>> >>> 2. Run this magic command:
> > > >>> >>>
> > > >>> >>>     git format-patch --stdout
> > > >>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> > > >>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> > > >>> >>>
> > > >>> >>> 3. Now fetch master, and create a new local branch from it:
> > > >>> >>>
> > > >>> >>>     git switch master; git pull; git checkout -B newbranch
> > > >>> >>>
> > > >>> >>> 4. Apply your tweaked patch:
> > > >>> >>>
> > > >>> >>>     git am /tmp/my.patch
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> This recognized ALL of my file moves/renames and applied them
> > > >>> correctly.
> > > >>> >>> It
> > > >>> >>> leaves only two issues:
> > > >>> >>> 1) Something similar will probably need to be done for Hyracks
> > > >>> changes
> > > >>> >>> 2) My pom changes didn't apply. This isn't so bad since there
> are
> > > >>> only a
> > > >>> >>> few pom files total.
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> I hope this helps,
> > > >>> >>> Steven
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <
> sjaco002@ucr.edu
> > >
> > > >>> wrote:
> > > >>> >>>
> > > >>> >>> Here is Chris's original solution to give context. I think
> > changing
> > > >>> the
> > > >>> >>>> REGEX might be enough to re-use the solution:
> > > >>> >>>>
> > > >>> >>>> 1. On your local branch, find the *parent* of the first commit
> > you
> > > >>> want
> > > >>> >>>> to
> > > >>> >>>> migrate onto the new master. If you were fully up-to-date
> before
> > > the
> > > >>> >>>> repackaging commits went in, this will be Till's
> > > >>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use
> > that
> > > >>> here.
> > > >>> >>>>
> > > >>> >>>> 2. Run this magic command:
> > > >>> >>>>
> > > >>> >>>>     git format-patch --stdout
> > > >>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> > > >>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> > > >>> >>>>
> > > >>> >>>> 3. Now fetch the new master, and create a new local branch
> from
> > > it:
> > > >>> >>>>
> > > >>> >>>>     git switch master; git pull; git checkout -B newbranch
> > > >>> >>>>
> > > >>> >>>> 4. Apply your tweaked patch:
> > > >>> >>>>
> > > >>> >>>>     git am /tmp/my.patch
> > > >>> >>>>
> > > >>> >>>>
> > > >>> >>>> Steven
> > > >>> >>>>
> > > >>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <
> > sjaco002@ucr.edu>
> > > >>> >>>> wrote:
> > > >>> >>>>
> > > >>> >>>> I've tried doing this now on my branch.
> > > >>> >>>>> As I feared, all of the files that are renamed/moved become
> > > >>> conflicts
> > > >>> >>>>> (just a few hundred conflicts in my case 😑).
> > > >>> >>>>> I'm wondering if we could use a similar technique for what we
> > did
> > > >>> >>>>> during
> > > >>> >>>>> the summer (for the apache change) to get around this.
> > > >>> >>>>>
> > > >>> >>>>> Steven
> > > >>> >>>>>
> > > >>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <
> > tillw@apache.org>
> > > >>> >>>>> wrote:
> > > >>> >>>>>
> > > >>> >>>>> I’m not sure I completely understand what you are saying. Is
> > > this a
> > > >>> >>>>>> temporary state that will get cleaned up later or is this
> > > >>> supposed to
> > > >>> >>>>>> stay this way (having "-fullstack" in the names)?
> > > >>> >>>>>>
> > > >>> >>>>>> Thanks,
> > > >>> >>>>>> Till
> > > >>> >>>>>>
> > > >>> >>>>>>
> > > >>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> > > >>> >>>>>>
> > > >>> >>>>>> I'm not sure if it was necessary to rename it, but the
> > original
> > > >>> issue
> > > >>> >>>>>> is
> > > >>> >>>>>>
> > > >>> >>>>>>> that the hyracks repo itself has a folder named hyracks,
> that
> > > >>> >>>>>>> contains
> > > >>> >>>>>>> hyracks. I thought this might confuse git if I did
> something
> > > like
> > > >>> >>>>>>> make a
> > > >>> >>>>>>> new temporary folder, move everything into that, and then
> > > rename
> > > >>> it
> > > >>> >>>>>>> to
> > > >>> >>>>>>> 'hyracks'.
> > > >>> >>>>>>>
> > > >>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <
> > > tillw@apache.org
> > > >>> >
> > > >>> >>>>>>> wrote:
> > > >>> >>>>>>>
> > > >>> >>>>>>> Interesting!
> > > >>> >>>>>>>
> > > >>> >>>>>>>> One thing I’m wondering about is why you’ve added
> > "-fullstack"
> > > >>> to
> > > >>> >>>>>>>> the
> > > >>> >>>>>>>> artifactId and the hyracks module.
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> Cheers,
> > > >>> >>>>>>>> Till
> > > >>> >>>>>>>>
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> I've gone ahead and tried merging my topic branch with
> this
> > > >>> change,
> > > >>> >>>>>>>> and it
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> turned out surprisingly well. I really didn't have many
> > > issues.
> > > >>> I'll
> > > >>> >>>>>>>>> summarize the process:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 1) Merge the change from asterixdb with your topic branch
> > > >>> checked
> > > >>> >>>>>>>>> out, so
> > > >>> >>>>>>>>> just 'git merge hyracks-merge2'.
> > > >>> >>>>>>>>> The only real conflict should be the pom, if you altered
> > > that.
> > > >>> I
> > > >>> >>>>>>>>> found it
> > > >>> >>>>>>>>> easiest to just replicate my changes and take the
> upstream,
> > > >>> rather
> > > >>> >>>>>>>>> than
> > > >>> >>>>>>>>> trying anything funny, since usually pom changes are not
> > > major.
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git
> remote
> > > add
> > > >>> >>>>>>>>> hyracks-local file:///home/...')
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git
> > > merge
> > > >>> >>>>>>>>> hyracks-local/imaxon/hdfs')
> > > >>> >>>>>>>>> This also worked pretty well, the only extra hiccup
> besides
> > > >>> the pom
> > > >>> >>>>>>>>> was
> > > >>> >>>>>>>>> files I had created. Those appeared at the top level
> again
> > > >>> after
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>> merge.
> > > >>> >>>>>>>>> But, all you have to do is move them back down one folder
> > > into
> > > >>> >>>>>>>>> hyracks-fullstack.
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> That's about it really. I went ahead and pushed this up
> to
> > > >>> github
> > > >>> >>>>>>>>> as
> > > >>> >>>>>>>>> well
> > > >>> >>>>>>>>> so if anyone would like to take a look at the process or
> > > check
> > > >>> out
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>> branch to see what happened (at least for me), the branch
> > is
> > > >>> here:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>>
> > >
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> Thanks,
> > > >>> >>>>>>>>> -Ian
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <
> imaxon@uci.edu
> > >
> > > >>> wrote:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> Chris found an issue with the way git histories were
> being
> > > >>> handled
> > > >>> >>>>>>>>> in
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> way I merged things, so I have revised the proposed
> branch:
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>>
> > >
> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> Basically I was trying to fit everything into one
> commit,
> > > >>> because
> > > >>> >>>>>>>>>> I
> > > >>> >>>>>>>>>> thought at first that I could submit it to Gerrit that
> > way.
> > > >>> >>>>>>>>>> However
> > > >>> >>>>>>>>>> that
> > > >>> >>>>>>>>>> doesn't work for other reasons, basically Gerrit tries
> to
> > > >>> treat
> > > >>> >>>>>>>>>> every new
> > > >>> >>>>>>>>>> commit from Hyracks as a new change. Splitting the
> commits
> > > of
> > > >>> the
> > > >>> >>>>>>>>>> repository merge fixes the issue.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> @Till, I think that creating a textual patch would just
> be
> > > >>> more
> > > >>> >>>>>>>>>> work. If
> > > >>> >>>>>>>>>> I
> > > >>> >>>>>>>>>> were to do it that way I would try fetching the Gerrit
> > > patch,
> > > >>> and
> > > >>> >>>>>>>>>> then
> > > >>> >>>>>>>>>> cherry-picking it onto a new branch that has the
> > > >>> hyracks+asterix
> > > >>> >>>>>>>>>> master
> > > >>> >>>>>>>>>> as
> > > >>> >>>>>>>>>> the head.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
> > > >>> tillw@apache.org>
> > > >>> >>>>>>>>>> wrote:
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> To get existing patches in, could we just create a
> textual
> > > >>> patch
> > > >>> >>>>>>>>>> (e.g.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> from gerrit), apply that with the necessary -p option
> to a
> > > new
> > > >>> >>>>>>>>>>> local
> > > >>> >>>>>>>>>>> checkout of the merged repositories and submit a new
> > review
> > > >>> to
> > > >>> >>>>>>>>>>> gerrit?
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> Thanks,
> > > >>> >>>>>>>>>>> Till
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> Hi all,
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and
> > AsterixDB
> > > >>> >>>>>>>>>>>> repositories
> > > >>> >>>>>>>>>>>> into one. Unfortunately this can't be reviewed in
> Gerrit
> > > so
> > > >>> you
> > > >>> >>>>>>>>>>>> all can
> > > >>> >>>>>>>>>>>> check it out here:
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>>
> > >
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> You will likely have to do some ugly rebasing for
> > whatever
> > > >>> >>>>>>>>>>>> changes
> > > >>> >>>>>>>>>>>> you
> > > >>> >>>>>>>>>>>> might have open once this gets done, since it moves
> > > >>> asterixdb
> > > >>> >>>>>>>>>>>> down
> > > >>> >>>>>>>>>>>> one
> > > >>> >>>>>>>>>>>> folder and swaps out pom.xml in the repository root.
> > > >>> Hyracks is
> > > >>> >>>>>>>>>>>> in
> > > >>> >>>>>>>>>>>> a
> > > >>> >>>>>>>>>>>> similar situation, though you would want to reapply
> your
> > > >>> change
> > > >>> >>>>>>>>>>>> to
> > > >>> >>>>>>>>>>>> the
> > > >>> >>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If
> you
> > > >>> would
> > > >>> >>>>>>>>>>>> like to
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> see
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>> how this affects your branch please do try fetching the
> > > >>> branch I
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>> linked
> > > >>> >>>>>>>>>>>> above and testing it out on a copy of your topic
> branch.
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> I'm still making sure all of the tests pass but
> > nothing's
> > > >>> >>>>>>>>>>>> failed so
> > > >>> >>>>>>>>>>>> far.
> > > >>> >>>>>>>>>>>> Unless anyone has objections I think we should push
> this
> > > >>> change
> > > >>> >>>>>>>>>>>> either
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> this
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>> week or early next week.
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>> Let me know what you all think.
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> Thanks,
> > > >>> >>>>>>>>>>>> - Ian
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>

Re: The Great Merge

Posted by Ian Maxon <im...@uci.edu>.
Say if you had an AsterixDB change in a topic branch that had both
AsterixDB and Hyracks components, it'd be like:

git checkout (topic branch)
git fetch origin
git merge origin/master
(resolve conflicts)
git commit
git remote add hyracks-local file://(absolute path to your hyracks repo)
git fetch hyracks-local
git merge hyracks-local/(name of your hyracks topic branch)
(resolve conflicts)


On Thu, Apr 7, 2016 at 7:31 PM, Taewoo Kim <wa...@gmail.com> wrote:

> @Ian:
>
> Could you show the process (the explicit command, if possible) step by
> step? I know you did. But, I have a hard time to apply your merge. Thank
> you. Steven's patch should work, also. But it didn't work on my branch. It
> looks like I miss something.
>
>
>
> Best,
> Taewoo
>
> On Thu, Apr 7, 2016 at 9:12 AM, Ian Maxon <im...@uci.edu> wrote:
>
> > Also, please don't push any new changes to Hyracks. You will need to
> merge
> > these into your asterix change or a new change if it is a hyracks change
> > with no asterixdb content. I have disabled the jenkins job that verifies
> > hyracks patches on Gerrit so hopefully nothing gets through.
> >
> > On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <im...@uci.edu> wrote:
> >
> > > Alright, everything is pushed. Please post here or start another thread
> > if
> > > you start to experience issues.
> > >
> > > On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <im...@uci.edu> wrote:
> > >
> > >> Just as a reminder, I'll be merging this tommorow since it seems like
> > >> everything should go fairly smoothly.
> > >>
> > >> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> > >>
> > >>> http://landbeforetime.wikia.com/wiki/Great_Valley
> > >>>
> > >>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu>
> > wrote:
> > >>>
> > >>> > No, we are living in the GREAT valley :)
> > >>> > Steven
> > >>> >
> > >>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com>
> > wrote:
> > >>> >
> > >>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared
> like
> > >>> the
> > >>> >> event that the name of this one keeps reminding me of:
> > >>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
> > >>> >>
> > >>> >>
> > >>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
> > >>> >>
> > >>> >>> It seems that I might be the only one concerned here, but it
> seems
> > >>> like
> > >>> >>> there should be others, so I am continuing this thread.
> > >>> >>>
> > >>> >>> I modified the perl REGEX from Chris' summer solution, and it
> > works!
> > >>> >>>
> > >>> >>> Once Ian has merged master:
> > >>> >>>
> > >>> >>> 1. On your local branch, find the *parent* of the first commit
> you
> > >>> want
> > >>> >>> to
> > >>> >>> migrate onto the new master, e.g.
> > >>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
> > >>> >>>
> > >>> >>> 2. Run this magic command:
> > >>> >>>
> > >>> >>>     git format-patch --stdout
> > >>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> > >>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> > >>> >>>
> > >>> >>> 3. Now fetch master, and create a new local branch from it:
> > >>> >>>
> > >>> >>>     git switch master; git pull; git checkout -B newbranch
> > >>> >>>
> > >>> >>> 4. Apply your tweaked patch:
> > >>> >>>
> > >>> >>>     git am /tmp/my.patch
> > >>> >>>
> > >>> >>>
> > >>> >>> This recognized ALL of my file moves/renames and applied them
> > >>> correctly.
> > >>> >>> It
> > >>> >>> leaves only two issues:
> > >>> >>> 1) Something similar will probably need to be done for Hyracks
> > >>> changes
> > >>> >>> 2) My pom changes didn't apply. This isn't so bad since there are
> > >>> only a
> > >>> >>> few pom files total.
> > >>> >>>
> > >>> >>>
> > >>> >>> I hope this helps,
> > >>> >>> Steven
> > >>> >>>
> > >>> >>>
> > >>> >>>
> > >>> >>>
> > >>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sjaco002@ucr.edu
> >
> > >>> wrote:
> > >>> >>>
> > >>> >>> Here is Chris's original solution to give context. I think
> changing
> > >>> the
> > >>> >>>> REGEX might be enough to re-use the solution:
> > >>> >>>>
> > >>> >>>> 1. On your local branch, find the *parent* of the first commit
> you
> > >>> want
> > >>> >>>> to
> > >>> >>>> migrate onto the new master. If you were fully up-to-date before
> > the
> > >>> >>>> repackaging commits went in, this will be Till's
> > >>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use
> that
> > >>> here.
> > >>> >>>>
> > >>> >>>> 2. Run this magic command:
> > >>> >>>>
> > >>> >>>>     git format-patch --stdout
> > >>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> > >>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> > >>> >>>>
> > >>> >>>> 3. Now fetch the new master, and create a new local branch from
> > it:
> > >>> >>>>
> > >>> >>>>     git switch master; git pull; git checkout -B newbranch
> > >>> >>>>
> > >>> >>>> 4. Apply your tweaked patch:
> > >>> >>>>
> > >>> >>>>     git am /tmp/my.patch
> > >>> >>>>
> > >>> >>>>
> > >>> >>>> Steven
> > >>> >>>>
> > >>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <
> sjaco002@ucr.edu>
> > >>> >>>> wrote:
> > >>> >>>>
> > >>> >>>> I've tried doing this now on my branch.
> > >>> >>>>> As I feared, all of the files that are renamed/moved become
> > >>> conflicts
> > >>> >>>>> (just a few hundred conflicts in my case 😑).
> > >>> >>>>> I'm wondering if we could use a similar technique for what we
> did
> > >>> >>>>> during
> > >>> >>>>> the summer (for the apache change) to get around this.
> > >>> >>>>>
> > >>> >>>>> Steven
> > >>> >>>>>
> > >>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <
> tillw@apache.org>
> > >>> >>>>> wrote:
> > >>> >>>>>
> > >>> >>>>> I’m not sure I completely understand what you are saying. Is
> > this a
> > >>> >>>>>> temporary state that will get cleaned up later or is this
> > >>> supposed to
> > >>> >>>>>> stay this way (having "-fullstack" in the names)?
> > >>> >>>>>>
> > >>> >>>>>> Thanks,
> > >>> >>>>>> Till
> > >>> >>>>>>
> > >>> >>>>>>
> > >>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> > >>> >>>>>>
> > >>> >>>>>> I'm not sure if it was necessary to rename it, but the
> original
> > >>> issue
> > >>> >>>>>> is
> > >>> >>>>>>
> > >>> >>>>>>> that the hyracks repo itself has a folder named hyracks, that
> > >>> >>>>>>> contains
> > >>> >>>>>>> hyracks. I thought this might confuse git if I did something
> > like
> > >>> >>>>>>> make a
> > >>> >>>>>>> new temporary folder, move everything into that, and then
> > rename
> > >>> it
> > >>> >>>>>>> to
> > >>> >>>>>>> 'hyracks'.
> > >>> >>>>>>>
> > >>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <
> > tillw@apache.org
> > >>> >
> > >>> >>>>>>> wrote:
> > >>> >>>>>>>
> > >>> >>>>>>> Interesting!
> > >>> >>>>>>>
> > >>> >>>>>>>> One thing I’m wondering about is why you’ve added
> "-fullstack"
> > >>> to
> > >>> >>>>>>>> the
> > >>> >>>>>>>> artifactId and the hyracks module.
> > >>> >>>>>>>>
> > >>> >>>>>>>> Cheers,
> > >>> >>>>>>>> Till
> > >>> >>>>>>>>
> > >>> >>>>>>>>
> > >>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> > >>> >>>>>>>>
> > >>> >>>>>>>> I've gone ahead and tried merging my topic branch with this
> > >>> change,
> > >>> >>>>>>>> and it
> > >>> >>>>>>>>
> > >>> >>>>>>>> turned out surprisingly well. I really didn't have many
> > issues.
> > >>> I'll
> > >>> >>>>>>>>> summarize the process:
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> 1) Merge the change from asterixdb with your topic branch
> > >>> checked
> > >>> >>>>>>>>> out, so
> > >>> >>>>>>>>> just 'git merge hyracks-merge2'.
> > >>> >>>>>>>>> The only real conflict should be the pom, if you altered
> > that.
> > >>> I
> > >>> >>>>>>>>> found it
> > >>> >>>>>>>>> easiest to just replicate my changes and take the upstream,
> > >>> rather
> > >>> >>>>>>>>> than
> > >>> >>>>>>>>> trying anything funny, since usually pom changes are not
> > major.
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote
> > add
> > >>> >>>>>>>>> hyracks-local file:///home/...')
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git
> > merge
> > >>> >>>>>>>>> hyracks-local/imaxon/hdfs')
> > >>> >>>>>>>>> This also worked pretty well, the only extra hiccup besides
> > >>> the pom
> > >>> >>>>>>>>> was
> > >>> >>>>>>>>> files I had created. Those appeared at the top level again
> > >>> after
> > >>> >>>>>>>>> the
> > >>> >>>>>>>>> merge.
> > >>> >>>>>>>>> But, all you have to do is move them back down one folder
> > into
> > >>> >>>>>>>>> hyracks-fullstack.
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> That's about it really. I went ahead and pushed this up to
> > >>> github
> > >>> >>>>>>>>> as
> > >>> >>>>>>>>> well
> > >>> >>>>>>>>> so if anyone would like to take a look at the process or
> > check
> > >>> out
> > >>> >>>>>>>>> the
> > >>> >>>>>>>>> branch to see what happened (at least for me), the branch
> is
> > >>> here:
> > >>> >>>>>>>>>
> > >>> >>>>>>>>>
> > >>> >>>>>>>>>
> > >>> >>>>>>>>>
> > >>>
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> Thanks,
> > >>> >>>>>>>>> -Ian
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <imaxon@uci.edu
> >
> > >>> wrote:
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> Chris found an issue with the way git histories were being
> > >>> handled
> > >>> >>>>>>>>> in
> > >>> >>>>>>>>> the
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> way I merged things, so I have revised the proposed branch:
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>>
> > >>>
> > https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> Basically I was trying to fit everything into one commit,
> > >>> because
> > >>> >>>>>>>>>> I
> > >>> >>>>>>>>>> thought at first that I could submit it to Gerrit that
> way.
> > >>> >>>>>>>>>> However
> > >>> >>>>>>>>>> that
> > >>> >>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to
> > >>> treat
> > >>> >>>>>>>>>> every new
> > >>> >>>>>>>>>> commit from Hyracks as a new change. Splitting the commits
> > of
> > >>> the
> > >>> >>>>>>>>>> repository merge fixes the issue.
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> @Till, I think that creating a textual patch would just be
> > >>> more
> > >>> >>>>>>>>>> work. If
> > >>> >>>>>>>>>> I
> > >>> >>>>>>>>>> were to do it that way I would try fetching the Gerrit
> > patch,
> > >>> and
> > >>> >>>>>>>>>> then
> > >>> >>>>>>>>>> cherry-picking it onto a new branch that has the
> > >>> hyracks+asterix
> > >>> >>>>>>>>>> master
> > >>> >>>>>>>>>> as
> > >>> >>>>>>>>>> the head.
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
> > >>> tillw@apache.org>
> > >>> >>>>>>>>>> wrote:
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> To get existing patches in, could we just create a textual
> > >>> patch
> > >>> >>>>>>>>>> (e.g.
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> from gerrit), apply that with the necessary -p option to a
> > new
> > >>> >>>>>>>>>>> local
> > >>> >>>>>>>>>>> checkout of the merged repositories and submit a new
> review
> > >>> to
> > >>> >>>>>>>>>>> gerrit?
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> Thanks,
> > >>> >>>>>>>>>>> Till
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> Hi all,
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and
> AsterixDB
> > >>> >>>>>>>>>>>> repositories
> > >>> >>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit
> > so
> > >>> you
> > >>> >>>>>>>>>>>> all can
> > >>> >>>>>>>>>>>> check it out here:
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>
> > >>>
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> You will likely have to do some ugly rebasing for
> whatever
> > >>> >>>>>>>>>>>> changes
> > >>> >>>>>>>>>>>> you
> > >>> >>>>>>>>>>>> might have open once this gets done, since it moves
> > >>> asterixdb
> > >>> >>>>>>>>>>>> down
> > >>> >>>>>>>>>>>> one
> > >>> >>>>>>>>>>>> folder and swaps out pom.xml in the repository root.
> > >>> Hyracks is
> > >>> >>>>>>>>>>>> in
> > >>> >>>>>>>>>>>> a
> > >>> >>>>>>>>>>>> similar situation, though you would want to reapply your
> > >>> change
> > >>> >>>>>>>>>>>> to
> > >>> >>>>>>>>>>>> the
> > >>> >>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you
> > >>> would
> > >>> >>>>>>>>>>>> like to
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>> see
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>> how this affects your branch please do try fetching the
> > >>> branch I
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>>> linked
> > >>> >>>>>>>>>>>> above and testing it out on a copy of your topic branch.
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>> I'm still making sure all of the tests pass but
> nothing's
> > >>> >>>>>>>>>>>> failed so
> > >>> >>>>>>>>>>>> far.
> > >>> >>>>>>>>>>>> Unless anyone has objections I think we should push this
> > >>> change
> > >>> >>>>>>>>>>>> either
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>> this
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>> week or early next week.
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>>> Let me know what you all think.
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>> Thanks,
> > >>> >>>>>>>>>>>> - Ian
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>
> > >>> >>
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Re: The Great Merge

Posted by Taewoo Kim <wa...@gmail.com>.
@Ian:

Could you show the process (the explicit command, if possible) step by
step? I know you did. But, I have a hard time to apply your merge. Thank
you. Steven's patch should work, also. But it didn't work on my branch. It
looks like I miss something.



Best,
Taewoo

On Thu, Apr 7, 2016 at 9:12 AM, Ian Maxon <im...@uci.edu> wrote:

> Also, please don't push any new changes to Hyracks. You will need to merge
> these into your asterix change or a new change if it is a hyracks change
> with no asterixdb content. I have disabled the jenkins job that verifies
> hyracks patches on Gerrit so hopefully nothing gets through.
>
> On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <im...@uci.edu> wrote:
>
> > Alright, everything is pushed. Please post here or start another thread
> if
> > you start to experience issues.
> >
> > On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <im...@uci.edu> wrote:
> >
> >> Just as a reminder, I'll be merging this tommorow since it seems like
> >> everything should go fairly smoothly.
> >>
> >> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu> wrote:
> >>
> >>> http://landbeforetime.wikia.com/wiki/Great_Valley
> >>>
> >>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> >>>
> >>> > No, we are living in the GREAT valley :)
> >>> > Steven
> >>> >
> >>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com>
> wrote:
> >>> >
> >>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared like
> >>> the
> >>> >> event that the name of this one keeps reminding me of:
> >>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
> >>> >>
> >>> >>
> >>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
> >>> >>
> >>> >>> It seems that I might be the only one concerned here, but it seems
> >>> like
> >>> >>> there should be others, so I am continuing this thread.
> >>> >>>
> >>> >>> I modified the perl REGEX from Chris' summer solution, and it
> works!
> >>> >>>
> >>> >>> Once Ian has merged master:
> >>> >>>
> >>> >>> 1. On your local branch, find the *parent* of the first commit you
> >>> want
> >>> >>> to
> >>> >>> migrate onto the new master, e.g.
> >>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
> >>> >>>
> >>> >>> 2. Run this magic command:
> >>> >>>
> >>> >>>     git format-patch --stdout
> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> >>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> >>> >>>
> >>> >>> 3. Now fetch master, and create a new local branch from it:
> >>> >>>
> >>> >>>     git switch master; git pull; git checkout -B newbranch
> >>> >>>
> >>> >>> 4. Apply your tweaked patch:
> >>> >>>
> >>> >>>     git am /tmp/my.patch
> >>> >>>
> >>> >>>
> >>> >>> This recognized ALL of my file moves/renames and applied them
> >>> correctly.
> >>> >>> It
> >>> >>> leaves only two issues:
> >>> >>> 1) Something similar will probably need to be done for Hyracks
> >>> changes
> >>> >>> 2) My pom changes didn't apply. This isn't so bad since there are
> >>> only a
> >>> >>> few pom files total.
> >>> >>>
> >>> >>>
> >>> >>> I hope this helps,
> >>> >>> Steven
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
> >>> wrote:
> >>> >>>
> >>> >>> Here is Chris's original solution to give context. I think changing
> >>> the
> >>> >>>> REGEX might be enough to re-use the solution:
> >>> >>>>
> >>> >>>> 1. On your local branch, find the *parent* of the first commit you
> >>> want
> >>> >>>> to
> >>> >>>> migrate onto the new master. If you were fully up-to-date before
> the
> >>> >>>> repackaging commits went in, this will be Till's
> >>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that
> >>> here.
> >>> >>>>
> >>> >>>> 2. Run this magic command:
> >>> >>>>
> >>> >>>>     git format-patch --stdout
> >>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> >>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> >>> >>>>
> >>> >>>> 3. Now fetch the new master, and create a new local branch from
> it:
> >>> >>>>
> >>> >>>>     git switch master; git pull; git checkout -B newbranch
> >>> >>>>
> >>> >>>> 4. Apply your tweaked patch:
> >>> >>>>
> >>> >>>>     git am /tmp/my.patch
> >>> >>>>
> >>> >>>>
> >>> >>>> Steven
> >>> >>>>
> >>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
> >>> >>>> wrote:
> >>> >>>>
> >>> >>>> I've tried doing this now on my branch.
> >>> >>>>> As I feared, all of the files that are renamed/moved become
> >>> conflicts
> >>> >>>>> (just a few hundred conflicts in my case 😑).
> >>> >>>>> I'm wondering if we could use a similar technique for what we did
> >>> >>>>> during
> >>> >>>>> the summer (for the apache change) to get around this.
> >>> >>>>>
> >>> >>>>> Steven
> >>> >>>>>
> >>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
> >>> >>>>> wrote:
> >>> >>>>>
> >>> >>>>> I’m not sure I completely understand what you are saying. Is
> this a
> >>> >>>>>> temporary state that will get cleaned up later or is this
> >>> supposed to
> >>> >>>>>> stay this way (having "-fullstack" in the names)?
> >>> >>>>>>
> >>> >>>>>> Thanks,
> >>> >>>>>> Till
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> >>> >>>>>>
> >>> >>>>>> I'm not sure if it was necessary to rename it, but the original
> >>> issue
> >>> >>>>>> is
> >>> >>>>>>
> >>> >>>>>>> that the hyracks repo itself has a folder named hyracks, that
> >>> >>>>>>> contains
> >>> >>>>>>> hyracks. I thought this might confuse git if I did something
> like
> >>> >>>>>>> make a
> >>> >>>>>>> new temporary folder, move everything into that, and then
> rename
> >>> it
> >>> >>>>>>> to
> >>> >>>>>>> 'hyracks'.
> >>> >>>>>>>
> >>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <
> tillw@apache.org
> >>> >
> >>> >>>>>>> wrote:
> >>> >>>>>>>
> >>> >>>>>>> Interesting!
> >>> >>>>>>>
> >>> >>>>>>>> One thing I’m wondering about is why you’ve added "-fullstack"
> >>> to
> >>> >>>>>>>> the
> >>> >>>>>>>> artifactId and the hyracks module.
> >>> >>>>>>>>
> >>> >>>>>>>> Cheers,
> >>> >>>>>>>> Till
> >>> >>>>>>>>
> >>> >>>>>>>>
> >>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> >>> >>>>>>>>
> >>> >>>>>>>> I've gone ahead and tried merging my topic branch with this
> >>> change,
> >>> >>>>>>>> and it
> >>> >>>>>>>>
> >>> >>>>>>>> turned out surprisingly well. I really didn't have many
> issues.
> >>> I'll
> >>> >>>>>>>>> summarize the process:
> >>> >>>>>>>>>
> >>> >>>>>>>>> 1) Merge the change from asterixdb with your topic branch
> >>> checked
> >>> >>>>>>>>> out, so
> >>> >>>>>>>>> just 'git merge hyracks-merge2'.
> >>> >>>>>>>>> The only real conflict should be the pom, if you altered
> that.
> >>> I
> >>> >>>>>>>>> found it
> >>> >>>>>>>>> easiest to just replicate my changes and take the upstream,
> >>> rather
> >>> >>>>>>>>> than
> >>> >>>>>>>>> trying anything funny, since usually pom changes are not
> major.
> >>> >>>>>>>>>
> >>> >>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote
> add
> >>> >>>>>>>>> hyracks-local file:///home/...')
> >>> >>>>>>>>>
> >>> >>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git
> merge
> >>> >>>>>>>>> hyracks-local/imaxon/hdfs')
> >>> >>>>>>>>> This also worked pretty well, the only extra hiccup besides
> >>> the pom
> >>> >>>>>>>>> was
> >>> >>>>>>>>> files I had created. Those appeared at the top level again
> >>> after
> >>> >>>>>>>>> the
> >>> >>>>>>>>> merge.
> >>> >>>>>>>>> But, all you have to do is move them back down one folder
> into
> >>> >>>>>>>>> hyracks-fullstack.
> >>> >>>>>>>>>
> >>> >>>>>>>>> That's about it really. I went ahead and pushed this up to
> >>> github
> >>> >>>>>>>>> as
> >>> >>>>>>>>> well
> >>> >>>>>>>>> so if anyone would like to take a look at the process or
> check
> >>> out
> >>> >>>>>>>>> the
> >>> >>>>>>>>> branch to see what happened (at least for me), the branch is
> >>> here:
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>> >>>>>>>>>
> >>>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> >>> >>>>>>>>>
> >>> >>>>>>>>> Thanks,
> >>> >>>>>>>>> -Ian
> >>> >>>>>>>>>
> >>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu>
> >>> wrote:
> >>> >>>>>>>>>
> >>> >>>>>>>>> Chris found an issue with the way git histories were being
> >>> handled
> >>> >>>>>>>>> in
> >>> >>>>>>>>> the
> >>> >>>>>>>>>
> >>> >>>>>>>>> way I merged things, so I have revised the proposed branch:
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>
> >>>
> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> Basically I was trying to fit everything into one commit,
> >>> because
> >>> >>>>>>>>>> I
> >>> >>>>>>>>>> thought at first that I could submit it to Gerrit that way.
> >>> >>>>>>>>>> However
> >>> >>>>>>>>>> that
> >>> >>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to
> >>> treat
> >>> >>>>>>>>>> every new
> >>> >>>>>>>>>> commit from Hyracks as a new change. Splitting the commits
> of
> >>> the
> >>> >>>>>>>>>> repository merge fixes the issue.
> >>> >>>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> @Till, I think that creating a textual patch would just be
> >>> more
> >>> >>>>>>>>>> work. If
> >>> >>>>>>>>>> I
> >>> >>>>>>>>>> were to do it that way I would try fetching the Gerrit
> patch,
> >>> and
> >>> >>>>>>>>>> then
> >>> >>>>>>>>>> cherry-picking it onto a new branch that has the
> >>> hyracks+asterix
> >>> >>>>>>>>>> master
> >>> >>>>>>>>>> as
> >>> >>>>>>>>>> the head.
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
> >>> tillw@apache.org>
> >>> >>>>>>>>>> wrote:
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> To get existing patches in, could we just create a textual
> >>> patch
> >>> >>>>>>>>>> (e.g.
> >>> >>>>>>>>>>
> >>> >>>>>>>>>> from gerrit), apply that with the necessary -p option to a
> new
> >>> >>>>>>>>>>> local
> >>> >>>>>>>>>>> checkout of the merged repositories and submit a new review
> >>> to
> >>> >>>>>>>>>>> gerrit?
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> Thanks,
> >>> >>>>>>>>>>> Till
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> Hi all,
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
> >>> >>>>>>>>>>>> repositories
> >>> >>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit
> so
> >>> you
> >>> >>>>>>>>>>>> all can
> >>> >>>>>>>>>>>> check it out here:
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
> >>> >>>>>>>>>>>> changes
> >>> >>>>>>>>>>>> you
> >>> >>>>>>>>>>>> might have open once this gets done, since it moves
> >>> asterixdb
> >>> >>>>>>>>>>>> down
> >>> >>>>>>>>>>>> one
> >>> >>>>>>>>>>>> folder and swaps out pom.xml in the repository root.
> >>> Hyracks is
> >>> >>>>>>>>>>>> in
> >>> >>>>>>>>>>>> a
> >>> >>>>>>>>>>>> similar situation, though you would want to reapply your
> >>> change
> >>> >>>>>>>>>>>> to
> >>> >>>>>>>>>>>> the
> >>> >>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you
> >>> would
> >>> >>>>>>>>>>>> like to
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> see
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> how this affects your branch please do try fetching the
> >>> branch I
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>> linked
> >>> >>>>>>>>>>>> above and testing it out on a copy of your topic branch.
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
> >>> >>>>>>>>>>>> failed so
> >>> >>>>>>>>>>>> far.
> >>> >>>>>>>>>>>> Unless anyone has objections I think we should push this
> >>> change
> >>> >>>>>>>>>>>> either
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> this
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>> week or early next week.
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>>> Let me know what you all think.
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>> Thanks,
> >>> >>>>>>>>>>>> - Ian
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>>
> >>> >>>>>>>>>>>
> >>> >>>>>>>>>>
> >>> >>
> >>> >
> >>>
> >>
> >>
> >
>

Re: The Great Merge

Posted by Ian Maxon <im...@uci.edu>.
Also, please don't push any new changes to Hyracks. You will need to merge
these into your asterix change or a new change if it is a hyracks change
with no asterixdb content. I have disabled the jenkins job that verifies
hyracks patches on Gerrit so hopefully nothing gets through.

On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <im...@uci.edu> wrote:

> Alright, everything is pushed. Please post here or start another thread if
> you start to experience issues.
>
> On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <im...@uci.edu> wrote:
>
>> Just as a reminder, I'll be merging this tommorow since it seems like
>> everything should go fairly smoothly.
>>
>> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu> wrote:
>>
>>> http://landbeforetime.wikia.com/wiki/Great_Valley
>>>
>>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu> wrote:
>>>
>>> > No, we are living in the GREAT valley :)
>>> > Steven
>>> >
>>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com> wrote:
>>> >
>>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared like
>>> the
>>> >> event that the name of this one keeps reminding me of:
>>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>>> >>
>>> >>
>>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>>> >>
>>> >>> It seems that I might be the only one concerned here, but it seems
>>> like
>>> >>> there should be others, so I am continuing this thread.
>>> >>>
>>> >>> I modified the perl REGEX from Chris' summer solution, and it works!
>>> >>>
>>> >>> Once Ian has merged master:
>>> >>>
>>> >>> 1. On your local branch, find the *parent* of the first commit you
>>> want
>>> >>> to
>>> >>> migrate onto the new master, e.g.
>>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
>>> >>>
>>> >>> 2. Run this magic command:
>>> >>>
>>> >>>     git format-patch --stdout
>>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>>> >>>
>>> >>> 3. Now fetch master, and create a new local branch from it:
>>> >>>
>>> >>>     git switch master; git pull; git checkout -B newbranch
>>> >>>
>>> >>> 4. Apply your tweaked patch:
>>> >>>
>>> >>>     git am /tmp/my.patch
>>> >>>
>>> >>>
>>> >>> This recognized ALL of my file moves/renames and applied them
>>> correctly.
>>> >>> It
>>> >>> leaves only two issues:
>>> >>> 1) Something similar will probably need to be done for Hyracks
>>> changes
>>> >>> 2) My pom changes didn't apply. This isn't so bad since there are
>>> only a
>>> >>> few pom files total.
>>> >>>
>>> >>>
>>> >>> I hope this helps,
>>> >>> Steven
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
>>> wrote:
>>> >>>
>>> >>> Here is Chris's original solution to give context. I think changing
>>> the
>>> >>>> REGEX might be enough to re-use the solution:
>>> >>>>
>>> >>>> 1. On your local branch, find the *parent* of the first commit you
>>> want
>>> >>>> to
>>> >>>> migrate onto the new master. If you were fully up-to-date before the
>>> >>>> repackaging commits went in, this will be Till's
>>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that
>>> here.
>>> >>>>
>>> >>>> 2. Run this magic command:
>>> >>>>
>>> >>>>     git format-patch --stdout
>>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>> >>>>
>>> >>>> 3. Now fetch the new master, and create a new local branch from it:
>>> >>>>
>>> >>>>     git switch master; git pull; git checkout -B newbranch
>>> >>>>
>>> >>>> 4. Apply your tweaked patch:
>>> >>>>
>>> >>>>     git am /tmp/my.patch
>>> >>>>
>>> >>>>
>>> >>>> Steven
>>> >>>>
>>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
>>> >>>> wrote:
>>> >>>>
>>> >>>> I've tried doing this now on my branch.
>>> >>>>> As I feared, all of the files that are renamed/moved become
>>> conflicts
>>> >>>>> (just a few hundred conflicts in my case 😑).
>>> >>>>> I'm wondering if we could use a similar technique for what we did
>>> >>>>> during
>>> >>>>> the summer (for the apache change) to get around this.
>>> >>>>>
>>> >>>>> Steven
>>> >>>>>
>>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
>>> >>>>> wrote:
>>> >>>>>
>>> >>>>> I’m not sure I completely understand what you are saying. Is this a
>>> >>>>>> temporary state that will get cleaned up later or is this
>>> supposed to
>>> >>>>>> stay this way (having "-fullstack" in the names)?
>>> >>>>>>
>>> >>>>>> Thanks,
>>> >>>>>> Till
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>> >>>>>>
>>> >>>>>> I'm not sure if it was necessary to rename it, but the original
>>> issue
>>> >>>>>> is
>>> >>>>>>
>>> >>>>>>> that the hyracks repo itself has a folder named hyracks, that
>>> >>>>>>> contains
>>> >>>>>>> hyracks. I thought this might confuse git if I did something like
>>> >>>>>>> make a
>>> >>>>>>> new temporary folder, move everything into that, and then rename
>>> it
>>> >>>>>>> to
>>> >>>>>>> 'hyracks'.
>>> >>>>>>>
>>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <tillw@apache.org
>>> >
>>> >>>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Interesting!
>>> >>>>>>>
>>> >>>>>>>> One thing I’m wondering about is why you’ve added "-fullstack"
>>> to
>>> >>>>>>>> the
>>> >>>>>>>> artifactId and the hyracks module.
>>> >>>>>>>>
>>> >>>>>>>> Cheers,
>>> >>>>>>>> Till
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>> >>>>>>>>
>>> >>>>>>>> I've gone ahead and tried merging my topic branch with this
>>> change,
>>> >>>>>>>> and it
>>> >>>>>>>>
>>> >>>>>>>> turned out surprisingly well. I really didn't have many issues.
>>> I'll
>>> >>>>>>>>> summarize the process:
>>> >>>>>>>>>
>>> >>>>>>>>> 1) Merge the change from asterixdb with your topic branch
>>> checked
>>> >>>>>>>>> out, so
>>> >>>>>>>>> just 'git merge hyracks-merge2'.
>>> >>>>>>>>> The only real conflict should be the pom, if you altered that.
>>> I
>>> >>>>>>>>> found it
>>> >>>>>>>>> easiest to just replicate my changes and take the upstream,
>>> rather
>>> >>>>>>>>> than
>>> >>>>>>>>> trying anything funny, since usually pom changes are not major.
>>> >>>>>>>>>
>>> >>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>> >>>>>>>>> hyracks-local file:///home/...')
>>> >>>>>>>>>
>>> >>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>> >>>>>>>>> hyracks-local/imaxon/hdfs')
>>> >>>>>>>>> This also worked pretty well, the only extra hiccup besides
>>> the pom
>>> >>>>>>>>> was
>>> >>>>>>>>> files I had created. Those appeared at the top level again
>>> after
>>> >>>>>>>>> the
>>> >>>>>>>>> merge.
>>> >>>>>>>>> But, all you have to do is move them back down one folder into
>>> >>>>>>>>> hyracks-fullstack.
>>> >>>>>>>>>
>>> >>>>>>>>> That's about it really. I went ahead and pushed this up to
>>> github
>>> >>>>>>>>> as
>>> >>>>>>>>> well
>>> >>>>>>>>> so if anyone would like to take a look at the process or check
>>> out
>>> >>>>>>>>> the
>>> >>>>>>>>> branch to see what happened (at least for me), the branch is
>>> here:
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks,
>>> >>>>>>>>> -Ian
>>> >>>>>>>>>
>>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu>
>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>> Chris found an issue with the way git histories were being
>>> handled
>>> >>>>>>>>> in
>>> >>>>>>>>> the
>>> >>>>>>>>>
>>> >>>>>>>>> way I merged things, so I have revised the proposed branch:
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>> >>>>>>>>>>
>>> >>>>>>>>>> Basically I was trying to fit everything into one commit,
>>> because
>>> >>>>>>>>>> I
>>> >>>>>>>>>> thought at first that I could submit it to Gerrit that way.
>>> >>>>>>>>>> However
>>> >>>>>>>>>> that
>>> >>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to
>>> treat
>>> >>>>>>>>>> every new
>>> >>>>>>>>>> commit from Hyracks as a new change. Splitting the commits of
>>> the
>>> >>>>>>>>>> repository merge fixes the issue.
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> @Till, I think that creating a textual patch would just be
>>> more
>>> >>>>>>>>>> work. If
>>> >>>>>>>>>> I
>>> >>>>>>>>>> were to do it that way I would try fetching the Gerrit patch,
>>> and
>>> >>>>>>>>>> then
>>> >>>>>>>>>> cherry-picking it onto a new branch that has the
>>> hyracks+asterix
>>> >>>>>>>>>> master
>>> >>>>>>>>>> as
>>> >>>>>>>>>> the head.
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
>>> tillw@apache.org>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> To get existing patches in, could we just create a textual
>>> patch
>>> >>>>>>>>>> (e.g.
>>> >>>>>>>>>>
>>> >>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
>>> >>>>>>>>>>> local
>>> >>>>>>>>>>> checkout of the merged repositories and submit a new review
>>> to
>>> >>>>>>>>>>> gerrit?
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Thanks,
>>> >>>>>>>>>>> Till
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Hi all,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>> >>>>>>>>>>>> repositories
>>> >>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so
>>> you
>>> >>>>>>>>>>>> all can
>>> >>>>>>>>>>>> check it out here:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
>>> >>>>>>>>>>>> changes
>>> >>>>>>>>>>>> you
>>> >>>>>>>>>>>> might have open once this gets done, since it moves
>>> asterixdb
>>> >>>>>>>>>>>> down
>>> >>>>>>>>>>>> one
>>> >>>>>>>>>>>> folder and swaps out pom.xml in the repository root.
>>> Hyracks is
>>> >>>>>>>>>>>> in
>>> >>>>>>>>>>>> a
>>> >>>>>>>>>>>> similar situation, though you would want to reapply your
>>> change
>>> >>>>>>>>>>>> to
>>> >>>>>>>>>>>> the
>>> >>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you
>>> would
>>> >>>>>>>>>>>> like to
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> see
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>> how this affects your branch please do try fetching the
>>> branch I
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> linked
>>> >>>>>>>>>>>> above and testing it out on a copy of your topic branch.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
>>> >>>>>>>>>>>> failed so
>>> >>>>>>>>>>>> far.
>>> >>>>>>>>>>>> Unless anyone has objections I think we should push this
>>> change
>>> >>>>>>>>>>>> either
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> this
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>> week or early next week.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Let me know what you all think.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>> - Ian
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>
>>> >
>>>
>>
>>
>

Re: The Great Merge

Posted by Ian Maxon <im...@uci.edu>.
Alright, everything is pushed. Please post here or start another thread if
you start to experience issues.

On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <im...@uci.edu> wrote:

> Just as a reminder, I'll be merging this tommorow since it seems like
> everything should go fairly smoothly.
>
> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu> wrote:
>
>> http://landbeforetime.wikia.com/wiki/Great_Valley
>>
>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu> wrote:
>>
>> > No, we are living in the GREAT valley :)
>> > Steven
>> >
>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com> wrote:
>> >
>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared like the
>> >> event that the name of this one keeps reminding me of:
>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>> >>
>> >>
>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>> >>
>> >>> It seems that I might be the only one concerned here, but it seems
>> like
>> >>> there should be others, so I am continuing this thread.
>> >>>
>> >>> I modified the perl REGEX from Chris' summer solution, and it works!
>> >>>
>> >>> Once Ian has merged master:
>> >>>
>> >>> 1. On your local branch, find the *parent* of the first commit you
>> want
>> >>> to
>> >>> migrate onto the new master, e.g.
>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
>> >>>
>> >>> 2. Run this magic command:
>> >>>
>> >>>     git format-patch --stdout
>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>> >>>
>> >>> 3. Now fetch master, and create a new local branch from it:
>> >>>
>> >>>     git switch master; git pull; git checkout -B newbranch
>> >>>
>> >>> 4. Apply your tweaked patch:
>> >>>
>> >>>     git am /tmp/my.patch
>> >>>
>> >>>
>> >>> This recognized ALL of my file moves/renames and applied them
>> correctly.
>> >>> It
>> >>> leaves only two issues:
>> >>> 1) Something similar will probably need to be done for Hyracks changes
>> >>> 2) My pom changes didn't apply. This isn't so bad since there are
>> only a
>> >>> few pom files total.
>> >>>
>> >>>
>> >>> I hope this helps,
>> >>> Steven
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
>> wrote:
>> >>>
>> >>> Here is Chris's original solution to give context. I think changing
>> the
>> >>>> REGEX might be enough to re-use the solution:
>> >>>>
>> >>>> 1. On your local branch, find the *parent* of the first commit you
>> want
>> >>>> to
>> >>>> migrate onto the new master. If you were fully up-to-date before the
>> >>>> repackaging commits went in, this will be Till's
>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that
>> here.
>> >>>>
>> >>>> 2. Run this magic command:
>> >>>>
>> >>>>     git format-patch --stdout
>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>> >>>>
>> >>>> 3. Now fetch the new master, and create a new local branch from it:
>> >>>>
>> >>>>     git switch master; git pull; git checkout -B newbranch
>> >>>>
>> >>>> 4. Apply your tweaked patch:
>> >>>>
>> >>>>     git am /tmp/my.patch
>> >>>>
>> >>>>
>> >>>> Steven
>> >>>>
>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
>> >>>> wrote:
>> >>>>
>> >>>> I've tried doing this now on my branch.
>> >>>>> As I feared, all of the files that are renamed/moved become
>> conflicts
>> >>>>> (just a few hundred conflicts in my case 😑).
>> >>>>> I'm wondering if we could use a similar technique for what we did
>> >>>>> during
>> >>>>> the summer (for the apache change) to get around this.
>> >>>>>
>> >>>>> Steven
>> >>>>>
>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
>> >>>>> wrote:
>> >>>>>
>> >>>>> I’m not sure I completely understand what you are saying. Is this a
>> >>>>>> temporary state that will get cleaned up later or is this supposed
>> to
>> >>>>>> stay this way (having "-fullstack" in the names)?
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> Till
>> >>>>>>
>> >>>>>>
>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>> >>>>>>
>> >>>>>> I'm not sure if it was necessary to rename it, but the original
>> issue
>> >>>>>> is
>> >>>>>>
>> >>>>>>> that the hyracks repo itself has a folder named hyracks, that
>> >>>>>>> contains
>> >>>>>>> hyracks. I thought this might confuse git if I did something like
>> >>>>>>> make a
>> >>>>>>> new temporary folder, move everything into that, and then rename
>> it
>> >>>>>>> to
>> >>>>>>> 'hyracks'.
>> >>>>>>>
>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Interesting!
>> >>>>>>>
>> >>>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to
>> >>>>>>>> the
>> >>>>>>>> artifactId and the hyracks module.
>> >>>>>>>>
>> >>>>>>>> Cheers,
>> >>>>>>>> Till
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>> >>>>>>>>
>> >>>>>>>> I've gone ahead and tried merging my topic branch with this
>> change,
>> >>>>>>>> and it
>> >>>>>>>>
>> >>>>>>>> turned out surprisingly well. I really didn't have many issues.
>> I'll
>> >>>>>>>>> summarize the process:
>> >>>>>>>>>
>> >>>>>>>>> 1) Merge the change from asterixdb with your topic branch
>> checked
>> >>>>>>>>> out, so
>> >>>>>>>>> just 'git merge hyracks-merge2'.
>> >>>>>>>>> The only real conflict should be the pom, if you altered that. I
>> >>>>>>>>> found it
>> >>>>>>>>> easiest to just replicate my changes and take the upstream,
>> rather
>> >>>>>>>>> than
>> >>>>>>>>> trying anything funny, since usually pom changes are not major.
>> >>>>>>>>>
>> >>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>> >>>>>>>>> hyracks-local file:///home/...')
>> >>>>>>>>>
>> >>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>> >>>>>>>>> hyracks-local/imaxon/hdfs')
>> >>>>>>>>> This also worked pretty well, the only extra hiccup besides the
>> pom
>> >>>>>>>>> was
>> >>>>>>>>> files I had created. Those appeared at the top level again after
>> >>>>>>>>> the
>> >>>>>>>>> merge.
>> >>>>>>>>> But, all you have to do is move them back down one folder into
>> >>>>>>>>> hyracks-fullstack.
>> >>>>>>>>>
>> >>>>>>>>> That's about it really. I went ahead and pushed this up to
>> github
>> >>>>>>>>> as
>> >>>>>>>>> well
>> >>>>>>>>> so if anyone would like to take a look at the process or check
>> out
>> >>>>>>>>> the
>> >>>>>>>>> branch to see what happened (at least for me), the branch is
>> here:
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>> -Ian
>> >>>>>>>>>
>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu>
>> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Chris found an issue with the way git histories were being
>> handled
>> >>>>>>>>> in
>> >>>>>>>>> the
>> >>>>>>>>>
>> >>>>>>>>> way I merged things, so I have revised the proposed branch:
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>> >>>>>>>>>>
>> >>>>>>>>>> Basically I was trying to fit everything into one commit,
>> because
>> >>>>>>>>>> I
>> >>>>>>>>>> thought at first that I could submit it to Gerrit that way.
>> >>>>>>>>>> However
>> >>>>>>>>>> that
>> >>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
>> >>>>>>>>>> every new
>> >>>>>>>>>> commit from Hyracks as a new change. Splitting the commits of
>> the
>> >>>>>>>>>> repository merge fixes the issue.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> @Till, I think that creating a textual patch would just be more
>> >>>>>>>>>> work. If
>> >>>>>>>>>> I
>> >>>>>>>>>> were to do it that way I would try fetching the Gerrit patch,
>> and
>> >>>>>>>>>> then
>> >>>>>>>>>> cherry-picking it onto a new branch that has the
>> hyracks+asterix
>> >>>>>>>>>> master
>> >>>>>>>>>> as
>> >>>>>>>>>> the head.
>> >>>>>>>>>>
>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
>> tillw@apache.org>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> To get existing patches in, could we just create a textual
>> patch
>> >>>>>>>>>> (e.g.
>> >>>>>>>>>>
>> >>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
>> >>>>>>>>>>> local
>> >>>>>>>>>>> checkout of the merged repositories and submit a new review to
>> >>>>>>>>>>> gerrit?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>> Till
>> >>>>>>>>>>>
>> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Hi all,
>> >>>>>>>>>>>
>> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>> >>>>>>>>>>>> repositories
>> >>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so
>> you
>> >>>>>>>>>>>> all can
>> >>>>>>>>>>>> check it out here:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
>> >>>>>>>>>>>> changes
>> >>>>>>>>>>>> you
>> >>>>>>>>>>>> might have open once this gets done, since it moves asterixdb
>> >>>>>>>>>>>> down
>> >>>>>>>>>>>> one
>> >>>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks
>> is
>> >>>>>>>>>>>> in
>> >>>>>>>>>>>> a
>> >>>>>>>>>>>> similar situation, though you would want to reapply your
>> change
>> >>>>>>>>>>>> to
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you
>> would
>> >>>>>>>>>>>> like to
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> see
>> >>>>>>>>>>>>
>> >>>>>>>>>>> how this affects your branch please do try fetching the
>> branch I
>> >>>>>>>>>>>
>> >>>>>>>>>>>> linked
>> >>>>>>>>>>>> above and testing it out on a copy of your topic branch.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
>> >>>>>>>>>>>> failed so
>> >>>>>>>>>>>> far.
>> >>>>>>>>>>>> Unless anyone has objections I think we should push this
>> change
>> >>>>>>>>>>>> either
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> this
>> >>>>>>>>>>>>
>> >>>>>>>>>>> week or early next week.
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Let me know what you all think.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>> - Ian
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>
>> >
>>
>
>

Re: The Great Merge

Posted by Ian Maxon <im...@uci.edu>.
Just as a reminder, I'll be merging this tommorow since it seems like
everything should go fairly smoothly.

On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sj...@ucr.edu> wrote:

> http://landbeforetime.wikia.com/wiki/Great_Valley
>
> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu> wrote:
>
> > No, we are living in the GREAT valley :)
> > Steven
> >
> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com> wrote:
> >
> >> Sounds like things are GOOD!  Excellent.  (So not to be feared like the
> >> event that the name of this one keeps reminding me of:
> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
> >>
> >>
> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
> >>
> >>> It seems that I might be the only one concerned here, but it seems like
> >>> there should be others, so I am continuing this thread.
> >>>
> >>> I modified the perl REGEX from Chris' summer solution, and it works!
> >>>
> >>> Once Ian has merged master:
> >>>
> >>> 1. On your local branch, find the *parent* of the first commit you want
> >>> to
> >>> migrate onto the new master, e.g.
> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
> >>>
> >>> 2. Run this magic command:
> >>>
> >>>     git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761
> |
> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> >>>
> >>> 3. Now fetch master, and create a new local branch from it:
> >>>
> >>>     git switch master; git pull; git checkout -B newbranch
> >>>
> >>> 4. Apply your tweaked patch:
> >>>
> >>>     git am /tmp/my.patch
> >>>
> >>>
> >>> This recognized ALL of my file moves/renames and applied them
> correctly.
> >>> It
> >>> leaves only two issues:
> >>> 1) Something similar will probably need to be done for Hyracks changes
> >>> 2) My pom changes didn't apply. This isn't so bad since there are only
> a
> >>> few pom files total.
> >>>
> >>>
> >>> I hope this helps,
> >>> Steven
> >>>
> >>>
> >>>
> >>>
> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> >>>
> >>> Here is Chris's original solution to give context. I think changing the
> >>>> REGEX might be enough to re-use the solution:
> >>>>
> >>>> 1. On your local branch, find the *parent* of the first commit you
> want
> >>>> to
> >>>> migrate onto the new master. If you were fully up-to-date before the
> >>>> repackaging commits went in, this will be Till's
> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that
> here.
> >>>>
> >>>> 2. Run this magic command:
> >>>>
> >>>>     git format-patch --stdout
> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> >>>>
> >>>> 3. Now fetch the new master, and create a new local branch from it:
> >>>>
> >>>>     git switch master; git pull; git checkout -B newbranch
> >>>>
> >>>> 4. Apply your tweaked patch:
> >>>>
> >>>>     git am /tmp/my.patch
> >>>>
> >>>>
> >>>> Steven
> >>>>
> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
> >>>> wrote:
> >>>>
> >>>> I've tried doing this now on my branch.
> >>>>> As I feared, all of the files that are renamed/moved become conflicts
> >>>>> (just a few hundred conflicts in my case 😑).
> >>>>> I'm wondering if we could use a similar technique for what we did
> >>>>> during
> >>>>> the summer (for the apache change) to get around this.
> >>>>>
> >>>>> Steven
> >>>>>
> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>> I’m not sure I completely understand what you are saying. Is this a
> >>>>>> temporary state that will get cleaned up later or is this supposed
> to
> >>>>>> stay this way (having "-fullstack" in the names)?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Till
> >>>>>>
> >>>>>>
> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> >>>>>>
> >>>>>> I'm not sure if it was necessary to rename it, but the original
> issue
> >>>>>> is
> >>>>>>
> >>>>>>> that the hyracks repo itself has a folder named hyracks, that
> >>>>>>> contains
> >>>>>>> hyracks. I thought this might confuse git if I did something like
> >>>>>>> make a
> >>>>>>> new temporary folder, move everything into that, and then rename it
> >>>>>>> to
> >>>>>>> 'hyracks'.
> >>>>>>>
> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> Interesting!
> >>>>>>>
> >>>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to
> >>>>>>>> the
> >>>>>>>> artifactId and the hyracks module.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Till
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> >>>>>>>>
> >>>>>>>> I've gone ahead and tried merging my topic branch with this
> change,
> >>>>>>>> and it
> >>>>>>>>
> >>>>>>>> turned out surprisingly well. I really didn't have many issues.
> I'll
> >>>>>>>>> summarize the process:
> >>>>>>>>>
> >>>>>>>>> 1) Merge the change from asterixdb with your topic branch checked
> >>>>>>>>> out, so
> >>>>>>>>> just 'git merge hyracks-merge2'.
> >>>>>>>>> The only real conflict should be the pom, if you altered that. I
> >>>>>>>>> found it
> >>>>>>>>> easiest to just replicate my changes and take the upstream,
> rather
> >>>>>>>>> than
> >>>>>>>>> trying anything funny, since usually pom changes are not major.
> >>>>>>>>>
> >>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
> >>>>>>>>> hyracks-local file:///home/...')
> >>>>>>>>>
> >>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
> >>>>>>>>> hyracks-local/imaxon/hdfs')
> >>>>>>>>> This also worked pretty well, the only extra hiccup besides the
> pom
> >>>>>>>>> was
> >>>>>>>>> files I had created. Those appeared at the top level again after
> >>>>>>>>> the
> >>>>>>>>> merge.
> >>>>>>>>> But, all you have to do is move them back down one folder into
> >>>>>>>>> hyracks-fullstack.
> >>>>>>>>>
> >>>>>>>>> That's about it really. I went ahead and pushed this up to github
> >>>>>>>>> as
> >>>>>>>>> well
> >>>>>>>>> so if anyone would like to take a look at the process or check
> out
> >>>>>>>>> the
> >>>>>>>>> branch to see what happened (at least for me), the branch is
> here:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> -Ian
> >>>>>>>>>
> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu>
> wrote:
> >>>>>>>>>
> >>>>>>>>> Chris found an issue with the way git histories were being
> handled
> >>>>>>>>> in
> >>>>>>>>> the
> >>>>>>>>>
> >>>>>>>>> way I merged things, so I have revised the proposed branch:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> >>>>>>>>>>
> >>>>>>>>>> Basically I was trying to fit everything into one commit,
> because
> >>>>>>>>>> I
> >>>>>>>>>> thought at first that I could submit it to Gerrit that way.
> >>>>>>>>>> However
> >>>>>>>>>> that
> >>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
> >>>>>>>>>> every new
> >>>>>>>>>> commit from Hyracks as a new change. Splitting the commits of
> the
> >>>>>>>>>> repository merge fixes the issue.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> @Till, I think that creating a textual patch would just be more
> >>>>>>>>>> work. If
> >>>>>>>>>> I
> >>>>>>>>>> were to do it that way I would try fetching the Gerrit patch,
> and
> >>>>>>>>>> then
> >>>>>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
> >>>>>>>>>> master
> >>>>>>>>>> as
> >>>>>>>>>> the head.
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
> tillw@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> To get existing patches in, could we just create a textual patch
> >>>>>>>>>> (e.g.
> >>>>>>>>>>
> >>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
> >>>>>>>>>>> local
> >>>>>>>>>>> checkout of the merged repositories and submit a new review to
> >>>>>>>>>>> gerrit?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Till
> >>>>>>>>>>>
> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi all,
> >>>>>>>>>>>
> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
> >>>>>>>>>>>> repositories
> >>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so
> you
> >>>>>>>>>>>> all can
> >>>>>>>>>>>> check it out here:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
> >>>>>>>>>>>> changes
> >>>>>>>>>>>> you
> >>>>>>>>>>>> might have open once this gets done, since it moves asterixdb
> >>>>>>>>>>>> down
> >>>>>>>>>>>> one
> >>>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks
> is
> >>>>>>>>>>>> in
> >>>>>>>>>>>> a
> >>>>>>>>>>>> similar situation, though you would want to reapply your
> change
> >>>>>>>>>>>> to
> >>>>>>>>>>>> the
> >>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
> >>>>>>>>>>>> like to
> >>>>>>>>>>>>
> >>>>>>>>>>>> see
> >>>>>>>>>>>>
> >>>>>>>>>>> how this affects your branch please do try fetching the branch
> I
> >>>>>>>>>>>
> >>>>>>>>>>>> linked
> >>>>>>>>>>>> above and testing it out on a copy of your topic branch.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
> >>>>>>>>>>>> failed so
> >>>>>>>>>>>> far.
> >>>>>>>>>>>> Unless anyone has objections I think we should push this
> change
> >>>>>>>>>>>> either
> >>>>>>>>>>>>
> >>>>>>>>>>>> this
> >>>>>>>>>>>>
> >>>>>>>>>>> week or early next week.
> >>>>>>>>>>>
> >>>>>>>>>>>> Let me know what you all think.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> - Ian
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>
> >
>

Re: The Great Merge

Posted by Steven Jacobs <sj...@ucr.edu>.
http://landbeforetime.wikia.com/wiki/Great_Valley

On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sj...@ucr.edu> wrote:

> No, we are living in the GREAT valley :)
> Steven
>
> On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com> wrote:
>
>> Sounds like things are GOOD!  Excellent.  (So not to be feared like the
>> event that the name of this one keeps reminding me of:
>> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>>
>>
>> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>>
>>> It seems that I might be the only one concerned here, but it seems like
>>> there should be others, so I am continuing this thread.
>>>
>>> I modified the perl REGEX from Chris' summer solution, and it works!
>>>
>>> Once Ian has merged master:
>>>
>>> 1. On your local branch, find the *parent* of the first commit you want
>>> to
>>> migrate onto the new master, e.g.
>>> de6e0da24c26037967eb9a937d2c77c6c43e8761
>>>
>>> 2. Run this magic command:
>>>
>>>     git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>>>
>>> 3. Now fetch master, and create a new local branch from it:
>>>
>>>     git switch master; git pull; git checkout -B newbranch
>>>
>>> 4. Apply your tweaked patch:
>>>
>>>     git am /tmp/my.patch
>>>
>>>
>>> This recognized ALL of my file moves/renames and applied them correctly.
>>> It
>>> leaves only two issues:
>>> 1) Something similar will probably need to be done for Hyracks changes
>>> 2) My pom changes didn't apply. This isn't so bad since there are only a
>>> few pom files total.
>>>
>>>
>>> I hope this helps,
>>> Steven
>>>
>>>
>>>
>>>
>>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>>>
>>> Here is Chris's original solution to give context. I think changing the
>>>> REGEX might be enough to re-use the solution:
>>>>
>>>> 1. On your local branch, find the *parent* of the first commit you want
>>>> to
>>>> migrate onto the new master. If you were fully up-to-date before the
>>>> repackaging commits went in, this will be Till's
>>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.
>>>>
>>>> 2. Run this magic command:
>>>>
>>>>     git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>>>
>>>> 3. Now fetch the new master, and create a new local branch from it:
>>>>
>>>>     git switch master; git pull; git checkout -B newbranch
>>>>
>>>> 4. Apply your tweaked patch:
>>>>
>>>>     git am /tmp/my.patch
>>>>
>>>>
>>>> Steven
>>>>
>>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
>>>> wrote:
>>>>
>>>> I've tried doing this now on my branch.
>>>>> As I feared, all of the files that are renamed/moved become conflicts
>>>>> (just a few hundred conflicts in my case 😑).
>>>>> I'm wondering if we could use a similar technique for what we did
>>>>> during
>>>>> the summer (for the apache change) to get around this.
>>>>>
>>>>> Steven
>>>>>
>>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
>>>>> wrote:
>>>>>
>>>>> I’m not sure I completely understand what you are saying. Is this a
>>>>>> temporary state that will get cleaned up later or is this supposed to
>>>>>> stay this way (having "-fullstack" in the names)?
>>>>>>
>>>>>> Thanks,
>>>>>> Till
>>>>>>
>>>>>>
>>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>>>>
>>>>>> I'm not sure if it was necessary to rename it, but the original issue
>>>>>> is
>>>>>>
>>>>>>> that the hyracks repo itself has a folder named hyracks, that
>>>>>>> contains
>>>>>>> hyracks. I thought this might confuse git if I did something like
>>>>>>> make a
>>>>>>> new temporary folder, move everything into that, and then rename it
>>>>>>> to
>>>>>>> 'hyracks'.
>>>>>>>
>>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Interesting!
>>>>>>>
>>>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to
>>>>>>>> the
>>>>>>>> artifactId and the hyracks module.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Till
>>>>>>>>
>>>>>>>>
>>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>>>>
>>>>>>>> I've gone ahead and tried merging my topic branch with this change,
>>>>>>>> and it
>>>>>>>>
>>>>>>>> turned out surprisingly well. I really didn't have many issues. I'll
>>>>>>>>> summarize the process:
>>>>>>>>>
>>>>>>>>> 1) Merge the change from asterixdb with your topic branch checked
>>>>>>>>> out, so
>>>>>>>>> just 'git merge hyracks-merge2'.
>>>>>>>>> The only real conflict should be the pom, if you altered that. I
>>>>>>>>> found it
>>>>>>>>> easiest to just replicate my changes and take the upstream, rather
>>>>>>>>> than
>>>>>>>>> trying anything funny, since usually pom changes are not major.
>>>>>>>>>
>>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>>>>>>>> hyracks-local file:///home/...')
>>>>>>>>>
>>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>>>>>>>> hyracks-local/imaxon/hdfs')
>>>>>>>>> This also worked pretty well, the only extra hiccup besides the pom
>>>>>>>>> was
>>>>>>>>> files I had created. Those appeared at the top level again after
>>>>>>>>> the
>>>>>>>>> merge.
>>>>>>>>> But, all you have to do is move them back down one folder into
>>>>>>>>> hyracks-fullstack.
>>>>>>>>>
>>>>>>>>> That's about it really. I went ahead and pushed this up to github
>>>>>>>>> as
>>>>>>>>> well
>>>>>>>>> so if anyone would like to take a look at the process or check out
>>>>>>>>> the
>>>>>>>>> branch to see what happened (at least for me), the branch is here:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -Ian
>>>>>>>>>
>>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>>>>>>>>
>>>>>>>>> Chris found an issue with the way git histories were being handled
>>>>>>>>> in
>>>>>>>>> the
>>>>>>>>>
>>>>>>>>> way I merged things, so I have revised the proposed branch:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>>>>>>
>>>>>>>>>> Basically I was trying to fit everything into one commit, because
>>>>>>>>>> I
>>>>>>>>>> thought at first that I could submit it to Gerrit that way.
>>>>>>>>>> However
>>>>>>>>>> that
>>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
>>>>>>>>>> every new
>>>>>>>>>> commit from Hyracks as a new change. Splitting the commits of the
>>>>>>>>>> repository merge fixes the issue.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> @Till, I think that creating a textual patch would just be more
>>>>>>>>>> work. If
>>>>>>>>>> I
>>>>>>>>>> were to do it that way I would try fetching the Gerrit patch, and
>>>>>>>>>> then
>>>>>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
>>>>>>>>>> master
>>>>>>>>>> as
>>>>>>>>>> the head.
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> To get existing patches in, could we just create a textual patch
>>>>>>>>>> (e.g.
>>>>>>>>>>
>>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
>>>>>>>>>>> local
>>>>>>>>>>> checkout of the merged repositories and submit a new review to
>>>>>>>>>>> gerrit?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Till
>>>>>>>>>>>
>>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>>>>>>>>> repositories
>>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you
>>>>>>>>>>>> all can
>>>>>>>>>>>> check it out here:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
>>>>>>>>>>>> changes
>>>>>>>>>>>> you
>>>>>>>>>>>> might have open once this gets done, since it moves asterixdb
>>>>>>>>>>>> down
>>>>>>>>>>>> one
>>>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is
>>>>>>>>>>>> in
>>>>>>>>>>>> a
>>>>>>>>>>>> similar situation, though you would want to reapply your change
>>>>>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
>>>>>>>>>>>> like to
>>>>>>>>>>>>
>>>>>>>>>>>> see
>>>>>>>>>>>>
>>>>>>>>>>> how this affects your branch please do try fetching the branch I
>>>>>>>>>>>
>>>>>>>>>>>> linked
>>>>>>>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
>>>>>>>>>>>> failed so
>>>>>>>>>>>> far.
>>>>>>>>>>>> Unless anyone has objections I think we should push this change
>>>>>>>>>>>> either
>>>>>>>>>>>>
>>>>>>>>>>>> this
>>>>>>>>>>>>
>>>>>>>>>>> week or early next week.
>>>>>>>>>>>
>>>>>>>>>>>> Let me know what you all think.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> - Ian
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>
>

Re: The Great Merge

Posted by Steven Jacobs <sj...@ucr.edu>.
No, we are living in the GREAT valley :)
Steven

On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dt...@gmail.com> wrote:

> Sounds like things are GOOD!  Excellent.  (So not to be feared like the
> event that the name of this one keeps reminding me of:
> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>
>
> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>
>> It seems that I might be the only one concerned here, but it seems like
>> there should be others, so I am continuing this thread.
>>
>> I modified the perl REGEX from Chris' summer solution, and it works!
>>
>> Once Ian has merged master:
>>
>> 1. On your local branch, find the *parent* of the first commit you want to
>> migrate onto the new master, e.g. de6e0da24c26037967eb9a937d2c77c6c43e8761
>>
>> 2. Run this magic command:
>>
>>     git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>>
>> 3. Now fetch master, and create a new local branch from it:
>>
>>     git switch master; git pull; git checkout -B newbranch
>>
>> 4. Apply your tweaked patch:
>>
>>     git am /tmp/my.patch
>>
>>
>> This recognized ALL of my file moves/renames and applied them correctly.
>> It
>> leaves only two issues:
>> 1) Something similar will probably need to be done for Hyracks changes
>> 2) My pom changes didn't apply. This isn't so bad since there are only a
>> few pom files total.
>>
>>
>> I hope this helps,
>> Steven
>>
>>
>>
>>
>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>>
>> Here is Chris's original solution to give context. I think changing the
>>> REGEX might be enough to re-use the solution:
>>>
>>> 1. On your local branch, find the *parent* of the first commit you want
>>> to
>>> migrate onto the new master. If you were fully up-to-date before the
>>> repackaging commits went in, this will be Till's
>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.
>>>
>>> 2. Run this magic command:
>>>
>>>     git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>>
>>> 3. Now fetch the new master, and create a new local branch from it:
>>>
>>>     git switch master; git pull; git checkout -B newbranch
>>>
>>> 4. Apply your tweaked patch:
>>>
>>>     git am /tmp/my.patch
>>>
>>>
>>> Steven
>>>
>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>>>
>>> I've tried doing this now on my branch.
>>>> As I feared, all of the files that are renamed/moved become conflicts
>>>> (just a few hundred conflicts in my case 😑).
>>>> I'm wondering if we could use a similar technique for what we did during
>>>> the summer (for the apache change) to get around this.
>>>>
>>>> Steven
>>>>
>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org> wrote:
>>>>
>>>> I’m not sure I completely understand what you are saying. Is this a
>>>>> temporary state that will get cleaned up later or is this supposed to
>>>>> stay this way (having "-fullstack" in the names)?
>>>>>
>>>>> Thanks,
>>>>> Till
>>>>>
>>>>>
>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>>>
>>>>> I'm not sure if it was necessary to rename it, but the original issue
>>>>> is
>>>>>
>>>>>> that the hyracks repo itself has a folder named hyracks, that contains
>>>>>> hyracks. I thought this might confuse git if I did something like
>>>>>> make a
>>>>>> new temporary folder, move everything into that, and then rename it to
>>>>>> 'hyracks'.
>>>>>>
>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> Interesting!
>>>>>>
>>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to the
>>>>>>> artifactId and the hyracks module.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Till
>>>>>>>
>>>>>>>
>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>>>
>>>>>>> I've gone ahead and tried merging my topic branch with this change,
>>>>>>> and it
>>>>>>>
>>>>>>> turned out surprisingly well. I really didn't have many issues. I'll
>>>>>>>> summarize the process:
>>>>>>>>
>>>>>>>> 1) Merge the change from asterixdb with your topic branch checked
>>>>>>>> out, so
>>>>>>>> just 'git merge hyracks-merge2'.
>>>>>>>> The only real conflict should be the pom, if you altered that. I
>>>>>>>> found it
>>>>>>>> easiest to just replicate my changes and take the upstream, rather
>>>>>>>> than
>>>>>>>> trying anything funny, since usually pom changes are not major.
>>>>>>>>
>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>>>>>>> hyracks-local file:///home/...')
>>>>>>>>
>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>>>>>>> hyracks-local/imaxon/hdfs')
>>>>>>>> This also worked pretty well, the only extra hiccup besides the pom
>>>>>>>> was
>>>>>>>> files I had created. Those appeared at the top level again after the
>>>>>>>> merge.
>>>>>>>> But, all you have to do is move them back down one folder into
>>>>>>>> hyracks-fullstack.
>>>>>>>>
>>>>>>>> That's about it really. I went ahead and pushed this up to github as
>>>>>>>> well
>>>>>>>> so if anyone would like to take a look at the process or check out
>>>>>>>> the
>>>>>>>> branch to see what happened (at least for me), the branch is here:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -Ian
>>>>>>>>
>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>>>>>>>
>>>>>>>> Chris found an issue with the way git histories were being handled
>>>>>>>> in
>>>>>>>> the
>>>>>>>>
>>>>>>>> way I merged things, so I have revised the proposed branch:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>>>>>
>>>>>>>>> Basically I was trying to fit everything into one commit, because I
>>>>>>>>> thought at first that I could submit it to Gerrit that way. However
>>>>>>>>> that
>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
>>>>>>>>> every new
>>>>>>>>> commit from Hyracks as a new change. Splitting the commits of the
>>>>>>>>> repository merge fixes the issue.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> @Till, I think that creating a textual patch would just be more
>>>>>>>>> work. If
>>>>>>>>> I
>>>>>>>>> were to do it that way I would try fetching the Gerrit patch, and
>>>>>>>>> then
>>>>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
>>>>>>>>> master
>>>>>>>>> as
>>>>>>>>> the head.
>>>>>>>>>
>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> To get existing patches in, could we just create a textual patch
>>>>>>>>> (e.g.
>>>>>>>>>
>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
>>>>>>>>>> local
>>>>>>>>>> checkout of the merged repositories and submit a new review to
>>>>>>>>>> gerrit?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Till
>>>>>>>>>>
>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>>>>>>>> repositories
>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you
>>>>>>>>>>> all can
>>>>>>>>>>> check it out here:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> You will likely have to do some ugly rebasing for whatever changes
>>>>>>>>>>> you
>>>>>>>>>>> might have open once this gets done, since it moves asterixdb
>>>>>>>>>>> down
>>>>>>>>>>> one
>>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is
>>>>>>>>>>> in
>>>>>>>>>>> a
>>>>>>>>>>> similar situation, though you would want to reapply your change
>>>>>>>>>>> to
>>>>>>>>>>> the
>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
>>>>>>>>>>> like to
>>>>>>>>>>>
>>>>>>>>>>> see
>>>>>>>>>>>
>>>>>>>>>> how this affects your branch please do try fetching the branch I
>>>>>>>>>>
>>>>>>>>>>> linked
>>>>>>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>>>>>>
>>>>>>>>>>> I'm still making sure all of the tests pass but nothing's failed
>>>>>>>>>>> so
>>>>>>>>>>> far.
>>>>>>>>>>> Unless anyone has objections I think we should push this change
>>>>>>>>>>> either
>>>>>>>>>>>
>>>>>>>>>>> this
>>>>>>>>>>>
>>>>>>>>>> week or early next week.
>>>>>>>>>>
>>>>>>>>>>> Let me know what you all think.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> - Ian
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>

Re: The Great Merge

Posted by Mike Carey <dt...@gmail.com>.
Sounds like things are GOOD!  Excellent.  (So not to be feared like the 
event that the name of this one keeps reminding me of: 
http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)

On 4/4/16 1:12 PM, Steven Jacobs wrote:
> It seems that I might be the only one concerned here, but it seems like
> there should be others, so I am continuing this thread.
>
> I modified the perl REGEX from Chris' summer solution, and it works!
>
> Once Ian has merged master:
>
> 1. On your local branch, find the *parent* of the first commit you want to
> migrate onto the new master, e.g. de6e0da24c26037967eb9a937d2c77c6c43e8761
>
> 2. Run this magic command:
>
>     git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>
> 3. Now fetch master, and create a new local branch from it:
>
>     git switch master; git pull; git checkout -B newbranch
>
> 4. Apply your tweaked patch:
>
>     git am /tmp/my.patch
>
>
> This recognized ALL of my file moves/renames and applied them correctly. It
> leaves only two issues:
> 1) Something similar will probably need to be done for Hyracks changes
> 2) My pom changes didn't apply. This isn't so bad since there are only a
> few pom files total.
>
>
> I hope this helps,
> Steven
>
>
>
>
> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>
>> Here is Chris's original solution to give context. I think changing the
>> REGEX might be enough to re-use the solution:
>>
>> 1. On your local branch, find the *parent* of the first commit you want to
>> migrate onto the new master. If you were fully up-to-date before the
>> repackaging commits went in, this will be Till's
>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.
>>
>> 2. Run this magic command:
>>
>>     git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>
>> 3. Now fetch the new master, and create a new local branch from it:
>>
>>     git switch master; git pull; git checkout -B newbranch
>>
>> 4. Apply your tweaked patch:
>>
>>     git am /tmp/my.patch
>>
>>
>> Steven
>>
>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>>
>>> I've tried doing this now on my branch.
>>> As I feared, all of the files that are renamed/moved become conflicts
>>> (just a few hundred conflicts in my case 😑).
>>> I'm wondering if we could use a similar technique for what we did during
>>> the summer (for the apache change) to get around this.
>>>
>>> Steven
>>>
>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org> wrote:
>>>
>>>> I’m not sure I completely understand what you are saying. Is this a
>>>> temporary state that will get cleaned up later or is this supposed to
>>>> stay this way (having "-fullstack" in the names)?
>>>>
>>>> Thanks,
>>>> Till
>>>>
>>>>
>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>>
>>>> I'm not sure if it was necessary to rename it, but the original issue is
>>>>> that the hyracks repo itself has a folder named hyracks, that contains
>>>>> hyracks. I thought this might confuse git if I did something like make a
>>>>> new temporary folder, move everything into that, and then rename it to
>>>>> 'hyracks'.
>>>>>
>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Interesting!
>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to the
>>>>>> artifactId and the hyracks module.
>>>>>>
>>>>>> Cheers,
>>>>>> Till
>>>>>>
>>>>>>
>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>>
>>>>>> I've gone ahead and tried merging my topic branch with this change,
>>>>>> and it
>>>>>>
>>>>>>> turned out surprisingly well. I really didn't have many issues. I'll
>>>>>>> summarize the process:
>>>>>>>
>>>>>>> 1) Merge the change from asterixdb with your topic branch checked
>>>>>>> out, so
>>>>>>> just 'git merge hyracks-merge2'.
>>>>>>> The only real conflict should be the pom, if you altered that. I
>>>>>>> found it
>>>>>>> easiest to just replicate my changes and take the upstream, rather
>>>>>>> than
>>>>>>> trying anything funny, since usually pom changes are not major.
>>>>>>>
>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>>>>>> hyracks-local file:///home/...')
>>>>>>>
>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>>>>>> hyracks-local/imaxon/hdfs')
>>>>>>> This also worked pretty well, the only extra hiccup besides the pom
>>>>>>> was
>>>>>>> files I had created. Those appeared at the top level again after the
>>>>>>> merge.
>>>>>>> But, all you have to do is move them back down one folder into
>>>>>>> hyracks-fullstack.
>>>>>>>
>>>>>>> That's about it really. I went ahead and pushed this up to github as
>>>>>>> well
>>>>>>> so if anyone would like to take a look at the process or check out the
>>>>>>> branch to see what happened (at least for me), the branch is here:
>>>>>>>
>>>>>>>
>>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -Ian
>>>>>>>
>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>>>>>>
>>>>>>> Chris found an issue with the way git histories were being handled in
>>>>>>> the
>>>>>>>
>>>>>>>> way I merged things, so I have revised the proposed branch:
>>>>>>>>
>>>>>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>>>>
>>>>>>>> Basically I was trying to fit everything into one commit, because I
>>>>>>>> thought at first that I could submit it to Gerrit that way. However
>>>>>>>> that
>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
>>>>>>>> every new
>>>>>>>> commit from Hyracks as a new change. Splitting the commits of the
>>>>>>>> repository merge fixes the issue.
>>>>>>>>
>>>>>>>>
>>>>>>>> @Till, I think that creating a textual patch would just be more
>>>>>>>> work. If
>>>>>>>> I
>>>>>>>> were to do it that way I would try fetching the Gerrit patch, and
>>>>>>>> then
>>>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
>>>>>>>> master
>>>>>>>> as
>>>>>>>> the head.
>>>>>>>>
>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> To get existing patches in, could we just create a textual patch
>>>>>>>> (e.g.
>>>>>>>>
>>>>>>>>> from gerrit), apply that with the necessary -p option to a new local
>>>>>>>>> checkout of the merged repositories and submit a new review to
>>>>>>>>> gerrit?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Till
>>>>>>>>>
>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>>>>>>> repositories
>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you
>>>>>>>>>> all can
>>>>>>>>>> check it out here:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> You will likely have to do some ugly rebasing for whatever changes
>>>>>>>>>> you
>>>>>>>>>> might have open once this gets done, since it moves asterixdb down
>>>>>>>>>> one
>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is in
>>>>>>>>>> a
>>>>>>>>>> similar situation, though you would want to reapply your change to
>>>>>>>>>> the
>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
>>>>>>>>>> like to
>>>>>>>>>>
>>>>>>>>>> see
>>>>>>>>> how this affects your branch please do try fetching the branch I
>>>>>>>>>> linked
>>>>>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>>>>>
>>>>>>>>>> I'm still making sure all of the tests pass but nothing's failed so
>>>>>>>>>> far.
>>>>>>>>>> Unless anyone has objections I think we should push this change
>>>>>>>>>> either
>>>>>>>>>>
>>>>>>>>>> this
>>>>>>>>> week or early next week.
>>>>>>>>>> Let me know what you all think.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> - Ian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>

Re: The Great Merge

Posted by Taewoo Kim <wa...@gmail.com>.
@Steven: Thanks. I will try.

Best,
Taewoo

On Thu, Apr 14, 2016 at 12:56 PM, Steven Jacobs <sj...@ucr.edu> wrote:

> Taewoo,
> I agree with Ildar now, after exploring both for a long time. The patch
> trick was breaking a lot of stuff without me realizing it until afterwards.
> The trick is, for files that you moved, you need to do the following three
> steps after merge:
>
> 1) git rm --cached      on the file from master
> 2) git add                   on the renamed file on your branch
> 3) git mv                    the file from your branch to the appropriate
> folder (one level deeper)
>
> This is mostly scriptable. I have 130 files with moves/renames and this
> took me less than half of an hour.
>
> Steven
>
> On Thu, Apr 14, 2016 at 12:11 PM, Ildar Absalyamov <
> ildar.absalyamov@gmail.com> wrote:
>
> > Taewoo,
> >
> > I believe Steven’s patch method will not work if you were using merge to
> > get master’s changes into your branch. You would have to do interactive
> > rebase and move all your changes as if they were applied on top of
> master,
> > which I found to be a lot of work by itself.
> > Actually I would refrain everyone from using patch method, since it’s
> > error-prone (simple search and replace).
> >
> > I do think the method proposed by Ian (which uses merge) is the best one
> > can do in this situation. Yes, it will generate conflicts (especially if
> > you moved files in your topic branch), but I found that conflict
> resolution
> > was more of less scriptable.
> >
> > > On Apr 14, 2016, at 11:31, Taewoo Kim <wa...@gmail.com> wrote:
> > >
> > > Hello Steven,
> > >
> > > I tried to use Ian's straightforward method and it generated a few
> > hundred
> > > conflicts since I touched a lot of test files. I tried to go through
> one
> > by
> > > one and it took long time. So, I want to explore your method, too. The
> > one
> > > thing that I don't understand on your method is "finding the parent".
> Is
> > > that the last commit of the master branch that was merged into my local
> > > branch? Thank you.
> > >
> > > Best,
> > > Taewoo
> > >
> > > On Mon, Apr 4, 2016 at 1:12 PM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> > >
> > >> It seems that I might be the only one concerned here, but it seems
> like
> > >> there should be others, so I am continuing this thread.
> > >>
> > >> I modified the perl REGEX from Chris' summer solution, and it works!
> > >>
> > >> Once Ian has merged master:
> > >>
> > >> 1. On your local branch, find the *parent* of the first commit you
> want
> > to
> > >> migrate onto the new master, e.g.
> > de6e0da24c26037967eb9a937d2c77c6c43e8761
> > >>
> > >> 2. Run this magic command:
> > >>
> > >>   git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> > >> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> > >>
> > >> 3. Now fetch master, and create a new local branch from it:
> > >>
> > >>   git switch master; git pull; git checkout -B newbranch
> > >>
> > >> 4. Apply your tweaked patch:
> > >>
> > >>   git am /tmp/my.patch
> > >>
> > >>
> > >> This recognized ALL of my file moves/renames and applied them
> > correctly. It
> > >> leaves only two issues:
> > >> 1) Something similar will probably need to be done for Hyracks changes
> > >> 2) My pom changes didn't apply. This isn't so bad since there are
> only a
> > >> few pom files total.
> > >>
> > >>
> > >> I hope this helps,
> > >> Steven
> > >>
> > >>
> > >>
> > >>
> > >> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
> > wrote:
> > >>
> > >>> Here is Chris's original solution to give context. I think changing
> the
> > >>> REGEX might be enough to re-use the solution:
> > >>>
> > >>> 1. On your local branch, find the *parent* of the first commit you
> want
> > >> to
> > >>> migrate onto the new master. If you were fully up-to-date before the
> > >>> repackaging commits went in, this will be Till's
> > >>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that
> here.
> > >>>
> > >>> 2. Run this magic command:
> > >>>
> > >>>   git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53
> |
> > >>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> > >>>
> > >>> 3. Now fetch the new master, and create a new local branch from it:
> > >>>
> > >>>   git switch master; git pull; git checkout -B newbranch
> > >>>
> > >>> 4. Apply your tweaked patch:
> > >>>
> > >>>   git am /tmp/my.patch
> > >>>
> > >>>
> > >>> Steven
> > >>>
> > >>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
> > wrote:
> > >>>
> > >>>> I've tried doing this now on my branch.
> > >>>> As I feared, all of the files that are renamed/moved become
> conflicts
> > >>>> (just a few hundred conflicts in my case 😑).
> > >>>> I'm wondering if we could use a similar technique for what we did
> > during
> > >>>> the summer (for the apache change) to get around this.
> > >>>>
> > >>>> Steven
> > >>>>
> > >>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
> > wrote:
> > >>>>
> > >>>>> I’m not sure I completely understand what you are saying. Is this a
> > >>>>> temporary state that will get cleaned up later or is this supposed
> to
> > >>>>> stay this way (having "-fullstack" in the names)?
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Till
> > >>>>>
> > >>>>>
> > >>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> > >>>>>
> > >>>>> I'm not sure if it was necessary to rename it, but the original
> issue
> > >> is
> > >>>>>> that the hyracks repo itself has a folder named hyracks, that
> > contains
> > >>>>>> hyracks. I thought this might confuse git if I did something like
> > >> make a
> > >>>>>> new temporary folder, move everything into that, and then rename
> it
> > to
> > >>>>>> 'hyracks'.
> > >>>>>>
> > >>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>> Interesting!
> > >>>>>>>
> > >>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to
> > the
> > >>>>>>> artifactId and the hyracks module.
> > >>>>>>>
> > >>>>>>> Cheers,
> > >>>>>>> Till
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> > >>>>>>>
> > >>>>>>> I've gone ahead and tried merging my topic branch with this
> change,
> > >>>>>>> and it
> > >>>>>>>
> > >>>>>>>> turned out surprisingly well. I really didn't have many issues.
> > I'll
> > >>>>>>>> summarize the process:
> > >>>>>>>>
> > >>>>>>>> 1) Merge the change from asterixdb with your topic branch
> checked
> > >>>>>>>> out, so
> > >>>>>>>> just 'git merge hyracks-merge2'.
> > >>>>>>>> The only real conflict should be the pom, if you altered that. I
> > >>>>>>>> found it
> > >>>>>>>> easiest to just replicate my changes and take the upstream,
> rather
> > >>>>>>>> than
> > >>>>>>>> trying anything funny, since usually pom changes are not major.
> > >>>>>>>>
> > >>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
> > >>>>>>>> hyracks-local file:///home/...')
> > >>>>>>>>
> > >>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
> > >>>>>>>> hyracks-local/imaxon/hdfs')
> > >>>>>>>> This also worked pretty well, the only extra hiccup besides the
> > pom
> > >>>>>>>> was
> > >>>>>>>> files I had created. Those appeared at the top level again after
> > the
> > >>>>>>>> merge.
> > >>>>>>>> But, all you have to do is move them back down one folder into
> > >>>>>>>> hyracks-fullstack.
> > >>>>>>>>
> > >>>>>>>> That's about it really. I went ahead and pushed this up to
> github
> > as
> > >>>>>>>> well
> > >>>>>>>> so if anyone would like to take a look at the process or check
> out
> > >> the
> > >>>>>>>> branch to see what happened (at least for me), the branch is
> here:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> -Ian
> > >>>>>>>>
> > >>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu>
> > wrote:
> > >>>>>>>>
> > >>>>>>>> Chris found an issue with the way git histories were being
> handled
> > >> in
> > >>>>>>>> the
> > >>>>>>>>
> > >>>>>>>>> way I merged things, so I have revised the proposed branch:
> > >>>>>>>>>
> > >>>>>>>>>
> > >>
> > https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> > >>>>>>>>>
> > >>>>>>>>> Basically I was trying to fit everything into one commit,
> > because I
> > >>>>>>>>> thought at first that I could submit it to Gerrit that way.
> > However
> > >>>>>>>>> that
> > >>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
> > >>>>>>>>> every new
> > >>>>>>>>> commit from Hyracks as a new change. Splitting the commits of
> the
> > >>>>>>>>> repository merge fixes the issue.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> @Till, I think that creating a textual patch would just be more
> > >>>>>>>>> work. If
> > >>>>>>>>> I
> > >>>>>>>>> were to do it that way I would try fetching the Gerrit patch,
> and
> > >>>>>>>>> then
> > >>>>>>>>> cherry-picking it onto a new branch that has the
> hyracks+asterix
> > >>>>>>>>> master
> > >>>>>>>>> as
> > >>>>>>>>> the head.
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
> tillw@apache.org
> > >
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> To get existing patches in, could we just create a textual
> patch
> > >>>>>>>>> (e.g.
> > >>>>>>>>>
> > >>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
> > >> local
> > >>>>>>>>>> checkout of the merged repositories and submit a new review to
> > >>>>>>>>>> gerrit?
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> Till
> > >>>>>>>>>>
> > >>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Hi all,
> > >>>>>>>>>>
> > >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
> > >>>>>>>>>>> repositories
> > >>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so
> you
> > >>>>>>>>>>> all can
> > >>>>>>>>>>> check it out here:
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
> > >> changes
> > >>>>>>>>>>> you
> > >>>>>>>>>>> might have open once this gets done, since it moves asterixdb
> > >> down
> > >>>>>>>>>>> one
> > >>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks
> is
> > >> in
> > >>>>>>>>>>> a
> > >>>>>>>>>>> similar situation, though you would want to reapply your
> change
> > >> to
> > >>>>>>>>>>> the
> > >>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you
> would
> > >>>>>>>>>>> like to
> > >>>>>>>>>>>
> > >>>>>>>>>>> see
> > >>>>>>>>>>
> > >>>>>>>>>> how this affects your branch please do try fetching the
> branch I
> > >>>>>>>>>>> linked
> > >>>>>>>>>>> above and testing it out on a copy of your topic branch.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
> > failed
> > >> so
> > >>>>>>>>>>> far.
> > >>>>>>>>>>> Unless anyone has objections I think we should push this
> change
> > >>>>>>>>>>> either
> > >>>>>>>>>>>
> > >>>>>>>>>>> this
> > >>>>>>>>>>
> > >>>>>>>>>> week or early next week.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Let me know what you all think.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> - Ian
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>
> > >>>
> > >>
> >
> > Best regards,
> > Ildar
> >
> >
>

Re: The Great Merge

Posted by Steven Jacobs <sj...@ucr.edu>.
Taewoo,
I agree with Ildar now, after exploring both for a long time. The patch
trick was breaking a lot of stuff without me realizing it until afterwards.
The trick is, for files that you moved, you need to do the following three
steps after merge:

1) git rm --cached      on the file from master
2) git add                   on the renamed file on your branch
3) git mv                    the file from your branch to the appropriate
folder (one level deeper)

This is mostly scriptable. I have 130 files with moves/renames and this
took me less than half of an hour.

Steven

On Thu, Apr 14, 2016 at 12:11 PM, Ildar Absalyamov <
ildar.absalyamov@gmail.com> wrote:

> Taewoo,
>
> I believe Steven’s patch method will not work if you were using merge to
> get master’s changes into your branch. You would have to do interactive
> rebase and move all your changes as if they were applied on top of master,
> which I found to be a lot of work by itself.
> Actually I would refrain everyone from using patch method, since it’s
> error-prone (simple search and replace).
>
> I do think the method proposed by Ian (which uses merge) is the best one
> can do in this situation. Yes, it will generate conflicts (especially if
> you moved files in your topic branch), but I found that conflict resolution
> was more of less scriptable.
>
> > On Apr 14, 2016, at 11:31, Taewoo Kim <wa...@gmail.com> wrote:
> >
> > Hello Steven,
> >
> > I tried to use Ian's straightforward method and it generated a few
> hundred
> > conflicts since I touched a lot of test files. I tried to go through one
> by
> > one and it took long time. So, I want to explore your method, too. The
> one
> > thing that I don't understand on your method is "finding the parent". Is
> > that the last commit of the master branch that was merged into my local
> > branch? Thank you.
> >
> > Best,
> > Taewoo
> >
> > On Mon, Apr 4, 2016 at 1:12 PM, Steven Jacobs <sj...@ucr.edu> wrote:
> >
> >> It seems that I might be the only one concerned here, but it seems like
> >> there should be others, so I am continuing this thread.
> >>
> >> I modified the perl REGEX from Chris' summer solution, and it works!
> >>
> >> Once Ian has merged master:
> >>
> >> 1. On your local branch, find the *parent* of the first commit you want
> to
> >> migrate onto the new master, e.g.
> de6e0da24c26037967eb9a937d2c77c6c43e8761
> >>
> >> 2. Run this magic command:
> >>
> >>   git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> >> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> >>
> >> 3. Now fetch master, and create a new local branch from it:
> >>
> >>   git switch master; git pull; git checkout -B newbranch
> >>
> >> 4. Apply your tweaked patch:
> >>
> >>   git am /tmp/my.patch
> >>
> >>
> >> This recognized ALL of my file moves/renames and applied them
> correctly. It
> >> leaves only two issues:
> >> 1) Something similar will probably need to be done for Hyracks changes
> >> 2) My pom changes didn't apply. This isn't so bad since there are only a
> >> few pom files total.
> >>
> >>
> >> I hope this helps,
> >> Steven
> >>
> >>
> >>
> >>
> >> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> >>
> >>> Here is Chris's original solution to give context. I think changing the
> >>> REGEX might be enough to re-use the solution:
> >>>
> >>> 1. On your local branch, find the *parent* of the first commit you want
> >> to
> >>> migrate onto the new master. If you were fully up-to-date before the
> >>> repackaging commits went in, this will be Till's
> >>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.
> >>>
> >>> 2. Run this magic command:
> >>>
> >>>   git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> >>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> >>>
> >>> 3. Now fetch the new master, and create a new local branch from it:
> >>>
> >>>   git switch master; git pull; git checkout -B newbranch
> >>>
> >>> 4. Apply your tweaked patch:
> >>>
> >>>   git am /tmp/my.patch
> >>>
> >>>
> >>> Steven
> >>>
> >>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> >>>
> >>>> I've tried doing this now on my branch.
> >>>> As I feared, all of the files that are renamed/moved become conflicts
> >>>> (just a few hundred conflicts in my case 😑).
> >>>> I'm wondering if we could use a similar technique for what we did
> during
> >>>> the summer (for the apache change) to get around this.
> >>>>
> >>>> Steven
> >>>>
> >>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
> wrote:
> >>>>
> >>>>> I’m not sure I completely understand what you are saying. Is this a
> >>>>> temporary state that will get cleaned up later or is this supposed to
> >>>>> stay this way (having "-fullstack" in the names)?
> >>>>>
> >>>>> Thanks,
> >>>>> Till
> >>>>>
> >>>>>
> >>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> >>>>>
> >>>>> I'm not sure if it was necessary to rename it, but the original issue
> >> is
> >>>>>> that the hyracks repo itself has a folder named hyracks, that
> contains
> >>>>>> hyracks. I thought this might confuse git if I did something like
> >> make a
> >>>>>> new temporary folder, move everything into that, and then rename it
> to
> >>>>>> 'hyracks'.
> >>>>>>
> >>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Interesting!
> >>>>>>>
> >>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to
> the
> >>>>>>> artifactId and the hyracks module.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Till
> >>>>>>>
> >>>>>>>
> >>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> >>>>>>>
> >>>>>>> I've gone ahead and tried merging my topic branch with this change,
> >>>>>>> and it
> >>>>>>>
> >>>>>>>> turned out surprisingly well. I really didn't have many issues.
> I'll
> >>>>>>>> summarize the process:
> >>>>>>>>
> >>>>>>>> 1) Merge the change from asterixdb with your topic branch checked
> >>>>>>>> out, so
> >>>>>>>> just 'git merge hyracks-merge2'.
> >>>>>>>> The only real conflict should be the pom, if you altered that. I
> >>>>>>>> found it
> >>>>>>>> easiest to just replicate my changes and take the upstream, rather
> >>>>>>>> than
> >>>>>>>> trying anything funny, since usually pom changes are not major.
> >>>>>>>>
> >>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
> >>>>>>>> hyracks-local file:///home/...')
> >>>>>>>>
> >>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
> >>>>>>>> hyracks-local/imaxon/hdfs')
> >>>>>>>> This also worked pretty well, the only extra hiccup besides the
> pom
> >>>>>>>> was
> >>>>>>>> files I had created. Those appeared at the top level again after
> the
> >>>>>>>> merge.
> >>>>>>>> But, all you have to do is move them back down one folder into
> >>>>>>>> hyracks-fullstack.
> >>>>>>>>
> >>>>>>>> That's about it really. I went ahead and pushed this up to github
> as
> >>>>>>>> well
> >>>>>>>> so if anyone would like to take a look at the process or check out
> >> the
> >>>>>>>> branch to see what happened (at least for me), the branch is here:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -Ian
> >>>>>>>>
> >>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu>
> wrote:
> >>>>>>>>
> >>>>>>>> Chris found an issue with the way git histories were being handled
> >> in
> >>>>>>>> the
> >>>>>>>>
> >>>>>>>>> way I merged things, so I have revised the proposed branch:
> >>>>>>>>>
> >>>>>>>>>
> >>
> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> >>>>>>>>>
> >>>>>>>>> Basically I was trying to fit everything into one commit,
> because I
> >>>>>>>>> thought at first that I could submit it to Gerrit that way.
> However
> >>>>>>>>> that
> >>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
> >>>>>>>>> every new
> >>>>>>>>> commit from Hyracks as a new change. Splitting the commits of the
> >>>>>>>>> repository merge fixes the issue.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> @Till, I think that creating a textual patch would just be more
> >>>>>>>>> work. If
> >>>>>>>>> I
> >>>>>>>>> were to do it that way I would try fetching the Gerrit patch, and
> >>>>>>>>> then
> >>>>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
> >>>>>>>>> master
> >>>>>>>>> as
> >>>>>>>>> the head.
> >>>>>>>>>
> >>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <tillw@apache.org
> >
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> To get existing patches in, could we just create a textual patch
> >>>>>>>>> (e.g.
> >>>>>>>>>
> >>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
> >> local
> >>>>>>>>>> checkout of the merged repositories and submit a new review to
> >>>>>>>>>> gerrit?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Till
> >>>>>>>>>>
> >>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi all,
> >>>>>>>>>>
> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
> >>>>>>>>>>> repositories
> >>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you
> >>>>>>>>>>> all can
> >>>>>>>>>>> check it out here:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
> >> changes
> >>>>>>>>>>> you
> >>>>>>>>>>> might have open once this gets done, since it moves asterixdb
> >> down
> >>>>>>>>>>> one
> >>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is
> >> in
> >>>>>>>>>>> a
> >>>>>>>>>>> similar situation, though you would want to reapply your change
> >> to
> >>>>>>>>>>> the
> >>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
> >>>>>>>>>>> like to
> >>>>>>>>>>>
> >>>>>>>>>>> see
> >>>>>>>>>>
> >>>>>>>>>> how this affects your branch please do try fetching the branch I
> >>>>>>>>>>> linked
> >>>>>>>>>>> above and testing it out on a copy of your topic branch.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
> failed
> >> so
> >>>>>>>>>>> far.
> >>>>>>>>>>> Unless anyone has objections I think we should push this change
> >>>>>>>>>>> either
> >>>>>>>>>>>
> >>>>>>>>>>> this
> >>>>>>>>>>
> >>>>>>>>>> week or early next week.
> >>>>>>>>>>>
> >>>>>>>>>>> Let me know what you all think.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> - Ian
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>
> >>>
> >>
>
> Best regards,
> Ildar
>
>

Re: The Great Merge

Posted by Taewoo Kim <wa...@gmail.com>.
Thanks Ildar for the detailed information.

Best,
Taewoo

On Thu, Apr 14, 2016 at 12:11 PM, Ildar Absalyamov <
ildar.absalyamov@gmail.com> wrote:

> Taewoo,
>
> I believe Steven’s patch method will not work if you were using merge to
> get master’s changes into your branch. You would have to do interactive
> rebase and move all your changes as if they were applied on top of master,
> which I found to be a lot of work by itself.
> Actually I would refrain everyone from using patch method, since it’s
> error-prone (simple search and replace).
>
> I do think the method proposed by Ian (which uses merge) is the best one
> can do in this situation. Yes, it will generate conflicts (especially if
> you moved files in your topic branch), but I found that conflict resolution
> was more of less scriptable.
>
> > On Apr 14, 2016, at 11:31, Taewoo Kim <wa...@gmail.com> wrote:
> >
> > Hello Steven,
> >
> > I tried to use Ian's straightforward method and it generated a few
> hundred
> > conflicts since I touched a lot of test files. I tried to go through one
> by
> > one and it took long time. So, I want to explore your method, too. The
> one
> > thing that I don't understand on your method is "finding the parent". Is
> > that the last commit of the master branch that was merged into my local
> > branch? Thank you.
> >
> > Best,
> > Taewoo
> >
> > On Mon, Apr 4, 2016 at 1:12 PM, Steven Jacobs <sj...@ucr.edu> wrote:
> >
> >> It seems that I might be the only one concerned here, but it seems like
> >> there should be others, so I am continuing this thread.
> >>
> >> I modified the perl REGEX from Chris' summer solution, and it works!
> >>
> >> Once Ian has merged master:
> >>
> >> 1. On your local branch, find the *parent* of the first commit you want
> to
> >> migrate onto the new master, e.g.
> de6e0da24c26037967eb9a937d2c77c6c43e8761
> >>
> >> 2. Run this magic command:
> >>
> >>   git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> >> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> >>
> >> 3. Now fetch master, and create a new local branch from it:
> >>
> >>   git switch master; git pull; git checkout -B newbranch
> >>
> >> 4. Apply your tweaked patch:
> >>
> >>   git am /tmp/my.patch
> >>
> >>
> >> This recognized ALL of my file moves/renames and applied them
> correctly. It
> >> leaves only two issues:
> >> 1) Something similar will probably need to be done for Hyracks changes
> >> 2) My pom changes didn't apply. This isn't so bad since there are only a
> >> few pom files total.
> >>
> >>
> >> I hope this helps,
> >> Steven
> >>
> >>
> >>
> >>
> >> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> >>
> >>> Here is Chris's original solution to give context. I think changing the
> >>> REGEX might be enough to re-use the solution:
> >>>
> >>> 1. On your local branch, find the *parent* of the first commit you want
> >> to
> >>> migrate onto the new master. If you were fully up-to-date before the
> >>> repackaging commits went in, this will be Till's
> >>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.
> >>>
> >>> 2. Run this magic command:
> >>>
> >>>   git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> >>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> >>>
> >>> 3. Now fetch the new master, and create a new local branch from it:
> >>>
> >>>   git switch master; git pull; git checkout -B newbranch
> >>>
> >>> 4. Apply your tweaked patch:
> >>>
> >>>   git am /tmp/my.patch
> >>>
> >>>
> >>> Steven
> >>>
> >>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu>
> wrote:
> >>>
> >>>> I've tried doing this now on my branch.
> >>>> As I feared, all of the files that are renamed/moved become conflicts
> >>>> (just a few hundred conflicts in my case 😑).
> >>>> I'm wondering if we could use a similar technique for what we did
> during
> >>>> the summer (for the apache change) to get around this.
> >>>>
> >>>> Steven
> >>>>
> >>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org>
> wrote:
> >>>>
> >>>>> I’m not sure I completely understand what you are saying. Is this a
> >>>>> temporary state that will get cleaned up later or is this supposed to
> >>>>> stay this way (having "-fullstack" in the names)?
> >>>>>
> >>>>> Thanks,
> >>>>> Till
> >>>>>
> >>>>>
> >>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> >>>>>
> >>>>> I'm not sure if it was necessary to rename it, but the original issue
> >> is
> >>>>>> that the hyracks repo itself has a folder named hyracks, that
> contains
> >>>>>> hyracks. I thought this might confuse git if I did something like
> >> make a
> >>>>>> new temporary folder, move everything into that, and then rename it
> to
> >>>>>> 'hyracks'.
> >>>>>>
> >>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Interesting!
> >>>>>>>
> >>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to
> the
> >>>>>>> artifactId and the hyracks module.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Till
> >>>>>>>
> >>>>>>>
> >>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> >>>>>>>
> >>>>>>> I've gone ahead and tried merging my topic branch with this change,
> >>>>>>> and it
> >>>>>>>
> >>>>>>>> turned out surprisingly well. I really didn't have many issues.
> I'll
> >>>>>>>> summarize the process:
> >>>>>>>>
> >>>>>>>> 1) Merge the change from asterixdb with your topic branch checked
> >>>>>>>> out, so
> >>>>>>>> just 'git merge hyracks-merge2'.
> >>>>>>>> The only real conflict should be the pom, if you altered that. I
> >>>>>>>> found it
> >>>>>>>> easiest to just replicate my changes and take the upstream, rather
> >>>>>>>> than
> >>>>>>>> trying anything funny, since usually pom changes are not major.
> >>>>>>>>
> >>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
> >>>>>>>> hyracks-local file:///home/...')
> >>>>>>>>
> >>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
> >>>>>>>> hyracks-local/imaxon/hdfs')
> >>>>>>>> This also worked pretty well, the only extra hiccup besides the
> pom
> >>>>>>>> was
> >>>>>>>> files I had created. Those appeared at the top level again after
> the
> >>>>>>>> merge.
> >>>>>>>> But, all you have to do is move them back down one folder into
> >>>>>>>> hyracks-fullstack.
> >>>>>>>>
> >>>>>>>> That's about it really. I went ahead and pushed this up to github
> as
> >>>>>>>> well
> >>>>>>>> so if anyone would like to take a look at the process or check out
> >> the
> >>>>>>>> branch to see what happened (at least for me), the branch is here:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> -Ian
> >>>>>>>>
> >>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu>
> wrote:
> >>>>>>>>
> >>>>>>>> Chris found an issue with the way git histories were being handled
> >> in
> >>>>>>>> the
> >>>>>>>>
> >>>>>>>>> way I merged things, so I have revised the proposed branch:
> >>>>>>>>>
> >>>>>>>>>
> >>
> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> >>>>>>>>>
> >>>>>>>>> Basically I was trying to fit everything into one commit,
> because I
> >>>>>>>>> thought at first that I could submit it to Gerrit that way.
> However
> >>>>>>>>> that
> >>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
> >>>>>>>>> every new
> >>>>>>>>> commit from Hyracks as a new change. Splitting the commits of the
> >>>>>>>>> repository merge fixes the issue.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> @Till, I think that creating a textual patch would just be more
> >>>>>>>>> work. If
> >>>>>>>>> I
> >>>>>>>>> were to do it that way I would try fetching the Gerrit patch, and
> >>>>>>>>> then
> >>>>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
> >>>>>>>>> master
> >>>>>>>>> as
> >>>>>>>>> the head.
> >>>>>>>>>
> >>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <tillw@apache.org
> >
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> To get existing patches in, could we just create a textual patch
> >>>>>>>>> (e.g.
> >>>>>>>>>
> >>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
> >> local
> >>>>>>>>>> checkout of the merged repositories and submit a new review to
> >>>>>>>>>> gerrit?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Till
> >>>>>>>>>>
> >>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi all,
> >>>>>>>>>>
> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
> >>>>>>>>>>> repositories
> >>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you
> >>>>>>>>>>> all can
> >>>>>>>>>>> check it out here:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
> >> changes
> >>>>>>>>>>> you
> >>>>>>>>>>> might have open once this gets done, since it moves asterixdb
> >> down
> >>>>>>>>>>> one
> >>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is
> >> in
> >>>>>>>>>>> a
> >>>>>>>>>>> similar situation, though you would want to reapply your change
> >> to
> >>>>>>>>>>> the
> >>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
> >>>>>>>>>>> like to
> >>>>>>>>>>>
> >>>>>>>>>>> see
> >>>>>>>>>>
> >>>>>>>>>> how this affects your branch please do try fetching the branch I
> >>>>>>>>>>> linked
> >>>>>>>>>>> above and testing it out on a copy of your topic branch.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm still making sure all of the tests pass but nothing's
> failed
> >> so
> >>>>>>>>>>> far.
> >>>>>>>>>>> Unless anyone has objections I think we should push this change
> >>>>>>>>>>> either
> >>>>>>>>>>>
> >>>>>>>>>>> this
> >>>>>>>>>>
> >>>>>>>>>> week or early next week.
> >>>>>>>>>>>
> >>>>>>>>>>> Let me know what you all think.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> - Ian
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>
> >>>
> >>
>
> Best regards,
> Ildar
>
>

Re: The Great Merge

Posted by Ildar Absalyamov <il...@gmail.com>.
Taewoo,

I believe Steven’s patch method will not work if you were using merge to get master’s changes into your branch. You would have to do interactive rebase and move all your changes as if they were applied on top of master, which I found to be a lot of work by itself.
Actually I would refrain everyone from using patch method, since it’s error-prone (simple search and replace).

I do think the method proposed by Ian (which uses merge) is the best one can do in this situation. Yes, it will generate conflicts (especially if you moved files in your topic branch), but I found that conflict resolution was more of less scriptable.

> On Apr 14, 2016, at 11:31, Taewoo Kim <wa...@gmail.com> wrote:
> 
> Hello Steven,
> 
> I tried to use Ian's straightforward method and it generated a few hundred
> conflicts since I touched a lot of test files. I tried to go through one by
> one and it took long time. So, I want to explore your method, too. The one
> thing that I don't understand on your method is "finding the parent". Is
> that the last commit of the master branch that was merged into my local
> branch? Thank you.
> 
> Best,
> Taewoo
> 
> On Mon, Apr 4, 2016 at 1:12 PM, Steven Jacobs <sj...@ucr.edu> wrote:
> 
>> It seems that I might be the only one concerned here, but it seems like
>> there should be others, so I am continuing this thread.
>> 
>> I modified the perl REGEX from Chris' summer solution, and it works!
>> 
>> Once Ian has merged master:
>> 
>> 1. On your local branch, find the *parent* of the first commit you want to
>> migrate onto the new master, e.g. de6e0da24c26037967eb9a937d2c77c6c43e8761
>> 
>> 2. Run this magic command:
>> 
>>   git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>> 
>> 3. Now fetch master, and create a new local branch from it:
>> 
>>   git switch master; git pull; git checkout -B newbranch
>> 
>> 4. Apply your tweaked patch:
>> 
>>   git am /tmp/my.patch
>> 
>> 
>> This recognized ALL of my file moves/renames and applied them correctly. It
>> leaves only two issues:
>> 1) Something similar will probably need to be done for Hyracks changes
>> 2) My pom changes didn't apply. This isn't so bad since there are only a
>> few pom files total.
>> 
>> 
>> I hope this helps,
>> Steven
>> 
>> 
>> 
>> 
>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>> 
>>> Here is Chris's original solution to give context. I think changing the
>>> REGEX might be enough to re-use the solution:
>>> 
>>> 1. On your local branch, find the *parent* of the first commit you want
>> to
>>> migrate onto the new master. If you were fully up-to-date before the
>>> repackaging commits went in, this will be Till's
>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.
>>> 
>>> 2. Run this magic command:
>>> 
>>>   git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>> 
>>> 3. Now fetch the new master, and create a new local branch from it:
>>> 
>>>   git switch master; git pull; git checkout -B newbranch
>>> 
>>> 4. Apply your tweaked patch:
>>> 
>>>   git am /tmp/my.patch
>>> 
>>> 
>>> Steven
>>> 
>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>>> 
>>>> I've tried doing this now on my branch.
>>>> As I feared, all of the files that are renamed/moved become conflicts
>>>> (just a few hundred conflicts in my case 😑).
>>>> I'm wondering if we could use a similar technique for what we did during
>>>> the summer (for the apache change) to get around this.
>>>> 
>>>> Steven
>>>> 
>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org> wrote:
>>>> 
>>>>> I’m not sure I completely understand what you are saying. Is this a
>>>>> temporary state that will get cleaned up later or is this supposed to
>>>>> stay this way (having "-fullstack" in the names)?
>>>>> 
>>>>> Thanks,
>>>>> Till
>>>>> 
>>>>> 
>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>>> 
>>>>> I'm not sure if it was necessary to rename it, but the original issue
>> is
>>>>>> that the hyracks repo itself has a folder named hyracks, that contains
>>>>>> hyracks. I thought this might confuse git if I did something like
>> make a
>>>>>> new temporary folder, move everything into that, and then rename it to
>>>>>> 'hyracks'.
>>>>>> 
>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>> Interesting!
>>>>>>> 
>>>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to the
>>>>>>> artifactId and the hyracks module.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Till
>>>>>>> 
>>>>>>> 
>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>>> 
>>>>>>> I've gone ahead and tried merging my topic branch with this change,
>>>>>>> and it
>>>>>>> 
>>>>>>>> turned out surprisingly well. I really didn't have many issues. I'll
>>>>>>>> summarize the process:
>>>>>>>> 
>>>>>>>> 1) Merge the change from asterixdb with your topic branch checked
>>>>>>>> out, so
>>>>>>>> just 'git merge hyracks-merge2'.
>>>>>>>> The only real conflict should be the pom, if you altered that. I
>>>>>>>> found it
>>>>>>>> easiest to just replicate my changes and take the upstream, rather
>>>>>>>> than
>>>>>>>> trying anything funny, since usually pom changes are not major.
>>>>>>>> 
>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>>>>>>> hyracks-local file:///home/...')
>>>>>>>> 
>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>>>>>>> hyracks-local/imaxon/hdfs')
>>>>>>>> This also worked pretty well, the only extra hiccup besides the pom
>>>>>>>> was
>>>>>>>> files I had created. Those appeared at the top level again after the
>>>>>>>> merge.
>>>>>>>> But, all you have to do is move them back down one folder into
>>>>>>>> hyracks-fullstack.
>>>>>>>> 
>>>>>>>> That's about it really. I went ahead and pushed this up to github as
>>>>>>>> well
>>>>>>>> so if anyone would like to take a look at the process or check out
>> the
>>>>>>>> branch to see what happened (at least for me), the branch is here:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> -Ian
>>>>>>>> 
>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>>>>>>> 
>>>>>>>> Chris found an issue with the way git histories were being handled
>> in
>>>>>>>> the
>>>>>>>> 
>>>>>>>>> way I merged things, so I have revised the proposed branch:
>>>>>>>>> 
>>>>>>>>> 
>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>>>>> 
>>>>>>>>> Basically I was trying to fit everything into one commit, because I
>>>>>>>>> thought at first that I could submit it to Gerrit that way. However
>>>>>>>>> that
>>>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
>>>>>>>>> every new
>>>>>>>>> commit from Hyracks as a new change. Splitting the commits of the
>>>>>>>>> repository merge fixes the issue.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> @Till, I think that creating a textual patch would just be more
>>>>>>>>> work. If
>>>>>>>>> I
>>>>>>>>> were to do it that way I would try fetching the Gerrit patch, and
>>>>>>>>> then
>>>>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
>>>>>>>>> master
>>>>>>>>> as
>>>>>>>>> the head.
>>>>>>>>> 
>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> To get existing patches in, could we just create a textual patch
>>>>>>>>> (e.g.
>>>>>>>>> 
>>>>>>>>>> from gerrit), apply that with the necessary -p option to a new
>> local
>>>>>>>>>> checkout of the merged repositories and submit a new review to
>>>>>>>>>> gerrit?
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> Till
>>>>>>>>>> 
>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi all,
>>>>>>>>>> 
>>>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>>>>>>>> repositories
>>>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you
>>>>>>>>>>> all can
>>>>>>>>>>> check it out here:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> You will likely have to do some ugly rebasing for whatever
>> changes
>>>>>>>>>>> you
>>>>>>>>>>> might have open once this gets done, since it moves asterixdb
>> down
>>>>>>>>>>> one
>>>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is
>> in
>>>>>>>>>>> a
>>>>>>>>>>> similar situation, though you would want to reapply your change
>> to
>>>>>>>>>>> the
>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
>>>>>>>>>>> like to
>>>>>>>>>>> 
>>>>>>>>>>> see
>>>>>>>>>> 
>>>>>>>>>> how this affects your branch please do try fetching the branch I
>>>>>>>>>>> linked
>>>>>>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>>>>>> 
>>>>>>>>>>> I'm still making sure all of the tests pass but nothing's failed
>> so
>>>>>>>>>>> far.
>>>>>>>>>>> Unless anyone has objections I think we should push this change
>>>>>>>>>>> either
>>>>>>>>>>> 
>>>>>>>>>>> this
>>>>>>>>>> 
>>>>>>>>>> week or early next week.
>>>>>>>>>>> 
>>>>>>>>>>> Let me know what you all think.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> - Ian
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>> 
>>> 
>> 

Best regards,
Ildar

Re: The Great Merge

Posted by Taewoo Kim <wa...@gmail.com>.
Hello Steven,

I tried to use Ian's straightforward method and it generated a few hundred
conflicts since I touched a lot of test files. I tried to go through one by
one and it took long time. So, I want to explore your method, too. The one
thing that I don't understand on your method is "finding the parent". Is
that the last commit of the master branch that was merged into my local
branch? Thank you.

Best,
Taewoo

On Mon, Apr 4, 2016 at 1:12 PM, Steven Jacobs <sj...@ucr.edu> wrote:

> It seems that I might be the only one concerned here, but it seems like
> there should be others, so I am continuing this thread.
>
> I modified the perl REGEX from Chris' summer solution, and it works!
>
> Once Ian has merged master:
>
> 1. On your local branch, find the *parent* of the first commit you want to
> migrate onto the new master, e.g. de6e0da24c26037967eb9a937d2c77c6c43e8761
>
> 2. Run this magic command:
>
>    git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>
> 3. Now fetch master, and create a new local branch from it:
>
>    git switch master; git pull; git checkout -B newbranch
>
> 4. Apply your tweaked patch:
>
>    git am /tmp/my.patch
>
>
> This recognized ALL of my file moves/renames and applied them correctly. It
> leaves only two issues:
> 1) Something similar will probably need to be done for Hyracks changes
> 2) My pom changes didn't apply. This isn't so bad since there are only a
> few pom files total.
>
>
> I hope this helps,
> Steven
>
>
>
>
> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>
> > Here is Chris's original solution to give context. I think changing the
> > REGEX might be enough to re-use the solution:
> >
> > 1. On your local branch, find the *parent* of the first commit you want
> to
> > migrate onto the new master. If you were fully up-to-date before the
> > repackaging commits went in, this will be Till's
> > change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.
> >
> > 2. Run this magic command:
> >
> >    git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> > perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> >
> > 3. Now fetch the new master, and create a new local branch from it:
> >
> >    git switch master; git pull; git checkout -B newbranch
> >
> > 4. Apply your tweaked patch:
> >
> >    git am /tmp/my.patch
> >
> >
> > Steven
> >
> > On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu> wrote:
> >
> >> I've tried doing this now on my branch.
> >> As I feared, all of the files that are renamed/moved become conflicts
> >> (just a few hundred conflicts in my case 😑).
> >> I'm wondering if we could use a similar technique for what we did during
> >> the summer (for the apache change) to get around this.
> >>
> >> Steven
> >>
> >> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org> wrote:
> >>
> >>> I’m not sure I completely understand what you are saying. Is this a
> >>> temporary state that will get cleaned up later or is this supposed to
> >>> stay this way (having "-fullstack" in the names)?
> >>>
> >>> Thanks,
> >>> Till
> >>>
> >>>
> >>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> >>>
> >>> I'm not sure if it was necessary to rename it, but the original issue
> is
> >>>> that the hyracks repo itself has a folder named hyracks, that contains
> >>>> hyracks. I thought this might confuse git if I did something like
> make a
> >>>> new temporary folder, move everything into that, and then rename it to
> >>>> 'hyracks'.
> >>>>
> >>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
> >>>> wrote:
> >>>>
> >>>> Interesting!
> >>>>>
> >>>>> One thing I’m wondering about is why you’ve added "-fullstack" to the
> >>>>> artifactId and the hyracks module.
> >>>>>
> >>>>> Cheers,
> >>>>> Till
> >>>>>
> >>>>>
> >>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> >>>>>
> >>>>> I've gone ahead and tried merging my topic branch with this change,
> >>>>> and it
> >>>>>
> >>>>>> turned out surprisingly well. I really didn't have many issues. I'll
> >>>>>> summarize the process:
> >>>>>>
> >>>>>> 1) Merge the change from asterixdb with your topic branch checked
> >>>>>> out, so
> >>>>>> just 'git merge hyracks-merge2'.
> >>>>>> The only real conflict should be the pom, if you altered that. I
> >>>>>> found it
> >>>>>> easiest to just replicate my changes and take the upstream, rather
> >>>>>> than
> >>>>>> trying anything funny, since usually pom changes are not major.
> >>>>>>
> >>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
> >>>>>> hyracks-local file:///home/...')
> >>>>>>
> >>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
> >>>>>> hyracks-local/imaxon/hdfs')
> >>>>>> This also worked pretty well, the only extra hiccup besides the pom
> >>>>>> was
> >>>>>> files I had created. Those appeared at the top level again after the
> >>>>>> merge.
> >>>>>> But, all you have to do is move them back down one folder into
> >>>>>> hyracks-fullstack.
> >>>>>>
> >>>>>> That's about it really. I went ahead and pushed this up to github as
> >>>>>> well
> >>>>>> so if anyone would like to take a look at the process or check out
> the
> >>>>>> branch to see what happened (at least for me), the branch is here:
> >>>>>>
> >>>>>>
> >>>>>>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -Ian
> >>>>>>
> >>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
> >>>>>>
> >>>>>> Chris found an issue with the way git histories were being handled
> in
> >>>>>> the
> >>>>>>
> >>>>>>> way I merged things, so I have revised the proposed branch:
> >>>>>>>
> >>>>>>>
> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> >>>>>>>
> >>>>>>> Basically I was trying to fit everything into one commit, because I
> >>>>>>> thought at first that I could submit it to Gerrit that way. However
> >>>>>>> that
> >>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
> >>>>>>> every new
> >>>>>>> commit from Hyracks as a new change. Splitting the commits of the
> >>>>>>> repository merge fixes the issue.
> >>>>>>>
> >>>>>>>
> >>>>>>> @Till, I think that creating a textual patch would just be more
> >>>>>>> work. If
> >>>>>>> I
> >>>>>>> were to do it that way I would try fetching the Gerrit patch, and
> >>>>>>> then
> >>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
> >>>>>>> master
> >>>>>>> as
> >>>>>>> the head.
> >>>>>>>
> >>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> To get existing patches in, could we just create a textual patch
> >>>>>>> (e.g.
> >>>>>>>
> >>>>>>>> from gerrit), apply that with the necessary -p option to a new
> local
> >>>>>>>> checkout of the merged repositories and submit a new review to
> >>>>>>>> gerrit?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Till
> >>>>>>>>
> >>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> >>>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
> >>>>>>>>> repositories
> >>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you
> >>>>>>>>> all can
> >>>>>>>>> check it out here:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> You will likely have to do some ugly rebasing for whatever
> changes
> >>>>>>>>> you
> >>>>>>>>> might have open once this gets done, since it moves asterixdb
> down
> >>>>>>>>> one
> >>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is
> in
> >>>>>>>>> a
> >>>>>>>>> similar situation, though you would want to reapply your change
> to
> >>>>>>>>> the
> >>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
> >>>>>>>>> like to
> >>>>>>>>>
> >>>>>>>>> see
> >>>>>>>>
> >>>>>>>> how this affects your branch please do try fetching the branch I
> >>>>>>>>> linked
> >>>>>>>>> above and testing it out on a copy of your topic branch.
> >>>>>>>>>
> >>>>>>>>> I'm still making sure all of the tests pass but nothing's failed
> so
> >>>>>>>>> far.
> >>>>>>>>> Unless anyone has objections I think we should push this change
> >>>>>>>>> either
> >>>>>>>>>
> >>>>>>>>> this
> >>>>>>>>
> >>>>>>>> week or early next week.
> >>>>>>>>>
> >>>>>>>>> Let me know what you all think.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> - Ian
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>
> >
>

Re: The Great Merge

Posted by Steven Jacobs <sj...@ucr.edu>.
It seems that I might be the only one concerned here, but it seems like
there should be others, so I am continuing this thread.

I modified the perl REGEX from Chris' summer solution, and it works!

Once Ian has merged master:

1. On your local branch, find the *parent* of the first commit you want to
migrate onto the new master, e.g. de6e0da24c26037967eb9a937d2c77c6c43e8761

2. Run this magic command:

   git format-patch --stdout de6e0da24c26037967eb9a937d2c77c6c43e8761 |
perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch

3. Now fetch master, and create a new local branch from it:

   git switch master; git pull; git checkout -B newbranch

4. Apply your tweaked patch:

   git am /tmp/my.patch


This recognized ALL of my file moves/renames and applied them correctly. It
leaves only two issues:
1) Something similar will probably need to be done for Hyracks changes
2) My pom changes didn't apply. This isn't so bad since there are only a
few pom files total.


I hope this helps,
Steven




On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sj...@ucr.edu> wrote:

> Here is Chris's original solution to give context. I think changing the
> REGEX might be enough to re-use the solution:
>
> 1. On your local branch, find the *parent* of the first commit you want to
> migrate onto the new master. If you were fully up-to-date before the
> repackaging commits went in, this will be Till's
> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.
>
> 2. Run this magic command:
>
>    git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>
> 3. Now fetch the new master, and create a new local branch from it:
>
>    git switch master; git pull; git checkout -B newbranch
>
> 4. Apply your tweaked patch:
>
>    git am /tmp/my.patch
>
>
> Steven
>
> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu> wrote:
>
>> I've tried doing this now on my branch.
>> As I feared, all of the files that are renamed/moved become conflicts
>> (just a few hundred conflicts in my case 😑).
>> I'm wondering if we could use a similar technique for what we did during
>> the summer (for the apache change) to get around this.
>>
>> Steven
>>
>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org> wrote:
>>
>>> I’m not sure I completely understand what you are saying. Is this a
>>> temporary state that will get cleaned up later or is this supposed to
>>> stay this way (having "-fullstack" in the names)?
>>>
>>> Thanks,
>>> Till
>>>
>>>
>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>
>>> I'm not sure if it was necessary to rename it, but the original issue is
>>>> that the hyracks repo itself has a folder named hyracks, that contains
>>>> hyracks. I thought this might confuse git if I did something like make a
>>>> new temporary folder, move everything into that, and then rename it to
>>>> 'hyracks'.
>>>>
>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org>
>>>> wrote:
>>>>
>>>> Interesting!
>>>>>
>>>>> One thing I’m wondering about is why you’ve added "-fullstack" to the
>>>>> artifactId and the hyracks module.
>>>>>
>>>>> Cheers,
>>>>> Till
>>>>>
>>>>>
>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>
>>>>> I've gone ahead and tried merging my topic branch with this change,
>>>>> and it
>>>>>
>>>>>> turned out surprisingly well. I really didn't have many issues. I'll
>>>>>> summarize the process:
>>>>>>
>>>>>> 1) Merge the change from asterixdb with your topic branch checked
>>>>>> out, so
>>>>>> just 'git merge hyracks-merge2'.
>>>>>> The only real conflict should be the pom, if you altered that. I
>>>>>> found it
>>>>>> easiest to just replicate my changes and take the upstream, rather
>>>>>> than
>>>>>> trying anything funny, since usually pom changes are not major.
>>>>>>
>>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>>>>> hyracks-local file:///home/...')
>>>>>>
>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>>>>> hyracks-local/imaxon/hdfs')
>>>>>> This also worked pretty well, the only extra hiccup besides the pom
>>>>>> was
>>>>>> files I had created. Those appeared at the top level again after the
>>>>>> merge.
>>>>>> But, all you have to do is move them back down one folder into
>>>>>> hyracks-fullstack.
>>>>>>
>>>>>> That's about it really. I went ahead and pushed this up to github as
>>>>>> well
>>>>>> so if anyone would like to take a look at the process or check out the
>>>>>> branch to see what happened (at least for me), the branch is here:
>>>>>>
>>>>>>
>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>>>
>>>>>> Thanks,
>>>>>> -Ian
>>>>>>
>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>>>>>
>>>>>> Chris found an issue with the way git histories were being handled in
>>>>>> the
>>>>>>
>>>>>>> way I merged things, so I have revised the proposed branch:
>>>>>>>
>>>>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>>>
>>>>>>> Basically I was trying to fit everything into one commit, because I
>>>>>>> thought at first that I could submit it to Gerrit that way. However
>>>>>>> that
>>>>>>> doesn't work for other reasons, basically Gerrit tries to treat
>>>>>>> every new
>>>>>>> commit from Hyracks as a new change. Splitting the commits of the
>>>>>>> repository merge fixes the issue.
>>>>>>>
>>>>>>>
>>>>>>> @Till, I think that creating a textual patch would just be more
>>>>>>> work. If
>>>>>>> I
>>>>>>> were to do it that way I would try fetching the Gerrit patch, and
>>>>>>> then
>>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
>>>>>>> master
>>>>>>> as
>>>>>>> the head.
>>>>>>>
>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> To get existing patches in, could we just create a textual patch
>>>>>>> (e.g.
>>>>>>>
>>>>>>>> from gerrit), apply that with the necessary -p option to a new local
>>>>>>>> checkout of the merged repositories and submit a new review to
>>>>>>>> gerrit?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Till
>>>>>>>>
>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>>>>>> repositories
>>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you
>>>>>>>>> all can
>>>>>>>>> check it out here:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>>>
>>>>>>>>
>>>>>>>>> You will likely have to do some ugly rebasing for whatever changes
>>>>>>>>> you
>>>>>>>>> might have open once this gets done, since it moves asterixdb down
>>>>>>>>> one
>>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is in
>>>>>>>>> a
>>>>>>>>> similar situation, though you would want to reapply your change to
>>>>>>>>> the
>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would
>>>>>>>>> like to
>>>>>>>>>
>>>>>>>>> see
>>>>>>>>
>>>>>>>> how this affects your branch please do try fetching the branch I
>>>>>>>>> linked
>>>>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>>>>
>>>>>>>>> I'm still making sure all of the tests pass but nothing's failed so
>>>>>>>>> far.
>>>>>>>>> Unless anyone has objections I think we should push this change
>>>>>>>>> either
>>>>>>>>>
>>>>>>>>> this
>>>>>>>>
>>>>>>>> week or early next week.
>>>>>>>>>
>>>>>>>>> Let me know what you all think.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> - Ian
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>
>

Re: The Great Merge

Posted by Steven Jacobs <sj...@ucr.edu>.
Here is Chris's original solution to give context. I think changing the
REGEX might be enough to re-use the solution:

1. On your local branch, find the *parent* of the first commit you want to
migrate onto the new master. If you were fully up-to-date before the
repackaging commits went in, this will be Till's
change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use that here.

2. Run this magic command:

   git format-patch --stdout 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch

3. Now fetch the new master, and create a new local branch from it:

   git switch master; git pull; git checkout -B newbranch

4. Apply your tweaked patch:

   git am /tmp/my.patch


Steven

On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sj...@ucr.edu> wrote:

> I've tried doing this now on my branch.
> As I feared, all of the files that are renamed/moved become conflicts
> (just a few hundred conflicts in my case 😑).
> I'm wondering if we could use a similar technique for what we did during
> the summer (for the apache change) to get around this.
>
> Steven
>
> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org> wrote:
>
>> I’m not sure I completely understand what you are saying. Is this a
>> temporary state that will get cleaned up later or is this supposed to
>> stay this way (having "-fullstack" in the names)?
>>
>> Thanks,
>> Till
>>
>>
>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>
>> I'm not sure if it was necessary to rename it, but the original issue is
>>> that the hyracks repo itself has a folder named hyracks, that contains
>>> hyracks. I thought this might confuse git if I did something like make a
>>> new temporary folder, move everything into that, and then rename it to
>>> 'hyracks'.
>>>
>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org> wrote:
>>>
>>> Interesting!
>>>>
>>>> One thing I’m wondering about is why you’ve added "-fullstack" to the
>>>> artifactId and the hyracks module.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>>
>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>
>>>> I've gone ahead and tried merging my topic branch with this change, and
>>>> it
>>>>
>>>>> turned out surprisingly well. I really didn't have many issues. I'll
>>>>> summarize the process:
>>>>>
>>>>> 1) Merge the change from asterixdb with your topic branch checked out,
>>>>> so
>>>>> just 'git merge hyracks-merge2'.
>>>>> The only real conflict should be the pom, if you altered that. I found
>>>>> it
>>>>> easiest to just replicate my changes and take the upstream, rather than
>>>>> trying anything funny, since usually pom changes are not major.
>>>>>
>>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>>>> hyracks-local file:///home/...')
>>>>>
>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>>>> hyracks-local/imaxon/hdfs')
>>>>> This also worked pretty well, the only extra hiccup besides the pom was
>>>>> files I had created. Those appeared at the top level again after the
>>>>> merge.
>>>>> But, all you have to do is move them back down one folder into
>>>>> hyracks-fullstack.
>>>>>
>>>>> That's about it really. I went ahead and pushed this up to github as
>>>>> well
>>>>> so if anyone would like to take a look at the process or check out the
>>>>> branch to see what happened (at least for me), the branch is here:
>>>>>
>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>>
>>>>> Thanks,
>>>>> -Ian
>>>>>
>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>>>>
>>>>> Chris found an issue with the way git histories were being handled in
>>>>> the
>>>>>
>>>>>> way I merged things, so I have revised the proposed branch:
>>>>>>
>>>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>>
>>>>>> Basically I was trying to fit everything into one commit, because I
>>>>>> thought at first that I could submit it to Gerrit that way. However
>>>>>> that
>>>>>> doesn't work for other reasons, basically Gerrit tries to treat every
>>>>>> new
>>>>>> commit from Hyracks as a new change. Splitting the commits of the
>>>>>> repository merge fixes the issue.
>>>>>>
>>>>>>
>>>>>> @Till, I think that creating a textual patch would just be more work.
>>>>>> If
>>>>>> I
>>>>>> were to do it that way I would try fetching the Gerrit patch, and then
>>>>>> cherry-picking it onto a new branch that has the hyracks+asterix
>>>>>> master
>>>>>> as
>>>>>> the head.
>>>>>>
>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> To get existing patches in, could we just create a textual patch (e.g.
>>>>>>
>>>>>>> from gerrit), apply that with the necessary -p option to a new local
>>>>>>> checkout of the merged repositories and submit a new review to
>>>>>>> gerrit?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Till
>>>>>>>
>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>>>>> repositories
>>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you all
>>>>>>>> can
>>>>>>>> check it out here:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>>
>>>>>>>
>>>>>>>> You will likely have to do some ugly rebasing for whatever changes
>>>>>>>> you
>>>>>>>> might have open once this gets done, since it moves asterixdb down
>>>>>>>> one
>>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is in a
>>>>>>>> similar situation, though you would want to reapply your change to
>>>>>>>> the
>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would like
>>>>>>>> to
>>>>>>>>
>>>>>>>> see
>>>>>>>
>>>>>>> how this affects your branch please do try fetching the branch I
>>>>>>>> linked
>>>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>>>
>>>>>>>> I'm still making sure all of the tests pass but nothing's failed so
>>>>>>>> far.
>>>>>>>> Unless anyone has objections I think we should push this change
>>>>>>>> either
>>>>>>>>
>>>>>>>> this
>>>>>>>
>>>>>>> week or early next week.
>>>>>>>>
>>>>>>>> Let me know what you all think.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> - Ian
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>

Re: The Great Merge

Posted by Steven Jacobs <sj...@ucr.edu>.
I've tried doing this now on my branch.
As I feared, all of the files that are renamed/moved become conflicts (just
a few hundred conflicts in my case 😑).
I'm wondering if we could use a similar technique for what we did during
the summer (for the apache change) to get around this.

Steven

On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <ti...@apache.org> wrote:

> I’m not sure I completely understand what you are saying. Is this a
> temporary state that will get cleaned up later or is this supposed to
> stay this way (having "-fullstack" in the names)?
>
> Thanks,
> Till
>
>
> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>
> I'm not sure if it was necessary to rename it, but the original issue is
>> that the hyracks repo itself has a folder named hyracks, that contains
>> hyracks. I thought this might confuse git if I did something like make a
>> new temporary folder, move everything into that, and then rename it to
>> 'hyracks'.
>>
>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org> wrote:
>>
>> Interesting!
>>>
>>> One thing I’m wondering about is why you’ve added "-fullstack" to the
>>> artifactId and the hyracks module.
>>>
>>> Cheers,
>>> Till
>>>
>>>
>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>
>>> I've gone ahead and tried merging my topic branch with this change, and
>>> it
>>>
>>>> turned out surprisingly well. I really didn't have many issues. I'll
>>>> summarize the process:
>>>>
>>>> 1) Merge the change from asterixdb with your topic branch checked out,
>>>> so
>>>> just 'git merge hyracks-merge2'.
>>>> The only real conflict should be the pom, if you altered that. I found
>>>> it
>>>> easiest to just replicate my changes and take the upstream, rather than
>>>> trying anything funny, since usually pom changes are not major.
>>>>
>>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>>> hyracks-local file:///home/...')
>>>>
>>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>>> hyracks-local/imaxon/hdfs')
>>>> This also worked pretty well, the only extra hiccup besides the pom was
>>>> files I had created. Those appeared at the top level again after the
>>>> merge.
>>>> But, all you have to do is move them back down one folder into
>>>> hyracks-fullstack.
>>>>
>>>> That's about it really. I went ahead and pushed this up to github as
>>>> well
>>>> so if anyone would like to take a look at the process or check out the
>>>> branch to see what happened (at least for me), the branch is here:
>>>>
>>>>
>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>
>>>> Thanks,
>>>> -Ian
>>>>
>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>>>
>>>> Chris found an issue with the way git histories were being handled in
>>>> the
>>>>
>>>>> way I merged things, so I have revised the proposed branch:
>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>
>>>>> Basically I was trying to fit everything into one commit, because I
>>>>> thought at first that I could submit it to Gerrit that way. However
>>>>> that
>>>>> doesn't work for other reasons, basically Gerrit tries to treat every
>>>>> new
>>>>> commit from Hyracks as a new change. Splitting the commits of the
>>>>> repository merge fixes the issue.
>>>>>
>>>>>
>>>>> @Till, I think that creating a textual patch would just be more work.
>>>>> If
>>>>> I
>>>>> were to do it that way I would try fetching the Gerrit patch, and then
>>>>> cherry-picking it onto a new branch that has the hyracks+asterix master
>>>>> as
>>>>> the head.
>>>>>
>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org>
>>>>> wrote:
>>>>>
>>>>> To get existing patches in, could we just create a textual patch (e.g.
>>>>>
>>>>>> from gerrit), apply that with the necessary -p option to a new local
>>>>>> checkout of the merged repositories and submit a new review to gerrit?
>>>>>>
>>>>>> Thanks,
>>>>>> Till
>>>>>>
>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>>>> repositories
>>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you all
>>>>>>> can
>>>>>>> check it out here:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>
>>>>>>
>>>>>>> You will likely have to do some ugly rebasing for whatever changes
>>>>>>> you
>>>>>>> might have open once this gets done, since it moves asterixdb down
>>>>>>> one
>>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is in a
>>>>>>> similar situation, though you would want to reapply your change to
>>>>>>> the
>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would like
>>>>>>> to
>>>>>>>
>>>>>>> see
>>>>>>
>>>>>> how this affects your branch please do try fetching the branch I
>>>>>>> linked
>>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>>
>>>>>>> I'm still making sure all of the tests pass but nothing's failed so
>>>>>>> far.
>>>>>>> Unless anyone has objections I think we should push this change
>>>>>>> either
>>>>>>>
>>>>>>> this
>>>>>>
>>>>>> week or early next week.
>>>>>>>
>>>>>>> Let me know what you all think.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> - Ian
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>

Re: The Great Merge

Posted by Till Westmann <ti...@apache.org>.
I’m not sure I completely understand what you are saying. Is this a
temporary state that will get cleaned up later or is this supposed to
stay this way (having "-fullstack" in the names)?

Thanks,
Till

On 31 Mar 2016, at 19:39, Ian Maxon wrote:

> I'm not sure if it was necessary to rename it, but the original issue 
> is
> that the hyracks repo itself has a folder named hyracks, that contains
> hyracks. I thought this might confuse git if I did something like make 
> a
> new temporary folder, move everything into that, and then rename it to
> 'hyracks'.
>
> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org> 
> wrote:
>
>> Interesting!
>>
>> One thing I’m wondering about is why you’ve added "-fullstack" to 
>> the
>> artifactId and the hyracks module.
>>
>> Cheers,
>> Till
>>
>>
>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>
>> I've gone ahead and tried merging my topic branch with this change, 
>> and it
>>> turned out surprisingly well. I really didn't have many issues. I'll
>>> summarize the process:
>>>
>>> 1) Merge the change from asterixdb with your topic branch checked 
>>> out, so
>>> just 'git merge hyracks-merge2'.
>>> The only real conflict should be the pom, if you altered that. I 
>>> found it
>>> easiest to just replicate my changes and take the upstream, rather 
>>> than
>>> trying anything funny, since usually pom changes are not major.
>>>
>>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>>> hyracks-local file:///home/...')
>>>
>>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>>> hyracks-local/imaxon/hdfs')
>>> This also worked pretty well, the only extra hiccup besides the pom 
>>> was
>>> files I had created. Those appeared at the top level again after the
>>> merge.
>>> But, all you have to do is move them back down one folder into
>>> hyracks-fullstack.
>>>
>>> That's about it really. I went ahead and pushed this up to github as 
>>> well
>>> so if anyone would like to take a look at the process or check out 
>>> the
>>> branch to see what happened (at least for me), the branch is here:
>>>
>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>
>>> Thanks,
>>> -Ian
>>>
>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>>
>>> Chris found an issue with the way git histories were being handled 
>>> in the
>>>> way I merged things, so I have revised the proposed branch:
>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>
>>>> Basically I was trying to fit everything into one commit, because I
>>>> thought at first that I could submit it to Gerrit that way. However 
>>>> that
>>>> doesn't work for other reasons, basically Gerrit tries to treat 
>>>> every new
>>>> commit from Hyracks as a new change. Splitting the commits of the
>>>> repository merge fixes the issue.
>>>>
>>>>
>>>> @Till, I think that creating a textual patch would just be more 
>>>> work. If
>>>> I
>>>> were to do it that way I would try fetching the Gerrit patch, and 
>>>> then
>>>> cherry-picking it onto a new branch that has the hyracks+asterix 
>>>> master
>>>> as
>>>> the head.
>>>>
>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org> 
>>>> wrote:
>>>>
>>>> To get existing patches in, could we just create a textual patch 
>>>> (e.g.
>>>>> from gerrit), apply that with the necessary -p option to a new 
>>>>> local
>>>>> checkout of the merged repositories and submit a new review to 
>>>>> gerrit?
>>>>>
>>>>> Thanks,
>>>>> Till
>>>>>
>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>>
>>>>> Hi all,
>>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>>> repositories
>>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you 
>>>>>> all can
>>>>>> check it out here:
>>>>>>
>>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>
>>>>>>
>>>>>> You will likely have to do some ugly rebasing for whatever 
>>>>>> changes you
>>>>>> might have open once this gets done, since it moves asterixdb 
>>>>>> down one
>>>>>> folder and swaps out pom.xml in the repository root. Hyracks is 
>>>>>> in a
>>>>>> similar situation, though you would want to reapply your change 
>>>>>> to the
>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would 
>>>>>> like to
>>>>>>
>>>>> see
>>>>>
>>>>>> how this affects your branch please do try fetching the branch I 
>>>>>> linked
>>>>>> above and testing it out on a copy of your topic branch.
>>>>>>
>>>>>> I'm still making sure all of the tests pass but nothing's failed 
>>>>>> so
>>>>>> far.
>>>>>> Unless anyone has objections I think we should push this change 
>>>>>> either
>>>>>>
>>>>> this
>>>>>
>>>>>> week or early next week.
>>>>>>
>>>>>> Let me know what you all think.
>>>>>>
>>>>>> Thanks,
>>>>>> - Ian
>>>>>>
>>>>>
>>>>>
>>>>
>>>>

Re: The Great Merge

Posted by Ian Maxon <im...@uci.edu>.
I'm not sure if it was necessary to rename it, but the original issue is
that the hyracks repo itself has a folder named hyracks, that contains
hyracks. I thought this might confuse git if I did something like make a
new temporary folder, move everything into that, and then rename it to
'hyracks'.

On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <ti...@apache.org> wrote:

> Interesting!
>
> One thing I’m wondering about is why you’ve added "-fullstack" to the
> artifactId and the hyracks module.
>
> Cheers,
> Till
>
>
> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>
> I've gone ahead and tried merging my topic branch with this change, and it
>> turned out surprisingly well. I really didn't have many issues. I'll
>> summarize the process:
>>
>> 1) Merge the change from asterixdb with your topic branch checked out, so
>> just 'git merge hyracks-merge2'.
>> The only real conflict should be the pom, if you altered that. I found it
>> easiest to just replicate my changes and take the upstream, rather than
>> trying anything funny, since usually pom changes are not major.
>>
>> 2) Add your hyracks folder as a remote (for me, 'git remote add
>> hyracks-local file:///home/...')
>>
>> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
>> hyracks-local/imaxon/hdfs')
>> This also worked pretty well, the only extra hiccup besides the pom was
>> files I had created. Those appeared at the top level again after the
>> merge.
>> But, all you have to do is move them back down one folder into
>> hyracks-fullstack.
>>
>> That's about it really. I went ahead and pushed this up to github as well
>> so if anyone would like to take a look at the process or check out the
>> branch to see what happened (at least for me), the branch is here:
>>
>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>
>> Thanks,
>> -Ian
>>
>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>>
>> Chris found an issue with the way git histories were being handled in the
>>> way I merged things, so I have revised the proposed branch:
>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>
>>> Basically I was trying to fit everything into one commit, because I
>>> thought at first that I could submit it to Gerrit that way. However that
>>> doesn't work for other reasons, basically Gerrit tries to treat every new
>>> commit from Hyracks as a new change. Splitting the commits of the
>>> repository merge fixes the issue.
>>>
>>>
>>> @Till, I think that creating a textual patch would just be more work. If
>>> I
>>> were to do it that way I would try fetching the Gerrit patch, and then
>>> cherry-picking it onto a new branch that has the hyracks+asterix master
>>> as
>>> the head.
>>>
>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org> wrote:
>>>
>>> To get existing patches in, could we just create a textual patch (e.g.
>>>> from gerrit), apply that with the necessary -p option to a new local
>>>> checkout of the merged repositories and submit a new review to gerrit?
>>>>
>>>> Thanks,
>>>> Till
>>>>
>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>>
>>>> Hi all,
>>>>> I went ahead preliminarily merged the Hyracks and AsterixDB
>>>>> repositories
>>>>> into one. Unfortunately this can't be reviewed in Gerrit so you all can
>>>>> check it out here:
>>>>>
>>>>>
>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>
>>>>>
>>>>> You will likely have to do some ugly rebasing for whatever changes you
>>>>> might have open once this gets done, since it moves asterixdb down one
>>>>> folder and swaps out pom.xml in the repository root. Hyracks is in a
>>>>> similar situation, though you would want to reapply your change to the
>>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would like to
>>>>>
>>>> see
>>>>
>>>>> how this affects your branch please do try fetching the branch I linked
>>>>> above and testing it out on a copy of your topic branch.
>>>>>
>>>>> I'm still making sure all of the tests pass but nothing's failed so
>>>>> far.
>>>>> Unless anyone has objections I think we should push this change either
>>>>>
>>>> this
>>>>
>>>>> week or early next week.
>>>>>
>>>>> Let me know what you all think.
>>>>>
>>>>> Thanks,
>>>>> - Ian
>>>>>
>>>>
>>>>
>>>
>>>

Re: The Great Merge

Posted by Till Westmann <ti...@apache.org>.
Interesting!

One thing I’m wondering about is why you’ve added "-fullstack" to 
the artifactId and the hyracks module.

Cheers,
Till

On 31 Mar 2016, at 17:21, Ian Maxon wrote:

> I've gone ahead and tried merging my topic branch with this change, 
> and it
> turned out surprisingly well. I really didn't have many issues. I'll
> summarize the process:
>
> 1) Merge the change from asterixdb with your topic branch checked out, 
> so
> just 'git merge hyracks-merge2'.
> The only real conflict should be the pom, if you altered that. I found 
> it
> easiest to just replicate my changes and take the upstream, rather 
> than
> trying anything funny, since usually pom changes are not major.
>
> 2) Add your hyracks folder as a remote (for me, 'git remote add
> hyracks-local file:///home/...')
>
> 3) Merge your hyracks topic branch into asterixdb ( ' git merge
> hyracks-local/imaxon/hdfs')
> This also worked pretty well, the only extra hiccup besides the pom 
> was
> files I had created. Those appeared at the top level again after the 
> merge.
> But, all you have to do is move them back down one folder into
> hyracks-fullstack.
>
> That's about it really. I went ahead and pushed this up to github as 
> well
> so if anyone would like to take a look at the process or check out the
> branch to see what happened (at least for me), the branch is here:
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>
> Thanks,
> -Ian
>
> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:
>
>> Chris found an issue with the way git histories were being handled in 
>> the
>> way I merged things, so I have revised the proposed branch:
>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>
>> Basically I was trying to fit everything into one commit, because I
>> thought at first that I could submit it to Gerrit that way. However 
>> that
>> doesn't work for other reasons, basically Gerrit tries to treat every 
>> new
>> commit from Hyracks as a new change. Splitting the commits of the
>> repository merge fixes the issue.
>>
>>
>> @Till, I think that creating a textual patch would just be more work. 
>> If I
>> were to do it that way I would try fetching the Gerrit patch, and 
>> then
>> cherry-picking it onto a new branch that has the hyracks+asterix 
>> master as
>> the head.
>>
>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org> 
>> wrote:
>>
>>> To get existing patches in, could we just create a textual patch 
>>> (e.g.
>>> from gerrit), apply that with the necessary -p option to a new local
>>> checkout of the merged repositories and submit a new review to 
>>> gerrit?
>>>
>>> Thanks,
>>> Till
>>>
>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>>
>>>> Hi all,
>>>> I went ahead preliminarily merged the Hyracks and AsterixDB 
>>>> repositories
>>>> into one. Unfortunately this can't be reviewed in Gerrit so you all 
>>>> can
>>>> check it out here:
>>>>
>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>
>>>> You will likely have to do some ugly rebasing for whatever changes 
>>>> you
>>>> might have open once this gets done, since it moves asterixdb down 
>>>> one
>>>> folder and swaps out pom.xml in the repository root. Hyracks is in 
>>>> a
>>>> similar situation, though you would want to reapply your change to 
>>>> the
>>>> AsterixDB repo from Hyracks (which is a bit odd). If you would like 
>>>> to
>>> see
>>>> how this affects your branch please do try fetching the branch I 
>>>> linked
>>>> above and testing it out on a copy of your topic branch.
>>>>
>>>> I'm still making sure all of the tests pass but nothing's failed so 
>>>> far.
>>>> Unless anyone has objections I think we should push this change 
>>>> either
>>> this
>>>> week or early next week.
>>>>
>>>> Let me know what you all think.
>>>>
>>>> Thanks,
>>>> - Ian
>>>
>>
>>

Re: The Great Merge

Posted by Ian Maxon <im...@uci.edu>.
I've gone ahead and tried merging my topic branch with this change, and it
turned out surprisingly well. I really didn't have many issues. I'll
summarize the process:

1) Merge the change from asterixdb with your topic branch checked out, so
just 'git merge hyracks-merge2'.
The only real conflict should be the pom, if you altered that. I found it
easiest to just replicate my changes and take the upstream, rather than
trying anything funny, since usually pom changes are not major.

2) Add your hyracks folder as a remote (for me, 'git remote add
hyracks-local file:///home/...')

3) Merge your hyracks topic branch into asterixdb ( ' git merge
hyracks-local/imaxon/hdfs')
This also worked pretty well, the only extra hiccup besides the pom was
files I had created. Those appeared at the top level again after the merge.
But, all you have to do is move them back down one folder into
hyracks-fullstack.

That's about it really. I went ahead and pushed this up to github as well
so if anyone would like to take a look at the process or check out the
branch to see what happened (at least for me), the branch is here:
https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks

Thanks,
-Ian

On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <im...@uci.edu> wrote:

> Chris found an issue with the way git histories were being handled in the
> way I merged things, so I have revised the proposed branch:
> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>
> Basically I was trying to fit everything into one commit, because I
> thought at first that I could submit it to Gerrit that way. However that
> doesn't work for other reasons, basically Gerrit tries to treat every new
> commit from Hyracks as a new change. Splitting the commits of the
> repository merge fixes the issue.
>
>
> @Till, I think that creating a textual patch would just be more work. If I
> were to do it that way I would try fetching the Gerrit patch, and then
> cherry-picking it onto a new branch that has the hyracks+asterix master as
> the head.
>
> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org> wrote:
>
>> To get existing patches in, could we just create a textual patch (e.g.
>> from gerrit), apply that with the necessary -p option to a new local
>> checkout of the merged repositories and submit a new review to gerrit?
>>
>> Thanks,
>> Till
>>
>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>>
>> > Hi all,
>> > I went ahead preliminarily merged the Hyracks and AsterixDB repositories
>> > into one. Unfortunately this can't be reviewed in Gerrit so you all can
>> > check it out here:
>> >
>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>> >
>> > You will likely have to do some ugly rebasing for whatever changes you
>> > might have open once this gets done, since it moves asterixdb down one
>> > folder and swaps out pom.xml in the repository root. Hyracks is in a
>> > similar situation, though you would want to reapply your change to the
>> > AsterixDB repo from Hyracks (which is a bit odd). If you would like to
>> see
>> > how this affects your branch please do try fetching the branch I linked
>> > above and testing it out on a copy of your topic branch.
>> >
>> > I'm still making sure all of the tests pass but nothing's failed so far.
>> > Unless anyone has objections I think we should push this change either
>> this
>> > week or early next week.
>> >
>> > Let me know what you all think.
>> >
>> > Thanks,
>> > - Ian
>>
>
>

Re: The Great Merge

Posted by Ian Maxon <im...@uci.edu>.
Chris found an issue with the way git histories were being handled in the
way I merged things, so I have revised the proposed branch:
https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2

Basically I was trying to fit everything into one commit, because I thought
at first that I could submit it to Gerrit that way. However that doesn't
work for other reasons, basically Gerrit tries to treat every new commit
from Hyracks as a new change. Splitting the commits of the repository merge
fixes the issue.


@Till, I think that creating a textual patch would just be more work. If I
were to do it that way I would try fetching the Gerrit patch, and then
cherry-picking it onto a new branch that has the hyracks+asterix master as
the head.

On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <ti...@apache.org> wrote:

> To get existing patches in, could we just create a textual patch (e.g.
> from gerrit), apply that with the necessary -p option to a new local
> checkout of the merged repositories and submit a new review to gerrit?
>
> Thanks,
> Till
>
> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
>
> > Hi all,
> > I went ahead preliminarily merged the Hyracks and AsterixDB repositories
> > into one. Unfortunately this can't be reviewed in Gerrit so you all can
> > check it out here:
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> >
> > You will likely have to do some ugly rebasing for whatever changes you
> > might have open once this gets done, since it moves asterixdb down one
> > folder and swaps out pom.xml in the repository root. Hyracks is in a
> > similar situation, though you would want to reapply your change to the
> > AsterixDB repo from Hyracks (which is a bit odd). If you would like to
> see
> > how this affects your branch please do try fetching the branch I linked
> > above and testing it out on a copy of your topic branch.
> >
> > I'm still making sure all of the tests pass but nothing's failed so far.
> > Unless anyone has objections I think we should push this change either
> this
> > week or early next week.
> >
> > Let me know what you all think.
> >
> > Thanks,
> > - Ian
>

Re: The Great Merge

Posted by Till Westmann <ti...@apache.org>.
To get existing patches in, could we just create a textual patch (e.g.
from gerrit), apply that with the necessary -p option to a new local
checkout of the merged repositories and submit a new review to gerrit?

Thanks,
Till

On 30 Mar 2016, at 12:36, Ian Maxon wrote:

> Hi all,
> I went ahead preliminarily merged the Hyracks and AsterixDB repositories
> into one. Unfortunately this can't be reviewed in Gerrit so you all can
> check it out here:
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>
> You will likely have to do some ugly rebasing for whatever changes you
> might have open once this gets done, since it moves asterixdb down one
> folder and swaps out pom.xml in the repository root. Hyracks is in a
> similar situation, though you would want to reapply your change to the
> AsterixDB repo from Hyracks (which is a bit odd). If you would like to see
> how this affects your branch please do try fetching the branch I linked
> above and testing it out on a copy of your topic branch.
>
> I'm still making sure all of the tests pass but nothing's failed so far.
> Unless anyone has objections I think we should push this change either this
> week or early next week.
>
> Let me know what you all think.
>
> Thanks,
> - Ian