You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by liwei li <hi...@gmail.com> on 2022/05/13 02:46:11 UTC
[discuss] keep the commit history when adding a new engine version
Hi, guys
When we want to add support for a new version of an engine, simply copying files from an old version to a new directory will cause git commit history to be lost, making it difficult to find file change records, we can only go to look for changes in the old path, but we don't know it in which one, and the old may have been deleted. Is there a better way to keep it?
I recommend that we first rename the old version to the new one, and then make a new copy as the old version.
For example, if we want to add Spark 3.3, we can do the following:
1. Change the path of version 3.2 from v3.2 to v3.3
2. Create a copy of v3.2 from v3.3
3. Modify the v3.2 file to make it work for spark 3.3 correct.
What do you think of the above? Or is it necessary? Or if there is another better way?
Thank you.
Liwei Li
hililiwei@gmail.com
Re: [discuss] keep the commit history when adding a new engine version
Posted by Ryan Blue <bl...@tabular.io>.
Sounds reasonable to me. Let's try it the next time we add an engine
version.
On Fri, May 13, 2022 at 12:14 AM Kyle Bendickson <ky...@tabular.io> wrote:
> I agree this is a good point.
>
> The git history is not retained when we port the way we currently do.
>
> So +1 as I understand it, the latest version will generally be the one to
> have the most git commit history. Possibly looking back for changes that
> occurred due to some other version.
>
> Thanks Liwei!
>
> - Kyle
>
> On Fri, May 13, 2022 at 12:10 AM Rajarshi Sarkar <rs...@gmail.com>
> wrote:
>
>> +1, good point.
>>
>> Regards,
>> Rajarshi Sarkar
>>
>>
>> On Fri, May 13, 2022 at 9:03 AM Reo Lei <le...@gmail.com> wrote:
>>
>>> That is great ! +1 for this.
>>>
>>> liwei li <hi...@gmail.com> 于2022年5月13日周五 11:12写道：
>>>
>>>> Correct a clerical errr:
>>>>
>>>> 3. Modify the v3.3 files to make it work for spark 3.3 correct.
>>>>
>>>> Liwei Li
>>>> ------------------------------
>>>> *From:* Steven Wu <st...@gmail.com>
>>>> *Sent:* Friday, May 13, 2022 11:06:26 AM
>>>> *To:* dev@iceberg.apache.org <de...@iceberg.apache.org>
>>>> *Subject:* Re: [discuss] keep the commit history when adding a new
>>>> engine version
>>>>
>>>> This is a good point. +1 for the proposal.
>>>>
>>>> On Thu, May 12, 2022 at 7:46 PM liwei li <hi...@gmail.com> wrote:
>>>>
>>>> Hi, guys
>>>> When we want to add support for a new version of an engine, simply
>>>> copying files from an old version to a new directory will cause git commit
>>>> history to be lost, making it difficult to find file change records, we can
>>>> only go to look for changes in the old path, but we don't know it in which
>>>> one, and the old may have been deleted. Is there a better way to keep it?
>>>> I recommend that we first rename the old version to the new one, and
>>>> then make a new copy as the old version.
>>>> For example, if we want to add Spark 3.3, we can do the following:
>>>> 1. Change the path of version 3.2 from v3.2 to v3.3
>>>> 2. Create a copy of v3.2 from v3.3
>>>> 3. Modify the v3.2 file to make it work for spark 3.3 correct.
>>>> What do you think of the above? Or is it necessary? Or if there is
>>>> another better way?
>>>> Thank you.
>>>>
>>>> Liwei Li
>>>> hililiwei@gmail.com
>>>>
>>>>
--
Ryan Blue
Tabular
Re: [discuss] keep the commit history when adding a new engine version
Posted by Kyle Bendickson <ky...@tabular.io>.
I agree this is a good point.
The git history is not retained when we port the way we currently do.
So +1 as I understand it, the latest version will generally be the one to
have the most git commit history. Possibly looking back for changes that
occurred due to some other version.
Thanks Liwei!
- Kyle
On Fri, May 13, 2022 at 12:10 AM Rajarshi Sarkar <rs...@gmail.com>
wrote:
> +1, good point.
>
> Regards,
> Rajarshi Sarkar
>
>
> On Fri, May 13, 2022 at 9:03 AM Reo Lei <le...@gmail.com> wrote:
>
>> That is great ! +1 for this.
>>
>> liwei li <hi...@gmail.com> 于2022年5月13日周五 11:12写道：
>>
>>> Correct a clerical errr:
>>>
>>> 3. Modify the v3.3 files to make it work for spark 3.3 correct.
>>>
>>> Liwei Li
>>> ------------------------------
>>> *From:* Steven Wu <st...@gmail.com>
>>> *Sent:* Friday, May 13, 2022 11:06:26 AM
>>> *To:* dev@iceberg.apache.org <de...@iceberg.apache.org>
>>> *Subject:* Re: [discuss] keep the commit history when adding a new
>>> engine version
>>>
>>> This is a good point. +1 for the proposal.
>>>
>>> On Thu, May 12, 2022 at 7:46 PM liwei li <hi...@gmail.com> wrote:
>>>
>>> Hi, guys
>>> When we want to add support for a new version of an engine, simply
>>> copying files from an old version to a new directory will cause git commit
>>> history to be lost, making it difficult to find file change records, we can
>>> only go to look for changes in the old path, but we don't know it in which
>>> one, and the old may have been deleted. Is there a better way to keep it?
>>> I recommend that we first rename the old version to the new one, and
>>> then make a new copy as the old version.
>>> For example, if we want to add Spark 3.3, we can do the following:
>>> 1. Change the path of version 3.2 from v3.2 to v3.3
>>> 2. Create a copy of v3.2 from v3.3
>>> 3. Modify the v3.2 file to make it work for spark 3.3 correct.
>>> What do you think of the above? Or is it necessary? Or if there is
>>> another better way?
>>> Thank you.
>>>
>>> Liwei Li
>>> hililiwei@gmail.com
>>>
>>>
Re: [discuss] keep the commit history when adding a new engine version
Posted by Rajarshi Sarkar <rs...@gmail.com>.
+1, good point.
Regards,
Rajarshi Sarkar
On Fri, May 13, 2022 at 9:03 AM Reo Lei <le...@gmail.com> wrote:
> That is great ! +1 for this.
>
> liwei li <hi...@gmail.com> 于2022年5月13日周五 11:12写道：
>
>> Correct a clerical errr:
>>
>> 3. Modify the v3.3 files to make it work for spark 3.3 correct.
>>
>> Liwei Li
>> ------------------------------
>> *From:* Steven Wu <st...@gmail.com>
>> *Sent:* Friday, May 13, 2022 11:06:26 AM
>> *To:* dev@iceberg.apache.org <de...@iceberg.apache.org>
>> *Subject:* Re: [discuss] keep the commit history when adding a new
>> engine version
>>
>> This is a good point. +1 for the proposal.
>>
>> On Thu, May 12, 2022 at 7:46 PM liwei li <hi...@gmail.com> wrote:
>>
>> Hi, guys
>> When we want to add support for a new version of an engine, simply
>> copying files from an old version to a new directory will cause git commit
>> history to be lost, making it difficult to find file change records, we can
>> only go to look for changes in the old path, but we don't know it in which
>> one, and the old may have been deleted. Is there a better way to keep it?
>> I recommend that we first rename the old version to the new one, and then
>> make a new copy as the old version.
>> For example, if we want to add Spark 3.3, we can do the following:
>> 1. Change the path of version 3.2 from v3.2 to v3.3
>> 2. Create a copy of v3.2 from v3.3
>> 3. Modify the v3.2 file to make it work for spark 3.3 correct.
>> What do you think of the above? Or is it necessary? Or if there is
>> another better way?
>> Thank you.
>>
>> Liwei Li
>> hililiwei@gmail.com
>>
>>
Re: [discuss] keep the commit history when adding a new engine version
Posted by Reo Lei <le...@gmail.com>.
That is great ! +1 for this.
liwei li <hi...@gmail.com> 于2022年5月13日周五 11:12写道：
> Correct a clerical errr:
>
> 3. Modify the v3.3 files to make it work for spark 3.3 correct.
>
> Liwei Li
> ------------------------------
> *From:* Steven Wu <st...@gmail.com>
> *Sent:* Friday, May 13, 2022 11:06:26 AM
> *To:* dev@iceberg.apache.org <de...@iceberg.apache.org>
> *Subject:* Re: [discuss] keep the commit history when adding a new engine
> version
>
> This is a good point. +1 for the proposal.
>
> On Thu, May 12, 2022 at 7:46 PM liwei li <hi...@gmail.com> wrote:
>
> Hi, guys
> When we want to add support for a new version of an engine, simply copying
> files from an old version to a new directory will cause git commit history
> to be lost, making it difficult to find file change records, we can only go
> to look for changes in the old path, but we don't know it in which one, and
> the old may have been deleted. Is there a better way to keep it?
> I recommend that we first rename the old version to the new one, and then
> make a new copy as the old version.
> For example, if we want to add Spark 3.3, we can do the following:
> 1. Change the path of version 3.2 from v3.2 to v3.3
> 2. Create a copy of v3.2 from v3.3
> 3. Modify the v3.2 file to make it work for spark 3.3 correct.
> What do you think of the above? Or is it necessary? Or if there is another
> better way?
> Thank you.
>
> Liwei Li
> hililiwei@gmail.com
>
>
Re: [discuss] keep the commit history when adding a new engine version
Posted by liwei li <hi...@gmail.com>.
Correct a clerical errr:
3. Modify the v3.3 files to make it work for spark 3.3 correct.
Liwei Li
________________________________
From: Steven Wu <st...@gmail.com>
Sent: Friday, May 13, 2022 11:06:26 AM
To: dev@iceberg.apache.org <de...@iceberg.apache.org>
Subject: Re: [discuss] keep the commit history when adding a new engine version
This is a good point. +1 for the proposal.
On Thu, May 12, 2022 at 7:46 PM liwei li <hi...@gmail.com>> wrote:
Hi, guys
When we want to add support for a new version of an engine, simply copying files from an old version to a new directory will cause git commit history to be lost, making it difficult to find file change records, we can only go to look for changes in the old path, but we don't know it in which one, and the old may have been deleted. Is there a better way to keep it?
I recommend that we first rename the old version to the new one, and then make a new copy as the old version.
For example, if we want to add Spark 3.3, we can do the following:
1. Change the path of version 3.2 from v3.2 to v3.3
2. Create a copy of v3.2 from v3.3
3. Modify the v3.2 file to make it work for spark 3.3 correct.
What do you think of the above? Or is it necessary? Or if there is another better way?
Thank you.
Liwei Li
hililiwei@gmail.com<ma...@gmail.com>
Re: [discuss] keep the commit history when adding a new engine version
Posted by Steven Wu <st...@gmail.com>.
This is a good point. +1 for the proposal.
On Thu, May 12, 2022 at 7:46 PM liwei li <hi...@gmail.com> wrote:
> Hi, guys
> When we want to add support for a new version of an engine, simply copying
> files from an old version to a new directory will cause git commit history
> to be lost, making it difficult to find file change records, we can only go
> to look for changes in the old path, but we don't know it in which one, and
> the old may have been deleted. Is there a better way to keep it?
> I recommend that we first rename the old version to the new one, and then
> make a new copy as the old version.
> For example, if we want to add Spark 3.3, we can do the following:
> 1. Change the path of version 3.2 from v3.2 to v3.3
> 2. Create a copy of v3.2 from v3.3
> 3. Modify the v3.2 file to make it work for spark 3.3 correct.
> What do you think of the above? Or is it necessary? Or if there is another
> better way?
> Thank you.
>
> Liwei Li
> hililiwei@gmail.com
>