You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Dmitry Pavlov <dp...@apache.org> on 2021/08/16 16:36:17 UTC
Storing Teamcity projects settings in Version Control
Hi Igniters,
Once there was a discussion about placing our build configurations (TC) settings in a VSC repository. This idea was suggested because we wanted to validate internals of configs using IDE/text editor.
https://www.jetbrains.com/help/teamcity/storing-project-settings-in-version-control.html
Now, with introduction of a second instance this idea migth be useful twice, we'll be able to
- view, track history, track authorship (? I'm not 100% sure, it is to be checked)
- use VSC as a signle source of thurth
- we can remove TC Bot's code for validating all configs using REST
How does that sound to you? Is it better to look at Kotlin DSL? Or could XML be enought?
Sincerely,
Dmitriy Pavlov
Re: Storing Teamcity projects settings in Version Control
Posted by Ivan Daschinsky <iv...@gmail.com>.
>> Btw, I was always against moving thin clients into separate repos.
Monorepo
Monorepo is a mostly cargo cult. What is suitable for google is not
suitable for others.
For example, we have already released new versions of pyignite with many
new features.
When this client was in our super repo, it was not released and published
for years.
вт, 17 авг. 2021 г. в 18:32, Pavel Tupitsyn <pt...@apache.org>:
> Ivan,
>
> 1. It will be clean. It will actually be better: good to know when build
> failure is caused by a config change, right?
>
> 2. Can you please provide an example in a well-known open-source project
> other than Ignite?
>
> Btw, I was always against moving thin clients into separate repos. Monorepo
> approach in some large companies is used for a reason.
> Dealing with multiple repositories is always hard.
>
>
> On Tue, Aug 17, 2021 at 6:23 PM Ivan Daschinsky <iv...@gmail.com>
> wrote:
>
> > 1. Clean commit history (As one developer said, git history is an api)
> > 2. We have separate thin client repos -- but TC thin client build depends
> > on ignite build also.
> >
> >
> > вт, 17 авг. 2021 г. в 18:08, Pavel Tupitsyn <pt...@apache.org>:
> >
> > > Ivan,
> > >
> > > > I'm sorry, but what about storing TC configs in separate repo?
> > > What are the pros of this approach? What do we gain?
> > > Separate repo always adds friction, and it is not clear how to handle
> > > config changes that are tied to code changes.
> > >
> > > > It is quite common approach.
> > > Can you provide an example of an open-source project with this
> approach?
> > >
> > > On Tue, Aug 17, 2021 at 6:05 PM Ivan Daschinsky <iv...@gmail.com>
> > > wrote:
> > >
> > > > I'm sorry, but what about storing TC configs in separate repo?
> > > > It is quite common approach.
> > > >
> > > > вт, 17 авг. 2021 г. в 17:33, Pavel Tupitsyn <pt...@apache.org>:
> > > >
> > > > > Anton,
> > > > >
> > > > > > This will kill repo history.
> > > > > > You'll see dozens of TC config updates vs a single Ignite fix
> > > > >
> > > > > Not really.
> > > > > I'm not suggesting something crazy, this is the modern way to do
> > CI/CD
> > > > > - see GitHub actions, Azure pipelines, etc - you write a config and
> > > store
> > > > > it in Git.
> > > > >
> > > > > > Where are you going to apply configs, do you have your own TC? ;)
> > > > >
> > > > > Maybe I do. That's the point, no matter how many TCs we have, all
> of
> > > them
> > > > > will use the same configs from the repo.
> > > > >
> > > > > On Tue, Aug 17, 2021 at 5:07 PM Petr Ivanov <mr...@gmail.com>
> > > wrote:
> > > > >
> > > > > > After initial setup, there won't be lot's of changes, at least
> for
> > > PRs
> > > > > > there will be single commit with both fix and TC changes.
> > > > > >
> > > > > >
> > > > > > > On 17 Aug 2021, at 13:05, Anton Vinogradov <av...@apache.org>
> > wrote:
> > > > > > >
> > > > > > > This will kill repo history.
> > > > > > > You'll see dozens of TC config updates vs a single Ignite fix
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Sincerely yours, Ivan Daschinskiy
> > > >
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>
--
Sincerely yours, Ivan Daschinskiy
Re: Storing Teamcity projects settings in Version Control
Posted by Pavel Tupitsyn <pt...@apache.org>.
Ivan,
1. It will be clean. It will actually be better: good to know when build
failure is caused by a config change, right?
2. Can you please provide an example in a well-known open-source project
other than Ignite?
Btw, I was always against moving thin clients into separate repos. Monorepo
approach in some large companies is used for a reason.
Dealing with multiple repositories is always hard.
On Tue, Aug 17, 2021 at 6:23 PM Ivan Daschinsky <iv...@gmail.com> wrote:
> 1. Clean commit history (As one developer said, git history is an api)
> 2. We have separate thin client repos -- but TC thin client build depends
> on ignite build also.
>
>
> вт, 17 авг. 2021 г. в 18:08, Pavel Tupitsyn <pt...@apache.org>:
>
> > Ivan,
> >
> > > I'm sorry, but what about storing TC configs in separate repo?
> > What are the pros of this approach? What do we gain?
> > Separate repo always adds friction, and it is not clear how to handle
> > config changes that are tied to code changes.
> >
> > > It is quite common approach.
> > Can you provide an example of an open-source project with this approach?
> >
> > On Tue, Aug 17, 2021 at 6:05 PM Ivan Daschinsky <iv...@gmail.com>
> > wrote:
> >
> > > I'm sorry, but what about storing TC configs in separate repo?
> > > It is quite common approach.
> > >
> > > вт, 17 авг. 2021 г. в 17:33, Pavel Tupitsyn <pt...@apache.org>:
> > >
> > > > Anton,
> > > >
> > > > > This will kill repo history.
> > > > > You'll see dozens of TC config updates vs a single Ignite fix
> > > >
> > > > Not really.
> > > > I'm not suggesting something crazy, this is the modern way to do
> CI/CD
> > > > - see GitHub actions, Azure pipelines, etc - you write a config and
> > store
> > > > it in Git.
> > > >
> > > > > Where are you going to apply configs, do you have your own TC? ;)
> > > >
> > > > Maybe I do. That's the point, no matter how many TCs we have, all of
> > them
> > > > will use the same configs from the repo.
> > > >
> > > > On Tue, Aug 17, 2021 at 5:07 PM Petr Ivanov <mr...@gmail.com>
> > wrote:
> > > >
> > > > > After initial setup, there won't be lot's of changes, at least for
> > PRs
> > > > > there will be single commit with both fix and TC changes.
> > > > >
> > > > >
> > > > > > On 17 Aug 2021, at 13:05, Anton Vinogradov <av...@apache.org>
> wrote:
> > > > > >
> > > > > > This will kill repo history.
> > > > > > You'll see dozens of TC config updates vs a single Ignite fix
> > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Sincerely yours, Ivan Daschinskiy
> > >
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>
Re: Storing Teamcity projects settings in Version Control
Posted by Ivan Daschinsky <iv...@gmail.com>.
1. Clean commit history (As one developer said, git history is an api)
2. We have separate thin client repos -- but TC thin client build depends
on ignite build also.
вт, 17 авг. 2021 г. в 18:08, Pavel Tupitsyn <pt...@apache.org>:
> Ivan,
>
> > I'm sorry, but what about storing TC configs in separate repo?
> What are the pros of this approach? What do we gain?
> Separate repo always adds friction, and it is not clear how to handle
> config changes that are tied to code changes.
>
> > It is quite common approach.
> Can you provide an example of an open-source project with this approach?
>
> On Tue, Aug 17, 2021 at 6:05 PM Ivan Daschinsky <iv...@gmail.com>
> wrote:
>
> > I'm sorry, but what about storing TC configs in separate repo?
> > It is quite common approach.
> >
> > вт, 17 авг. 2021 г. в 17:33, Pavel Tupitsyn <pt...@apache.org>:
> >
> > > Anton,
> > >
> > > > This will kill repo history.
> > > > You'll see dozens of TC config updates vs a single Ignite fix
> > >
> > > Not really.
> > > I'm not suggesting something crazy, this is the modern way to do CI/CD
> > > - see GitHub actions, Azure pipelines, etc - you write a config and
> store
> > > it in Git.
> > >
> > > > Where are you going to apply configs, do you have your own TC? ;)
> > >
> > > Maybe I do. That's the point, no matter how many TCs we have, all of
> them
> > > will use the same configs from the repo.
> > >
> > > On Tue, Aug 17, 2021 at 5:07 PM Petr Ivanov <mr...@gmail.com>
> wrote:
> > >
> > > > After initial setup, there won't be lot's of changes, at least for
> PRs
> > > > there will be single commit with both fix and TC changes.
> > > >
> > > >
> > > > > On 17 Aug 2021, at 13:05, Anton Vinogradov <av...@apache.org> wrote:
> > > > >
> > > > > This will kill repo history.
> > > > > You'll see dozens of TC config updates vs a single Ignite fix
> > > >
> > > >
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>
--
Sincerely yours, Ivan Daschinskiy
Re: Storing Teamcity projects settings in Version Control
Posted by Pavel Tupitsyn <pt...@apache.org>.
Ivan,
> I'm sorry, but what about storing TC configs in separate repo?
What are the pros of this approach? What do we gain?
Separate repo always adds friction, and it is not clear how to handle
config changes that are tied to code changes.
> It is quite common approach.
Can you provide an example of an open-source project with this approach?
On Tue, Aug 17, 2021 at 6:05 PM Ivan Daschinsky <iv...@gmail.com> wrote:
> I'm sorry, but what about storing TC configs in separate repo?
> It is quite common approach.
>
> вт, 17 авг. 2021 г. в 17:33, Pavel Tupitsyn <pt...@apache.org>:
>
> > Anton,
> >
> > > This will kill repo history.
> > > You'll see dozens of TC config updates vs a single Ignite fix
> >
> > Not really.
> > I'm not suggesting something crazy, this is the modern way to do CI/CD
> > - see GitHub actions, Azure pipelines, etc - you write a config and store
> > it in Git.
> >
> > > Where are you going to apply configs, do you have your own TC? ;)
> >
> > Maybe I do. That's the point, no matter how many TCs we have, all of them
> > will use the same configs from the repo.
> >
> > On Tue, Aug 17, 2021 at 5:07 PM Petr Ivanov <mr...@gmail.com> wrote:
> >
> > > After initial setup, there won't be lot's of changes, at least for PRs
> > > there will be single commit with both fix and TC changes.
> > >
> > >
> > > > On 17 Aug 2021, at 13:05, Anton Vinogradov <av...@apache.org> wrote:
> > > >
> > > > This will kill repo history.
> > > > You'll see dozens of TC config updates vs a single Ignite fix
> > >
> > >
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>
Re: Storing Teamcity projects settings in Version Control
Posted by Ivan Daschinsky <iv...@gmail.com>.
I'm sorry, but what about storing TC configs in separate repo?
It is quite common approach.
вт, 17 авг. 2021 г. в 17:33, Pavel Tupitsyn <pt...@apache.org>:
> Anton,
>
> > This will kill repo history.
> > You'll see dozens of TC config updates vs a single Ignite fix
>
> Not really.
> I'm not suggesting something crazy, this is the modern way to do CI/CD
> - see GitHub actions, Azure pipelines, etc - you write a config and store
> it in Git.
>
> > Where are you going to apply configs, do you have your own TC? ;)
>
> Maybe I do. That's the point, no matter how many TCs we have, all of them
> will use the same configs from the repo.
>
> On Tue, Aug 17, 2021 at 5:07 PM Petr Ivanov <mr...@gmail.com> wrote:
>
> > After initial setup, there won't be lot's of changes, at least for PRs
> > there will be single commit with both fix and TC changes.
> >
> >
> > > On 17 Aug 2021, at 13:05, Anton Vinogradov <av...@apache.org> wrote:
> > >
> > > This will kill repo history.
> > > You'll see dozens of TC config updates vs a single Ignite fix
> >
> >
>
--
Sincerely yours, Ivan Daschinskiy
Re: Storing Teamcity projects settings in Version Control
Posted by Pavel Tupitsyn <pt...@apache.org>.
Anton,
> This will kill repo history.
> You'll see dozens of TC config updates vs a single Ignite fix
Not really.
I'm not suggesting something crazy, this is the modern way to do CI/CD
- see GitHub actions, Azure pipelines, etc - you write a config and store
it in Git.
> Where are you going to apply configs, do you have your own TC? ;)
Maybe I do. That's the point, no matter how many TCs we have, all of them
will use the same configs from the repo.
On Tue, Aug 17, 2021 at 5:07 PM Petr Ivanov <mr...@gmail.com> wrote:
> After initial setup, there won't be lot's of changes, at least for PRs
> there will be single commit with both fix and TC changes.
>
>
> > On 17 Aug 2021, at 13:05, Anton Vinogradov <av...@apache.org> wrote:
> >
> > This will kill repo history.
> > You'll see dozens of TC config updates vs a single Ignite fix
>
>
Re: Storing Teamcity projects settings in Version Control
Posted by Petr Ivanov <mr...@gmail.com>.
After initial setup, there won't be lot's of changes, at least for PRs there will be single commit with both fix and TC changes.
> On 17 Aug 2021, at 13:05, Anton Vinogradov <av...@apache.org> wrote:
>
> This will kill repo history.
> You'll see dozens of TC config updates vs a single Ignite fix
Re: Storing Teamcity projects settings in Version Control
Posted by Anton Vinogradov <av...@apache.org>.
> I think TC config should be stored in the same repo as the corresponding
> code (2.x config in 2.x repo, 3.x in 3.x, etc).
This will kill repo history.
You'll see dozens of TC config updates vs a single Ignite fix
> it would be great to be able to test them by simply creating a new branch
> in a single repo.
Where are you going to apply configs, do you have your own TC? ;)
On Tue, Aug 17, 2021 at 10:27 AM Pavel Tupitsyn <pt...@apache.org>
wrote:
> Dmitry, Petr,
>
> I think TC config should be stored in the same repo as the corresponding
> code (2.x config in 2.x repo, 3.x in 3.x, etc).
>
> Changes and updates to build scripts and project structure often come
> together with changes to TC configuration,
> it would be great to be able to test them by simply creating a new branch
> in a single repo.
>
> On Mon, Aug 16, 2021 at 10:17 PM Petr Ivanov <mr...@gmail.com> wrote:
>
> > Hi, Dmitry.
> >
> >
> > I think we should start with adding current projects as-is in form of
> > autogenerated Kotlin code, but continue to use UI for editing.
> > Later at some point we should replace autogenerated code with valid one
> > (this can be done configuration by configuration), that will allow use
> the
> > power of DSL and programming language to create dynamic configuration and
> > use other benefits.
> >
> > Also, while I was thinking about it, I wondered — should we add whole TC
> > config to single repository, or divide per project (AI 2.x, AI 3.x,
> > Extensions, Thin Clients, etc.) and add corresponding TeamCity
> > configuration as part of the project into the same repository.
> > TeamCity configuration is added in form of maven project in .teamcity
> > directory in the project's root and will not interfere with the project
> > itself. Also this scheme is preferable due to linear development cycle
> (no
> > parallel versions in development for each project).
> >
> > I think I am ready to drive this initiative when and if we agree on this
> > matter and will discuss all the details of such migration.
> >
> >
> > > On 16 Aug 2021, at 19:36, Dmitry Pavlov <dp...@apache.org> wrote:
> > >
> > > Hi Igniters,
> > >
> > > Once there was a discussion about placing our build configurations (TC)
> > settings in a VSC repository. This idea was suggested because we wanted
> to
> > validate internals of configs using IDE/text editor.
> > >
> > >
> >
> https://www.jetbrains.com/help/teamcity/storing-project-settings-in-version-control.html
> > >
> > > Now, with introduction of a second instance this idea migth be useful
> > twice, we'll be able to
> > > - view, track history, track authorship (? I'm not 100% sure, it is to
> > be checked)
> > > - use VSC as a signle source of thurth
> > > - we can remove TC Bot's code for validating all configs using REST
> > >
> > > How does that sound to you? Is it better to look at Kotlin DSL? Or
> could
> > XML be enought?
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> >
> >
>
Re: Storing Teamcity projects settings in Version Control
Posted by Petr Ivanov <mr...@gmail.com>.
Unfortunately, it won't work. At least in the nearest future.
Currently, you cannot see project structure on TC in any branch except default (master/main).
And some things like snapshot dependencies are ONLY taken from default branch.
> On 17 Aug 2021, at 10:25, Pavel Tupitsyn <pt...@apache.org> wrote:
>
> Changes and updates to build scripts and project structure often come
> together with changes to TC configuration,
> it would be great to be able to test them by simply creating a new branch
> in a single repo.
Re: Storing Teamcity projects settings in Version Control
Posted by Pavel Tupitsyn <pt...@apache.org>.
Dmitry, Petr,
I think TC config should be stored in the same repo as the corresponding
code (2.x config in 2.x repo, 3.x in 3.x, etc).
Changes and updates to build scripts and project structure often come
together with changes to TC configuration,
it would be great to be able to test them by simply creating a new branch
in a single repo.
On Mon, Aug 16, 2021 at 10:17 PM Petr Ivanov <mr...@gmail.com> wrote:
> Hi, Dmitry.
>
>
> I think we should start with adding current projects as-is in form of
> autogenerated Kotlin code, but continue to use UI for editing.
> Later at some point we should replace autogenerated code with valid one
> (this can be done configuration by configuration), that will allow use the
> power of DSL and programming language to create dynamic configuration and
> use other benefits.
>
> Also, while I was thinking about it, I wondered — should we add whole TC
> config to single repository, or divide per project (AI 2.x, AI 3.x,
> Extensions, Thin Clients, etc.) and add corresponding TeamCity
> configuration as part of the project into the same repository.
> TeamCity configuration is added in form of maven project in .teamcity
> directory in the project's root and will not interfere with the project
> itself. Also this scheme is preferable due to linear development cycle (no
> parallel versions in development for each project).
>
> I think I am ready to drive this initiative when and if we agree on this
> matter and will discuss all the details of such migration.
>
>
> > On 16 Aug 2021, at 19:36, Dmitry Pavlov <dp...@apache.org> wrote:
> >
> > Hi Igniters,
> >
> > Once there was a discussion about placing our build configurations (TC)
> settings in a VSC repository. This idea was suggested because we wanted to
> validate internals of configs using IDE/text editor.
> >
> >
> https://www.jetbrains.com/help/teamcity/storing-project-settings-in-version-control.html
> >
> > Now, with introduction of a second instance this idea migth be useful
> twice, we'll be able to
> > - view, track history, track authorship (? I'm not 100% sure, it is to
> be checked)
> > - use VSC as a signle source of thurth
> > - we can remove TC Bot's code for validating all configs using REST
> >
> > How does that sound to you? Is it better to look at Kotlin DSL? Or could
> XML be enought?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
>
>
Re: Storing Teamcity projects settings in Version Control
Posted by Petr Ivanov <mr...@gmail.com>.
Hi, Dmitry.
I think we should start with adding current projects as-is in form of autogenerated Kotlin code, but continue to use UI for editing.
Later at some point we should replace autogenerated code with valid one (this can be done configuration by configuration), that will allow use the power of DSL and programming language to create dynamic configuration and use other benefits.
Also, while I was thinking about it, I wondered — should we add whole TC config to single repository, or divide per project (AI 2.x, AI 3.x, Extensions, Thin Clients, etc.) and add corresponding TeamCity configuration as part of the project into the same repository.
TeamCity configuration is added in form of maven project in .teamcity directory in the project's root and will not interfere with the project itself. Also this scheme is preferable due to linear development cycle (no parallel versions in development for each project).
I think I am ready to drive this initiative when and if we agree on this matter and will discuss all the details of such migration.
> On 16 Aug 2021, at 19:36, Dmitry Pavlov <dp...@apache.org> wrote:
>
> Hi Igniters,
>
> Once there was a discussion about placing our build configurations (TC) settings in a VSC repository. This idea was suggested because we wanted to validate internals of configs using IDE/text editor.
>
> https://www.jetbrains.com/help/teamcity/storing-project-settings-in-version-control.html
>
> Now, with introduction of a second instance this idea migth be useful twice, we'll be able to
> - view, track history, track authorship (? I'm not 100% sure, it is to be checked)
> - use VSC as a signle source of thurth
> - we can remove TC Bot's code for validating all configs using REST
>
> How does that sound to you? Is it better to look at Kotlin DSL? Or could XML be enought?
>
> Sincerely,
> Dmitriy Pavlov
>