You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Nikolay Izhikov <ni...@apache.org> on 2018/02/27 11:03:56 UTC
[TeamCity] DataFrame Examples
Hello, Igniters.
I've created several examples for a Ignite DataFrame.
Currently, they are not tested on Team City.
As far as I can understand, they should be checked in "Ignite Scalar Examples [Scala]".
But it marked as obsolete [1]
Please, give me advice:
1. Should we test Ignite DataFrame examples on Team City?
2. Which plan should be used for that?
[1] https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters
Re: [TeamCity] DataFrame Examples
Posted by Nikolay Izhikov <ni...@apache.org>.
Petr, thanks for help with Team City configuration issue!
В Пн, 19/03/2018 в 18:14 +0300, Nikolay Izhikov пишет:
> Yes.
>
> В Пн, 19/03/2018 в 18:13 +0300, Petr Ivanov пишет:
> > Have you detached template for build Ignite Examples?
> >
> >
> > > On 19 Mar 2018, at 18:08, Nikolay Izhikov <ni...@apache.org> wrote:
> > >
> > > Hello, guys.
> > >
> > > I merged to master my fix of tests. [1]
> > >
> > > I edited "Ignite Examples" configuration to Team City to enable testing of Spark examples.
> > >
> > > 1. `spark` profile enabled for test plan.
> > > 2. `IgniteExamplesSparkSelfTestSuite` added to tests suites list.
> > >
> > > Test results [2] are OK.
> > > Please, Team City gurus, validate my edits.
> > >
> > > [1] https://github.com/apache/ignite/commit/ce6a15782c93a95c58ccb46c44f391b39d4c4ac9
> > > [2] https://ci.ignite.apache.org/viewLog.html?buildId=1145334&buildTypeId=IgniteTests24Java8_IgniteExamples&tab=testsInfo
> > >
> > > В Пн, 19/03/2018 в 06:12 -0700, vveider пишет:
> > > > +1 to sticking to one suite for now.
> > > >
> > > > Maybe someday when examples' tests will surpass adequate build time, we'll
> > > > consider splitting them.
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: [TeamCity] DataFrame Examples
Posted by Nikolay Izhikov <ni...@apache.org>.
Yes.
В Пн, 19/03/2018 в 18:13 +0300, Petr Ivanov пишет:
> Have you detached template for build Ignite Examples?
>
>
> > On 19 Mar 2018, at 18:08, Nikolay Izhikov <ni...@apache.org> wrote:
> >
> > Hello, guys.
> >
> > I merged to master my fix of tests. [1]
> >
> > I edited "Ignite Examples" configuration to Team City to enable testing of Spark examples.
> >
> > 1. `spark` profile enabled for test plan.
> > 2. `IgniteExamplesSparkSelfTestSuite` added to tests suites list.
> >
> > Test results [2] are OK.
> > Please, Team City gurus, validate my edits.
> >
> > [1] https://github.com/apache/ignite/commit/ce6a15782c93a95c58ccb46c44f391b39d4c4ac9
> > [2] https://ci.ignite.apache.org/viewLog.html?buildId=1145334&buildTypeId=IgniteTests24Java8_IgniteExamples&tab=testsInfo
> >
> > В Пн, 19/03/2018 в 06:12 -0700, vveider пишет:
> > > +1 to sticking to one suite for now.
> > >
> > > Maybe someday when examples' tests will surpass adequate build time, we'll
> > > consider splitting them.
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>
>
Re: [TeamCity] DataFrame Examples
Posted by Petr Ivanov <mr...@gmail.com>.
Have you detached template for build Ignite Examples?
> On 19 Mar 2018, at 18:08, Nikolay Izhikov <ni...@apache.org> wrote:
>
> Hello, guys.
>
> I merged to master my fix of tests. [1]
>
> I edited "Ignite Examples" configuration to Team City to enable testing of Spark examples.
>
> 1. `spark` profile enabled for test plan.
> 2. `IgniteExamplesSparkSelfTestSuite` added to tests suites list.
>
> Test results [2] are OK.
> Please, Team City gurus, validate my edits.
>
> [1] https://github.com/apache/ignite/commit/ce6a15782c93a95c58ccb46c44f391b39d4c4ac9
> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1145334&buildTypeId=IgniteTests24Java8_IgniteExamples&tab=testsInfo
>
> В Пн, 19/03/2018 в 06:12 -0700, vveider пишет:
>> +1 to sticking to one suite for now.
>>
>> Maybe someday when examples' tests will surpass adequate build time, we'll
>> consider splitting them.
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: [TeamCity] DataFrame Examples
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, guys.
I merged to master my fix of tests. [1]
I edited "Ignite Examples" configuration to Team City to enable testing of Spark examples.
1. `spark` profile enabled for test plan.
2. `IgniteExamplesSparkSelfTestSuite` added to tests suites list.
Test results [2] are OK.
Please, Team City gurus, validate my edits.
[1] https://github.com/apache/ignite/commit/ce6a15782c93a95c58ccb46c44f391b39d4c4ac9
[2] https://ci.ignite.apache.org/viewLog.html?buildId=1145334&buildTypeId=IgniteTests24Java8_IgniteExamples&tab=testsInfo
В Пн, 19/03/2018 в 06:12 -0700, vveider пишет:
> +1 to sticking to one suite for now.
>
> Maybe someday when examples' tests will surpass adequate build time, we'll
> consider splitting them.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: [TeamCity] DataFrame Examples
Posted by vveider <mr...@gmail.com>.
+1 to sticking to one suite for now.
Maybe someday when examples' tests will surpass adequate build time, we'll
consider splitting them.
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: [TeamCity] DataFrame Examples
Posted by Dmitry Pavlov <dp...@gmail.com>.
Hi Igniters,
It seems to me it is better to keep one suite. It is reasonable to
separate suites if tests requires too much time to run.
Using the same suite will save remarkable amount of agent time.
Sincerely,
Dmitriy Pavlov
вс, 18 мар. 2018 г. в 19:34, Nikolay Izhikov <ni...@apache.org>:
> Hello, guys.
>
> I think we are ready to enable testing of Spark examples on Team City.
> Can you help me with best choice to do it?
>
> Currently, we have 'Ignite Examples' to run `IgniteExamplesSelfTestSuite`.
>
> We also have a `IgniteExamplesSparkSelfTestSuite` to test spark example.
> `IgniteExamplesSparkSelfTestSuite` exists in source only if `spark`
> profile enabled so we can't just add it to `IgniteExamplesSelfTestSuite`.
>
> I see following option here:
>
> 1. Edit 'Ignite Examples' to run both `IgniteExamplesSelfTestSuite` and
> `IgniteExamplesSparkSelfTestSuite`.
> 2. Create new test plan to run `IgniteExamplesSparkSelfTestSuite`.
>
> Thoughts?
>
> P.S. We also need minor changes [1] to make sure Spark examples are
> running correctly.
>
> [1] https://github.com/apache/ignite/pull/3590
>
> В Вт, 27/02/2018 в 11:56 -0800, Valentin Kulichenko пишет:
> > If we can run Scala examples there, then yes.
> >
> > -Val
> >
> > On Tue, Feb 27, 2018 at 11:54 AM, Dmitry Pavlov <dp...@gmail.com>
> > wrote:
> >
> > > Nikolay, Val,
> > >
> > > Could these examples be tested in standard Ignite Examples test suite?
> As
> > > far as I know Ignite ML and Ignite IGFS are also tested there.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> > > valentin.kulichenko@gmail.com>:
> > >
> > > > Nikolay,
> > > >
> > > > Spark integration is not related to Scalar, the only thing they have
> in
> > > > common is Scala. I think we should have a separate configuration for
> > > > ignite-spark module. If anything Spark related is currently in Scalar
> > > > suite, it should be moved from there.
> > > >
> > > > -Val
> > > >
> > > > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov <
> nizhikov@apache.org>
> > > > wrote:
> > > >
> > > > > Hello, Igniters.
> > > > >
> > > > > I've created several examples for a Ignite DataFrame.
> > > > > Currently, they are not tested on Team City.
> > > > >
> > > > > As far as I can understand, they should be checked in "Ignite
> Scalar
> > > > > Examples [Scala]".
> > > > > But it marked as obsolete [1]
> > > > >
> > > > > Please, give me advice:
> > > > >
> > > > > 1. Should we test Ignite DataFrame examples on Team City?
> > > > >
> > > > > 2. Which plan should be used for that?
> > > > >
> > > > > [1]
> > > >
> > > >
> https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=
> > > > > IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters
Re: [TeamCity] DataFrame Examples
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, guys.
I think we are ready to enable testing of Spark examples on Team City.
Can you help me with best choice to do it?
Currently, we have 'Ignite Examples' to run `IgniteExamplesSelfTestSuite`.
We also have a `IgniteExamplesSparkSelfTestSuite` to test spark example.
`IgniteExamplesSparkSelfTestSuite` exists in source only if `spark` profile enabled so we can't just add it to `IgniteExamplesSelfTestSuite`.
I see following option here:
1. Edit 'Ignite Examples' to run both `IgniteExamplesSelfTestSuite` and `IgniteExamplesSparkSelfTestSuite`.
2. Create new test plan to run `IgniteExamplesSparkSelfTestSuite`.
Thoughts?
P.S. We also need minor changes [1] to make sure Spark examples are running correctly.
[1] https://github.com/apache/ignite/pull/3590
В Вт, 27/02/2018 в 11:56 -0800, Valentin Kulichenko пишет:
> If we can run Scala examples there, then yes.
>
> -Val
>
> On Tue, Feb 27, 2018 at 11:54 AM, Dmitry Pavlov <dp...@gmail.com>
> wrote:
>
> > Nikolay, Val,
> >
> > Could these examples be tested in standard Ignite Examples test suite? As
> > far as I know Ignite ML and Ignite IGFS are also tested there.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> > valentin.kulichenko@gmail.com>:
> >
> > > Nikolay,
> > >
> > > Spark integration is not related to Scalar, the only thing they have in
> > > common is Scala. I think we should have a separate configuration for
> > > ignite-spark module. If anything Spark related is currently in Scalar
> > > suite, it should be moved from there.
> > >
> > > -Val
> > >
> > > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov <ni...@apache.org>
> > > wrote:
> > >
> > > > Hello, Igniters.
> > > >
> > > > I've created several examples for a Ignite DataFrame.
> > > > Currently, they are not tested on Team City.
> > > >
> > > > As far as I can understand, they should be checked in "Ignite Scalar
> > > > Examples [Scala]".
> > > > But it marked as obsolete [1]
> > > >
> > > > Please, give me advice:
> > > >
> > > > 1. Should we test Ignite DataFrame examples on Team City?
> > > >
> > > > 2. Which plan should be used for that?
> > > >
> > > > [1]
> > >
> > > https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=
> > > > IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters
Re: [TeamCity] DataFrame Examples
Posted by Valentin Kulichenko <va...@gmail.com>.
If we can run Scala examples there, then yes.
-Val
On Tue, Feb 27, 2018 at 11:54 AM, Dmitry Pavlov <dp...@gmail.com>
wrote:
> Nikolay, Val,
>
> Could these examples be tested in standard Ignite Examples test suite? As
> far as I know Ignite ML and Ignite IGFS are also tested there.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
> > Nikolay,
> >
> > Spark integration is not related to Scalar, the only thing they have in
> > common is Scala. I think we should have a separate configuration for
> > ignite-spark module. If anything Spark related is currently in Scalar
> > suite, it should be moved from there.
> >
> > -Val
> >
> > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov <ni...@apache.org>
> > wrote:
> >
> > > Hello, Igniters.
> > >
> > > I've created several examples for a Ignite DataFrame.
> > > Currently, they are not tested on Team City.
> > >
> > > As far as I can understand, they should be checked in "Ignite Scalar
> > > Examples [Scala]".
> > > But it marked as obsolete [1]
> > >
> > > Please, give me advice:
> > >
> > > 1. Should we test Ignite DataFrame examples on Team City?
> > >
> > > 2. Which plan should be used for that?
> > >
> > > [1]
> > https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=
> > > IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters
> >
>
Re: [TeamCity] DataFrame Examples
Posted by Dmitry Pavlov <dp...@gmail.com>.
I prefer to use existing profile. We have a lot of suites, and it is hard
to manage and monitor.
вт, 27 февр. 2018 г. в 23:02, Nikolay Izhikov <ni...@apache.org>:
> I've tested it locally so yes, Spark examples can be tested in Ignite
> Examples.
>
> We need to enable `scala` profile for test plan and add
> `IgniteExamplesSparkSelfTestSuite` to Team City properties.
>
> Dmitry, please, give me advice.
> Should I edit Ignite Examples settings or should I create copy of test
> plan to adjust settings?
>
>
>
> В Вт, 27/02/2018 в 19:54 +0000, Dmitry Pavlov пишет:
> > Nikolay, Val,
> >
> > Could these examples be tested in standard Ignite Examples test suite? As
> > far as I know Ignite ML and Ignite IGFS are also tested there.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> > valentin.kulichenko@gmail.com>:
> >
> > > Nikolay,
> > >
> > > Spark integration is not related to Scalar, the only thing they have in
> > > common is Scala. I think we should have a separate configuration for
> > > ignite-spark module. If anything Spark related is currently in Scalar
> > > suite, it should be moved from there.
> > >
> > > -Val
> > >
> > > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov <ni...@apache.org>
> > > wrote:
> > >
> > > > Hello, Igniters.
> > > >
> > > > I've created several examples for a Ignite DataFrame.
> > > > Currently, they are not tested on Team City.
> > > >
> > > > As far as I can understand, they should be checked in "Ignite Scalar
> > > > Examples [Scala]".
> > > > But it marked as obsolete [1]
> > > >
> > > > Please, give me advice:
> > > >
> > > > 1. Should we test Ignite DataFrame examples on Team City?
> > > >
> > > > 2. Which plan should be used for that?
> > > >
> > > > [1]
> > >
> > > https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=
> > > > IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters
Re: [TeamCity] DataFrame Examples
Posted by Nikolay Izhikov <ni...@apache.org>.
I've tested it locally so yes, Spark examples can be tested in Ignite Examples.
We need to enable `scala` profile for test plan and add `IgniteExamplesSparkSelfTestSuite` to Team City properties.
Dmitry, please, give me advice.
Should I edit Ignite Examples settings or should I create copy of test plan to adjust settings?
В Вт, 27/02/2018 в 19:54 +0000, Dmitry Pavlov пишет:
> Nikolay, Val,
>
> Could these examples be tested in standard Ignite Examples test suite? As
> far as I know Ignite ML and Ignite IGFS are also tested there.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
> > Nikolay,
> >
> > Spark integration is not related to Scalar, the only thing they have in
> > common is Scala. I think we should have a separate configuration for
> > ignite-spark module. If anything Spark related is currently in Scalar
> > suite, it should be moved from there.
> >
> > -Val
> >
> > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov <ni...@apache.org>
> > wrote:
> >
> > > Hello, Igniters.
> > >
> > > I've created several examples for a Ignite DataFrame.
> > > Currently, they are not tested on Team City.
> > >
> > > As far as I can understand, they should be checked in "Ignite Scalar
> > > Examples [Scala]".
> > > But it marked as obsolete [1]
> > >
> > > Please, give me advice:
> > >
> > > 1. Should we test Ignite DataFrame examples on Team City?
> > >
> > > 2. Which plan should be used for that?
> > >
> > > [1]
> >
> > https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=
> > > IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters
Re: [TeamCity] DataFrame Examples
Posted by Dmitry Pavlov <dp...@gmail.com>.
Nikolay, Val,
Could these examples be tested in standard Ignite Examples test suite? As
far as I know Ignite ML and Ignite IGFS are also tested there.
Sincerely,
Dmitriy Pavlov
вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
valentin.kulichenko@gmail.com>:
> Nikolay,
>
> Spark integration is not related to Scalar, the only thing they have in
> common is Scala. I think we should have a separate configuration for
> ignite-spark module. If anything Spark related is currently in Scalar
> suite, it should be moved from there.
>
> -Val
>
> On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov <ni...@apache.org>
> wrote:
>
> > Hello, Igniters.
> >
> > I've created several examples for a Ignite DataFrame.
> > Currently, they are not tested on Team City.
> >
> > As far as I can understand, they should be checked in "Ignite Scalar
> > Examples [Scala]".
> > But it marked as obsolete [1]
> >
> > Please, give me advice:
> >
> > 1. Should we test Ignite DataFrame examples on Team City?
> >
> > 2. Which plan should be used for that?
> >
> > [1]
> https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=
> > IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters
>
Re: [TeamCity] DataFrame Examples
Posted by Valentin Kulichenko <va...@gmail.com>.
Nikolay,
Spark integration is not related to Scalar, the only thing they have in
common is Scala. I think we should have a separate configuration for
ignite-spark module. If anything Spark related is currently in Scalar
suite, it should be moved from there.
-Val
On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov <ni...@apache.org>
wrote:
> Hello, Igniters.
>
> I've created several examples for a Ignite DataFrame.
> Currently, they are not tested on Team City.
>
> As far as I can understand, they should be checked in "Ignite Scalar
> Examples [Scala]".
> But it marked as obsolete [1]
>
> Please, give me advice:
>
> 1. Should we test Ignite DataFrame examples on Team City?
>
> 2. Which plan should be used for that?
>
> [1] https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=
> IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters